Bochs icache perf when writing memory
"Lander Brandt" <[email protected]> Sat, 01 Aug 2020 19:00:48 -0700
| Newsgroups | gmane.comp.emulators.bochs.devel |
|---|---|
| Message-ID | <[email protected]> |
Hello,
I was recently doing some profiling of Bochs and noticed a lot of time in my build spent in the instruction decoder caused by icache misses. After reading the code intimately for a few days I *think* I have a pretty decent understanding of how it works and may have fixed some performance issues, but I would like to present my changes and see if what I changed is "legal".
In memory.cc I noticed that I was hitting the memcpy-type case in the BX_MEM_C::{read,write}PhysicalPage methods and in every case I documented it was a read/write of 16 bytes. For writes this seemed particularly bad because it would call `pageWriteStampTable.decWriteStamp(a20addr);`, which if my understanding of the icache is correct may cause an entire page to be flushed from the icache for even small writes. I "fixed" this (it still has unnecessary work when the length is not a multiple of 8) and a similar condition in `BX_CPU_C::v2h_write_byte`.
Anecdotally I noticed a very good performance increase by just eyeballing the IPS counter in the bottom-left of the GUI and the number of calls to `bxICache_c::handleSMC` with an 0xFFFF_FFFF mask dropped from 16,125,122 for ~1BN instructions to just 34. Again if my understanding of the icache is correct (and please explain it to me if my understanding is fundamentally broken), this resulted in a substantially fewer whole pages being flushed from the icache if they were previously mapped in. I also did not observe any *visible* negative impacts of this patch and my snapshot appeared to operate normally.
Thanks!
Here is my patch:
diff --git a/bochs_src/cpu/access.cc b/bochs_src/cpu/access.cc
index da1148ad..4a72f5ce 100644
--- a/bochs_src/cpu/access.cc
+++ b/bochs_src/cpu/access.cc
@@ -513,7 +513,7 @@ BX_CPU_C::v2h_write_byte(bx_address laddr, bx_bool user)
bx_hostpageaddr_t hostPageAddr = tlbEntry->hostPageAddr;
Bit32u pageOffset = PAGE_OFFSET(laddr);
Bit8u *hostAddr = (Bit8u*) (hostPageAddr | pageOffset);
- pageWriteStampTable.decWriteStamp(tlbEntry->ppf);
+ pageWriteStampTable.decWriteStamp(tlbEntry->ppf, 1);
return hostAddr;
}
}
diff --git a/bochs_src/memory/memory.cc b/bochs_src/memory/memory.cc
index e896f1d1..d3fa737c 100644
--- a/bochs_src/memory/memory.cc
+++ b/bochs_src/memory/memory.cc
@@ -110,8 +110,6 @@ mem_write:
// len == other, just fall thru to special cases handling
}
- pageWriteStampTable.decWriteStamp(a20addr);
-
#ifdef BX_LITTLE_ENDIAN
data_ptr = (Bit8u *) data;
#else // BX_BIG_ENDIAN
@@ -122,18 +120,38 @@ mem_write:
{
// addr *not* in range 000A0000 .. 000FFFFF
while(1) {
- *(BX_MEM_THIS get_vector(a20addr)) = *data_ptr;
- if (len == 1) return;
- len--;
- a20addr++;
-#ifdef BX_LITTLE_ENDIAN
- data_ptr++;
-#else // BX_BIG_ENDIAN
- data_ptr--;
-#endif
+ // Read in chunks of 8 bytes if we can
+ if (len % 8 == 0) {
+ pageWriteStampTable.decWriteStamp(a20addr, 8);
+ WriteHostQWordToLittleEndian(BX_MEM_THIS get_vector(a20addr), *(Bit64u*)data_ptr);
+ len -= 8;
+ a20addr += 8;
+ #ifdef BX_LITTLE_ENDIAN
+ data_ptr += 8;
+ #else
+ data_ptr -= 8;
+ #endif
+
+ if (len == 0) return;
+ } else {
+ pageWriteStampTable.decWriteStamp(a20addr, 1);
+ *(BX_MEM_THIS get_vector(a20addr)) = *data_ptr;
+ if (len == 1) return;
+ len--;
+ a20addr++;
+ #ifdef BX_LITTLE_ENDIAN
+ data_ptr++;
+ #else // BX_BIG_ENDIAN
+ data_ptr--;
+ #endif
+ }
}
}
+
+ printf("LENGTH NOT HANDLED!!! %d\n", len);
+ pageWriteStampTable.decWriteStamp(a20addr);
+
// addr must be in range 000A0000 .. 000FFFFF
for(unsigned i=0; i<len; i++) {
@@ -273,15 +291,29 @@ mem_read:
{
// addr *not* in range 000A0000 .. 000FFFFF
while(1) {
- *data_ptr = *(BX_MEM_THIS get_vector(a20addr));
- if (len == 1) return;
- len--;
- a20addr++;
-#ifdef BX_LITTLE_ENDIAN
- data_ptr++;
-#else // BX_BIG_ENDIAN
- data_ptr--;
-#endif
+ // Read in chunks of 8 bytes if we can
+ if (len % 8 == 0) {
+ ReadHostQWordFromLittleEndian(BX_MEM_THIS get_vector(a20addr), *(Bit64u*)data_ptr);
+ len -= 8;
+ a20addr += 8;
+ #ifdef BX_LITTLE_ENDIAN
+ data_ptr += 8;
+ #else
+ data_ptr -= 8;
+ #endif
+
+ if (len == 0) return;
+ } else {
+ *data_ptr = *(BX_MEM_THIS get_vector(a20addr));
+ if (len == 1) return;
+ len--;
+ a20addr++;
+ #ifdef BX_LITTLE_ENDIAN
+ data_ptr++;
+ #else // BX_BIG_ENDIAN
+ data_ptr--;
+ #endif
+ }
}
}