Re: SVM VMRUN Weirdness

Hi,

 

I thought about it a lot and didn’t find anything better than to ignore the ‘entering paged real mode’ check inside the check_CR0.

I don’t want to skip the check_CR0 or change the intercept priority.

It would be interesting to check vs real AMD hardware and figure out what the intercept priority is in the real hardware and how it supposed to work.

 

The check to do could be:

 

Try to set some other illegal combination under SVM guest, like set CR0.NW with CR0.CD cleared

Or attempt to enable paging when EFER.LME is set but CR4.PAE is not.

If I am right – these conditions still supposed to fire exception in the guest and not get to CR0 write intercept.

 

Thanks,

Stanislav

 

From: [email protected] [mailto:[email protected]] 
Sent: Tuesday, February 26, 2013 1:56 AM
To: Stanislav
Subject: Re: [Bochs-developers] SVM VMRUN Weirdness

 

Stanislav,

     In regards to the CR0 situation, I will always defer to your superior knowledge on such things, as before now I have only ever dabbled in machine level functionality.  I just thought I would explain what I encountered.  I am booting DOS 6.22 as my real mode guest.  One of the things that the boot process does  (I think in HIMEM.SYS) is transfer data to extended memory via BIOS INT 15, AH=87 (SYSTEM - COPY EXTENDED MEMORY).  You may already know that the BOCHS BIOS INT 15/87 code sets up a GDT, enters pmode, copies the data, and then exits back to real mode.

     My guest utilizes the hardware's (ie. BOCHS) BIOS for now, and the problem I ran into is when the guest BIOS attempts to switch back to real mode, setting the CR0 from E0000011 to E0000010.  The PG bit that remains set is normally invalid for real mode PE=0 and I get the failure below.

(0).[24658006] [0x00000000000f4136] 0020:0000000000004136 (unk. ctxt): mov cr0, eax              ; 0f22c0
00024658006p[CPU0 ] >>PANIC<< exception(): 3rd (13) exception with no resolution

     I *would* catch this in the hypervisor and allow it to pass, since it is really a valid state for a VM guest.  However the exception in my BOCHS debugger output above gets triggered by 

if (! check_CR0(val)) return 0;

     And because of the "return 0", the intercept later in SetCR0 is never triggered and my hypervisor never sees the instruction.

     So ... the guest is simply trying to exit pmode, which of course it doesn't touch the PG bit to do.  Thus the PG bit stays set and gets caught by BOCHS exception code (Check_CR0) before the hypervisor has a chance to intercept it.  My BOCHS SetCR0 change simply bypassed the Check_CR0 when the machine is in guest mode with a CR0 intercept set, to allow the hypervisor to interrogate the CR0 and decide if there is an error condition. 

     Maybe I am thinking about this in the wrong way.  Maybe it is supposed to work the way it does, and I am supposed to intercept the exception and deal with it?  Or would a "real machine" BIOS INT 15 not enter pmode to do the copy of data to extended memory?

     You do not have to answer those questions ... they are just to provoke thought. (mostly mine)  I can test on my AMD Box.  I will need to add some exception logic though, to catch the mov cr0,eax attempt if the AMD box doesn't like the guest doing so.  And of course the exception is #13, the most generic of all exceptions, just to make things a little more ...interesting. :-)

     I will put my other paged real mode work on the side and get my program ready to run on my hardware for the first time.  I will need to also debug my own BIOS a bit and see how it handles INT 15/87.  Hopefully it does a pmode switch as I anticipate.  I will write up the results when I have them and let you know.

     Hmmm ... looks like I have to go hunt down a floppy drive too!  My AMD box doesn't have one, but my program is run from the boot sector of a floppy disk image.  Luckily I have lots of old machines lying around.

Best Regards,

Chris

  _____  

From: "Stanislav" <[email protected]>
To: [email protected]
Cc: [email protected]
Sent: Monday, February 25, 2013 2:35:03 PM
Subject: RE: [Bochs-developers] SVM VMRUN Weirdness




Hello Chris,

 

I added to SVN rev11643 missed SVM intercepts I found (RSM, IRET, TR read/write, LDTR read/write).

This was really big finding by you !

 

About CR0 check I need to think about it more. I am not sure you are right.

The AMD docs say that CR0 intercept is done after all exceptions and reserved bit combinations checks in the CR0:

Exact quote:

Checks non-memory exceptions (CPL, illegal bit combinations, etc.) before the intercept

 

I don’t know if the check of PM=0, PG=1 is legal here. In my opinion this combination is illegal to the software and therefore software cannot try to set it on its own.

It is hypervisor which would set the ‘paged real mode’, not the software.

But again – the only proof would be vs real hardware, if you show some case which works on real AMD machine and doesn’t in Bochs emulation.

 

In my opinion the hypervisor has to intercept all CR0 reads in order to fake to the guest ‘correct’ CR0 with no paged real mode so guest software will be unaware of page real mode existence.

The paged real mode is just a way to run real mode guest when in fact it is impossible to disable paging without doing problems with the host.

 

Stanislav

 

From: [email protected] [mailto:[email protected]] 
Sent: Monday, February 25, 2013 3:04 PM
To: Stanislav
Subject: Re: [Bochs-developers] SVM VMRUN Weirdness

 

Stanislav,

     I do have an AMD Phenom II X4 955 processor to test the segment register base logic on.  I will be able to test my program soon on it, but still have a couple of issues to overcome.

    One easy issue I did overcome is that the CR0 write intercepts (there are 2) were not working.  It was a very simple issue though.  There is a call to check_CR0(val) that happens before the intercept code, and if the CR0 value isn't valid for the host, an exception is thrown.  Of course since paged real mode on the guest allows you to set PG=1 while PM=0, the CR0 value can definitely be invalid for the host but should still be presented to the hypervisor.   I put a set of conditions around the call to check_CR0(val) so that it doesn't get called if SVM guest is active.  That way the intercept is triggered and the host can figure out what to do with the attempt to set CR0.

ccregs.cc : top of BX_CPU_C::SetCR0
***CODE***
#if BX_SUPPORT_SVM
  if (BX_CPU_THIS_PTR in_svm_guest)
    if(!SVM_CR_WRITE_INTERCEPTED(0))
#endif
  if (! check_CR0(val)) return 0;
************
(the last line already existed)

     Another thing is I could not find code for the IRET intercept.  I put some in ctrl_xfer16.cc so that the real mode IRET is intercepted, but have not attempted to find where code should go for pmode.  It was important for me to intercept IRET because for the purposes of this real mode program, when using nested paging, I am moving the DMA buffer contents from the real mode memory address to the guest's paged address.

     I can supply a patch for CR0 if it is the proper approach.  I don't think I should supply a patch for the real mode IRET until I work on where to intercept protected mode IRETs as well.

     However before I can run my program and test on the hardware, I need to understand something about paged real mode w/o nested paging.  The way I got it to work was to create page tables *in the guest memory*, set CR3 to the guest tables, and set PG=1 (with PM=0).  It works great and I don't have any page faults to intercept, right up until the real mode OS (DOS 6.22) clobbers my page tables.  I could get around that by not allowing the guest access to the page table memory, but I think this is the wrong approach.  I have never done shadow page tables before, but I think that SPT is the answer, and that the SPT solution would not require me to put page tables in guest memory, OR set CR3 to them.  It would however require me to intercept page faults and deal with them appropriately.  Once I get this figured out, I'm good to go with a test.  

     Please let me know your thoughts on the CR0 and IRET finds above.  I will keep plugging away at paged real mode and as soon as I am successful, I'll do some segment register base/limit tests on the real hardware.

     I think you will turn out to be correct because I know that a long time ago I heard about "unreal mode" in which you enter protected mode, set up bases and limits of the registers, and then exit back to real mode, and that this would give you access to the full 32 bit memory space.  Doing so would not work unless the bases & limits were actively used during real mode operation.  I looked up unreal mode on Wikipedia and found the following:

"For efficiency reasons, the 80386 and later x86 processors use the base address stored in their internal segment descriptor cache whenever accessing memory, regardless of whether they are operating in real or protected mode."

     Talk to you soon!

Regards,

Chris

  _____  

From: "Stanislav" <[email protected]>
To: [email protected], [email protected]
Sent: Thursday, February 21, 2013 2:43:30 PM
Subject: RE: [Bochs-developers] SVM VMRUN Weirdness



BTW, I never tried it with real AMD SVM hardware so I don’t know if Bochs is right here or not.

In theory, when in real mode the base is calculated directly from segment index and therefore not necessary should be in the VMCB so Bochs might be wrong.

Do you have the real AMD machine to try ?

 

Stanislav

 

From: [email protected] [mailto:[email protected]] 
Sent: Thursday, February 21, 2013 9:43 AM
To: [email protected]
Subject: Re: [Bochs-developers] SVM VMRUN Weirdness

 

Hi,

As I suspected, the problem was mine.  Being that I have coded mostly for protected mode, I was not aware that I needed to maintain the the descriptor data for each segment register in real mode.  I basically set all of the descriptors to all zeroes (except for the attribute) in the VMCB.  I have since learned by debugging BOCHS enough, that I need to set the base for each descriptor even in real mode.

I hope I haven't wasted anyone's time.  My apologies if I have.

Regards,

Chris

  _____  

From: [email protected]
To: [email protected]
Sent: Tuesday, February 19, 2013 12:44:52 PM
Subject: [Bochs-developers] SVM VMRUN Weirdness

Hi,

I have run into an issue with (I think) the bochs SVM functionality.  In debugging some new code, I found that when I step through a VMRUN call (first guest instructions executed), everything functions as expected. (see VMRun-Step.txt attached)  However if I turn trace on and don't step through, something odd happens. (see VMRun-Trace.txt also attached)

Regarding this code in the attached files:
0000:7C00    JMP +60
...
...
0000:7C36    cli

If I step through VMRUN, the first guest instruction executed is 0000:7C00, as it should be.  Then the 2nd instruction executed is 0000:7C36, also as it should be.

However, if I trace through VMRUN, the next instruction executed says it is at 0000:7C00 and is JMP +60, but then the next instruction executed is 0000:7C01.  This is the problem.  In my case 7C01 has an invalid opcode (0x3c90) so I can verify that tracing executes 7C01 while stepping doesn't.

I tried to track down the problem but haven't yet fully come to understand the instruction execution functionality, especially the prefetching.  However on a whim I did hack a fix in SVM.cc by setting async_event=1 unconditionally just before the call to handleCpuContextChange.  Note that I didn't do so because I knew what async_event would do ... it was just a wild guess.

After that change, when I run the modified bochs code, both stepping and tracing execute instruction #2 at 7C36.

But there's more, and it could be related.  With the "problem fixed" I then added code to my program to process intercepted interrupts from the guest.  So when the guest calls, for example, Int 13, I intercept it, put the guest Flags:CS:IP on the stack, decrement the SP, and point the guest at the vector CS:IP via the VMCB.  But when all my hypervisor work is done and VMRUN is called, the wrong guest code is executed.  In this case the debugger *says* that f000:e3fe is being executed, but the logical addy to the left of the segmented address says: [0x000000000000e3fe]

QUOTE:
(0) [0x00000000000502da] 0008:00000000000002da (unk. ctxt): vmrun rax                 ; 0f01d8
00016993704i[XGUI ] Mouse capture off
<bochs:30> s
Next at t=16993705
(0) [0x000000000000e3fe] f000:e3fe (unk. ctxt): add byte ptr ds:[bx+si], al ; 0000

I know for a fact that I populated the CS in the VMCB correctly, and it shows in the segmented address.

I am not sure that there is any relation between the two issues.  After all, the hack for the first issue doesn't fix the 2nd.  And no, the hack didn't CAUSE the 2nd issue.  I verified that by removing the hack and stepping through the code until I hit the line above. (f000:e3fe).  It still says:

(0) [0x000000000000e3fe] f000:e3fe (unk. ctxt): add byte ptr ds:[bx+si], al ; 0000

If someone can point me in the right direction, I'd be glad to help track down the issue(s) ... if they are indeed issues and not some problems with my code.  I've been looking at the bochs code quite a bit but it takes time to figure out how each element works.

Thanks,

Chris


------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
bochs-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bochs-developers

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb

_______________________________________________
bochs-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bochs-developers