x86/entry/64: Fix CR3 restore in paranoid_exit()

Web:        https://git.kernel.org/torvalds/c/e48657573481a5dff7cfdc3d57005c80aa816500
Commit:     e48657573481a5dff7cfdc3d57005c80aa816500
Parent:     24dbc6000f4b9b0ef5a9daecb161f1907733765a
Refname:    refs/heads/master
Author:     Ingo Molnar <[email protected]>
AuthorDate: Wed Feb 14 08:39:11 2018 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Thu Feb 15 01:15:54 2018 +0100

    x86/entry/64: Fix CR3 restore in paranoid_exit()
    
    Josh Poimboeuf noticed the following bug:
    
     "The paranoid exit code only restores the saved CR3 when it switches back
      to the user GS.  However, even in the kernel GS case, it's possible that
      it needs to restore a user CR3, if for example, the paranoid exception
      occurred in the syscall exit path between SWITCH_TO_USER_CR3_STACK and
      SWAPGS."
    
    Josh also confirmed via targeted testing that it's possible to hit this bug.
    
    Fix the bug by also restoring CR3 in the paranoid_exit_no_swapgs branch.
    
    The reason we haven't seen this bug reported by users yet is probably because
    "paranoid" entry points are limited to the following cases:
    
     idtentry double_fault       do_double_fault  has_error_code=1  paranoid=2
     idtentry debug              do_debug         has_error_code=0  paranoid=1 shift_ist=DEBUG_STACK
     idtentry int3               do_int3          has_error_code=0  paranoid=1 shift_ist=DEBUG_STACK
     idtentry machine_check      do_mce           has_error_code=0  paranoid=1
    
    Amongst those entry points only machine_check is one that will interrupt an
    IRQS-off critical section asynchronously - and machine check events are rare.
    
    The other main asynchronous entries are NMI entries, which can be very high-freq
    with perf profiling, but they are special: they don't use the 'idtentry' macro but
    are open coded and restore user CR3 unconditionally so don't have this bug.
    
    Reported-and-tested-by: Josh Poimboeuf <[email protected]>
    Reviewed-by: Andy Lutomirski <[email protected]>
    Acked-by: Thomas Gleixner <[email protected]>
    Cc: Arjan van de Ven <[email protected]>
    Cc: Borislav Petkov <[email protected]>
    Cc: Dan Williams <[email protected]>
    Cc: Dave Hansen <[email protected]>
    Cc: David Woodhouse <[email protected]>
    Cc: Greg Kroah-Hartman <[email protected]>
    Cc: Linus Torvalds <[email protected]>
    Cc: Peter Zijlstra <[email protected]>
    Link: http://lkml.kernel.org/r/[email protected]
    Signed-off-by: Ingo Molnar <[email protected]>
---
 arch/x86/entry/entry_64.S | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index 1c54204207d8..4fd9044e72e7 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -1168,6 +1168,7 @@ ENTRY(paranoid_exit)
 	jmp	.Lparanoid_exit_restore
 .Lparanoid_exit_no_swapgs:
 	TRACE_IRQS_IRETQ_DEBUG
+	RESTORE_CR3	scratch_reg=%rbx save_reg=%r14
 .Lparanoid_exit_restore:
 	jmp restore_regs_and_return_to_kernel
 END(paranoid_exit)
--
To unsubscribe from this list: send the line "unsubscribe git-commits-head" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html