Xen Security Advisory 267 (CVE-2018-3665) - Speculative register leakage from lazy FPU context switching
Xen.org security team <[email protected]>
| Newsgroups | gmane.comp.emulators.xen.announce |
|---|---|
| Message-ID | <E1fTCt4-00012V-G3__8047.33991758457$1528924073$gmane$org@xenbits.xenproject.org> |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Xen Security Advisory CVE-2018-3665 / XSA-267
version 3
Speculative register leakage from lazy FPU context switching
UPDATES IN VERSION 3
====================
Public release.
ISSUE DESCRIPTION
=================
x86 has a hardware mechanism for lazy FPU context switching. On a task
switch, %cr0.ts (Task Switched) gets set, and the next instruction to
touch floating point state raises an #NM (No Math, later known as Device
Not Available) exception.
Traditionally, FPU state has been large in comparison to available
bandwidth (and therefore slow to switch) and not used as frequently as
cpu tasks tend to switch. This mechanism allows the OS to only switch
FPU when necessary, which in turn increases performance.
Some CPUs however speculate past an #NM exception, allowing register
content to be leaked by a side-channel.
For more details, see:
https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00145.html
IMPACT
======
An attacker can read x87/MMX/SSE/AVX/AVX-512 register state belonging to
another vCPU previously scheduled on the same processor. This can be
state belonging a different guest, or state belonging to a different
thread inside the same guest.
Furthermore, similar changes are expected for OS kernels. Consult your
operating system provider for more information.
VULNERABLE SYSTEMS
==================
Systems running all versions of Xen are affected.
Only x86 processors are vulnerable. ARM processors are not known to be
affected.
Only Intel Core based processors (from at least Nehalem onwards) are
potentially affected. Other processor designs (Intel Atom/Knights
range), and other manufacturers (AMD) are not known to be affected.
MITIGATION
==========
Depending on the availability of host resources, leakage can be
prevented between VMs by using cpupools or cpu pinning to isolate the
vCPUs from different VMs to separate pCPUs.
CREDITS
=======
This issue was discovered by Julian Stecklina ([email protected]) from
Amazon and Thomas Prescher ([email protected]) from
Cyberus Technology.
It was also independenty discovered by Zdenek Sojka from SYSGO
(http://sysgo.com) and by Colin Percival.
RESOLUTION
==========
Applying the appropriate attached patch resolves this issue.
xsa267-[12].patch xen-unstable
xsa267-4.10-[12].patch Xen 4.10.x
xsa267-4.9-[12].patch Xen 4.9.x, 4.8.x
xsa267-4.7-[12].patch Xen 4.7.x
xsa267-4.6-[12].patch Xen 4.6.x
Alternatively, the following patches can be used to create livepatches for
running hypervisors.
xsa267-livepatch.patch xen-unstable, Xen 4.10.x, 4.9.x
xsa267-4.8-livepatch.patch Xen 4.8.x
$ sha256sum xsa267*
d126e57ac6151e661294da9211a9d556845255a9d1909d73ec58a28c81b4a79d xsa267-1.patch
00ec30c3738c3fcac8ca24a03308fc2d2dacab78640c17e5bb078e474b263719 xsa267-2.patch
9172c51e3652498740aa54c7953fb70c6df3902b382a9e9fa25a82943f70849d xsa267-4.6-1.patch
8579fa847aea19b3666db39c9c844c32b543e5504f49074e48600c4958fa9eba xsa267-4.6-2.patch
0fb7c123947a95963537ddeb156718d93a3d04b42486009fc520eaaeeba8aad6 xsa267-4.7-1.patch
418a71f8fc5b3ff1a5eb5cf4d161dea9c88697b50d84d8b8eec1ecf594f798f1 xsa267-4.7-2.patch
488f769e19acfe4ca59c731f58c5d464ec694e3c1923fbb3a26e6ed85afa68f8 xsa267-4.8-livepatch.patch
b4d1712b48c71ca541b6a39c182c3a134ff4d36cbf52ef6d65444ce84729c4b3 xsa267-4.9-1.patch
5ab13ae9ea070b2eee6ecf31324518f8315b7c0e523295d7892e5263fccb9d1f xsa267-4.9-2.patch
9703a2e661f67408a108b540d296439cd349027a322b2e360780319897386753 xsa267-4.10-1.patch
d30dcb4887cb1963b460f850f34f0cd179704a2cdc8cdaf72bd16e495a0d63f1 xsa267-4.10-2.patch
7832229d987ac9b7292eb815d54b78e9884b892795d9ac3f11f0752f6c59d312 xsa267-livepatch.patch
$
DEPLOYMENT DURING EMBARGO
=========================
Deployment of the patches and/or mitigations described above (or
others which are substantially similar) is permitted during the
embargo, even on public-facing systems with untrusted guest users and
administrators.
But: Distribution of updated software is prohibited (except to other
members of the predisclosure list).
Predisclosure list members who wish to deploy significantly different
patches and/or mitigations, please contact the Xen Project Security
Team.
(Note: this during-embargo deployment notice is retained in
post-embargo publicly released Xen Project advisories, even though it
is then no longer applicable. This is to enable the community to have
oversight of the Xen Project Security Team's decisionmaking.)
For more information about permissible uses of embargoed information,
consult the Xen Project community's agreed Security Policy:
http://www.xenproject.org/security-policy.html
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAEBCAAGBQJbIX1aAAoJEIP+FMlX6CvZiA4H/iwQn5aa+9+iE6wVNhfI3XX4
YvogEPGW4Zp3Brq5ATDoyanIdabWU+5Tq4MtAyR3IyaFrnoevLFfumKIQjnqI3sk
ef4PuxEYtVyiqwC+01o0Uk1+1K83xA4dG2wukuJbOtEF44d4X5fq9RWqdLprBADx
FW7MrFeXcoQsbbRLfZzUZpjtQQ1Lys8gHbU+Un9l9yZjRUErxUFLhhsrpIwkYFF9
6zhlYGPdpZQ1s7W6OclD/Tm5ZpauggjJfLWSAckAuobNaR6bKh6iwr3AMWH0w+2w
H7U2oKHQPw3kpiEz42cEEN9FDm/9mGNgNYkC+aPtn40zYuKhyBnORBMgssmA0Tk=
=By/q
-----END PGP SIGNATURE-----
_______________________________________________
Xen-announce mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-announce
xsa267-1.patch
(application/octet-stream, 2.1 KB)
From: Andrew Cooper <[email protected]> Subject: x86: Support fully eager FPU context switching This is controlled on a per-vcpu bases for flexibility. This is part of XSA-267 / CVE-2018-3665 Signed-off-by: Andrew Cooper <[email protected]> Reviewed-by: Jan Beulich <[email protected]> diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index 8845252..50116d5 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -210,7 +210,7 @@ void vcpu_restore_fpu_eager(struct vcpu *v) ASSERT(!is_idle_vcpu(v)); /* Restore nonlazy extended state (i.e. parts not tracked by CR0.TS). */ - if ( !v->arch.nonlazy_xstate_used ) + if ( !v->arch.fully_eager_fpu && !v->arch.nonlazy_xstate_used ) return; /* Avoid recursion */ @@ -221,11 +221,19 @@ void vcpu_restore_fpu_eager(struct vcpu *v) * above) we also need to restore full state, to prevent subsequently * saving state belonging to another vCPU. */ - if ( xstate_all(v) ) + if ( v->arch.fully_eager_fpu || (v->arch.xsave_area && xstate_all(v)) ) { - fpu_xrstor(v, XSTATE_ALL); + if ( cpu_has_xsave ) + fpu_xrstor(v, XSTATE_ALL); + else + fpu_fxrstor(v); + v->fpu_initialised = 1; v->fpu_dirtied = 1; + + /* Xen doesn't need TS set, but the guest might. */ + if ( is_pv_vcpu(v) && (v->arch.pv_vcpu.ctrlreg[0] & X86_CR0_TS) ) + stts(); } else { @@ -247,6 +255,8 @@ void vcpu_restore_fpu_lazy(struct vcpu *v) if ( v->fpu_dirtied ) return; + ASSERT(!v->arch.fully_eager_fpu); + if ( cpu_has_xsave ) fpu_xrstor(v, XSTATE_LAZY); else diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h index 197f8d6..e0d413c 100644 --- a/xen/include/asm-x86/domain.h +++ b/xen/include/asm-x86/domain.h @@ -563,6 +563,9 @@ struct arch_vcpu * and thus should be saved/restored. */ bool_t nonlazy_xstate_used; + /* Restore all FPU state (lazy and non-lazy state) on context switch? */ + bool fully_eager_fpu; + struct vmce vmce; struct paging_vcpu paging;
xsa267-2.patch
(application/octet-stream, 8.1 KB)
From: Andrew Cooper <[email protected]> Subject: x86/spec-ctrl: Mitigations for LazyFPU Intel Core processors since at least Nehalem speculate past #NM, which is the mechanism by which lazy FPU context switching is implemented. On affected processors, Xen must use fully eager FPU context switching to prevent guests from being able to read FPU state (SSE/AVX/etc) from previously scheduled vcpus. This is part of XSA-267 / CVE-2018-3665 Signed-off-by: Andrew Cooper <[email protected]> Reviewed-by: Jan Beulich <[email protected]> diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown index 8712a83..075e5ea 100644 --- a/docs/misc/xen-command-line.markdown +++ b/docs/misc/xen-command-line.markdown @@ -1758,7 +1758,7 @@ false disable the quirk workaround, which is also the default. ### spec-ctrl (x86) > `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>, -> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd}=<bool> ]` +> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd,eager-fpu}=<bool> ]` Controls for speculative execution sidechannel mitigations. By default, Xen will pick the most appropriate mitigations based on compiled in support, @@ -1808,6 +1808,11 @@ hardware, this is a global option applied at boot, and not virtualised for guest use. On Intel hardware, the feature is virtualised for guests, independently of Xen's choice of setting. +On all hardware, the `eager-fpu=` option can be used to force or prevent Xen +from using fully eager FPU context switches. This is currently implemented as +a global control. By default, Xen will choose to use fully eager context +switches on hardware believed to speculate past #NM exceptions. + ### sync\_console > `= <boolean>` diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index 50116d5..dbdf1b4 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -15,6 +15,7 @@ #include <asm/i387.h> #include <asm/xstate.h> #include <asm/asm_defns.h> +#include <asm/spec_ctrl.h> /*******************************/ /* FPU Restore Functions */ @@ -307,6 +308,8 @@ int vcpu_init_fpu(struct vcpu *v) { int rc; + v->arch.fully_eager_fpu = opt_eager_fpu; + if ( (rc = xstate_alloc_save_area(v)) != 0 ) return rc; diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index fd938c3..08e6784 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -44,6 +44,7 @@ static enum ind_thunk { static int8_t __initdata opt_ibrs = -1; bool __read_mostly opt_ibpb = true; bool __read_mostly opt_ssbd = false; +int8_t __read_mostly opt_eager_fpu = -1; bool __initdata bsp_delay_spec_ctrl; uint8_t __read_mostly default_xen_spec_ctrl; @@ -130,6 +131,7 @@ static int __init parse_spec_ctrl(const char *s) opt_thunk = THUNK_JMP; opt_ibrs = 0; opt_ibpb = false; + opt_eager_fpu = 0; } else if ( val > 0 ) rc = -EINVAL; @@ -183,6 +185,8 @@ static int __init parse_spec_ctrl(const char *s) opt_ibpb = val; else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 ) opt_ssbd = val; + else if ( (val = parse_boolean("eager-fpu", s, ss)) >= 0 ) + opt_eager_fpu = val; else rc = -EINVAL; @@ -236,15 +240,19 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps) * Alternatives blocks for protecting against and/or virtualising * mitigation support for guests. */ - printk(" Support for VMs: PV:%s%s%s, HVM:%s%s%s\n", + printk(" Support for VMs: PV:%s%s%s%s, HVM:%s%s%s%s\n", (boot_cpu_has(X86_FEATURE_SC_MSR_PV) || - boot_cpu_has(X86_FEATURE_SC_RSB_PV)) ? "" : " None", + boot_cpu_has(X86_FEATURE_SC_RSB_PV) || + opt_eager_fpu) ? "" : " None", boot_cpu_has(X86_FEATURE_SC_MSR_PV) ? " MSR_SPEC_CTRL" : "", boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB" : "", + opt_eager_fpu ? " EAGER_FPU" : "", (boot_cpu_has(X86_FEATURE_SC_MSR_HVM) || - boot_cpu_has(X86_FEATURE_SC_RSB_HVM)) ? "" : " None", + boot_cpu_has(X86_FEATURE_SC_RSB_HVM) || + opt_eager_fpu) ? "" : " None", boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ? " MSR_SPEC_CTRL" : "", - boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : ""); + boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : "", + opt_eager_fpu ? " EAGER_FPU" : ""); printk(" XPTI (64-bit PV only): Dom0 %s, DomU %s\n", opt_xpti & OPT_XPTI_DOM0 ? "enabled" : "disabled", @@ -334,6 +342,82 @@ static bool __init retpoline_safe(uint64_t caps) } } +/* Calculate whether this CPU speculates past #NM */ +static bool __init should_use_eager_fpu(void) +{ + /* + * Assume all unrecognised processors are ok. This is only known to + * affect Intel Family 6 processors. + */ + if ( boot_cpu_data.x86_vendor != X86_VENDOR_INTEL || + boot_cpu_data.x86 != 6 ) + return false; + + switch ( boot_cpu_data.x86_model ) + { + /* + * Core processors since at least Nehalem are vulnerable. + */ + case 0x1e: /* Nehalem */ + case 0x1f: /* Auburndale / Havendale */ + case 0x1a: /* Nehalem EP */ + case 0x2e: /* Nehalem EX */ + case 0x25: /* Westmere */ + case 0x2c: /* Westmere EP */ + case 0x2f: /* Westmere EX */ + case 0x2a: /* SandyBridge */ + case 0x2d: /* SandyBridge EP/EX */ + case 0x3a: /* IvyBridge */ + case 0x3e: /* IvyBridge EP/EX */ + case 0x3c: /* Haswell */ + case 0x3f: /* Haswell EX/EP */ + case 0x45: /* Haswell D */ + case 0x46: /* Haswell H */ + case 0x3d: /* Broadwell */ + case 0x47: /* Broadwell H */ + case 0x4f: /* Broadwell EP/EX */ + case 0x56: /* Broadwell D */ + case 0x4e: /* Skylake M */ + case 0x55: /* Skylake X */ + case 0x5e: /* Skylake D */ + case 0x66: /* Cannonlake */ + case 0x67: /* Cannonlake? */ + case 0x8e: /* Kabylake M */ + case 0x9e: /* Kabylake D */ + return true; + + /* + * Atom processors are not vulnerable. + */ + case 0x1c: /* Pineview */ + case 0x26: /* Lincroft */ + case 0x27: /* Penwell */ + case 0x35: /* Cloverview */ + case 0x36: /* Cedarview */ + case 0x37: /* Baytrail / Valleyview (Silvermont) */ + case 0x4d: /* Avaton / Rangely (Silvermont) */ + case 0x4c: /* Cherrytrail / Brasswell */ + case 0x4a: /* Merrifield */ + case 0x5a: /* Moorefield */ + case 0x5c: /* Goldmont */ + case 0x5f: /* Denverton */ + case 0x7a: /* Gemini Lake */ + return false; + + /* + * Knights processors are not vulnerable. + */ + case 0x57: /* Knights Landing */ + case 0x85: /* Knights Mill */ + return false; + + default: + printk("Unrecognised CPU model %#x - assuming vulnerable to LazyFPU\n", + boot_cpu_data.x86_model); + return true; + } +} + #define OPT_XPTI_DEFAULT 0xff uint8_t __read_mostly opt_xpti = OPT_XPTI_DEFAULT; @@ -530,6 +614,10 @@ void __init init_speculation_mitigations(void) if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) ) opt_ibpb = false; + /* Check whether Eager FPU should be enabled by default. */ + if ( opt_eager_fpu == -1 ) + opt_eager_fpu = should_use_eager_fpu(); + /* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */ init_shadow_spec_ctrl_state(); diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h index 91bed1b..5b40afb 100644 --- a/xen/include/asm-x86/spec_ctrl.h +++ b/xen/include/asm-x86/spec_ctrl.h @@ -28,6 +28,7 @@ void init_speculation_mitigations(void); extern bool opt_ibpb; extern bool opt_ssbd; +extern int8_t opt_eager_fpu; extern bool bsp_delay_spec_ctrl; extern uint8_t default_xen_spec_ctrl;
xsa267-4.6-1.patch
(application/octet-stream, 1.9 KB)
From: Andrew Cooper <[email protected]> Subject: x86: Support fully eager FPU context switching This is controlled on a per-vcpu bases for flexibility. This is part of XSA-267 / CVE-2018-3665 Signed-off-by: Andrew Cooper <[email protected]> Reviewed-by: Jan Beulich <[email protected]> diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index 5584535..f9cf059 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -215,8 +215,25 @@ void vcpu_restore_fpu_eager(struct vcpu *v) { ASSERT(!is_idle_vcpu(v)); + if ( v->arch.fully_eager_fpu ) + { + /* Avoid recursion */ + clts(); + + if ( cpu_has_xsave ) + fpu_xrstor(v, XSTATE_ALL); + else + fpu_fxrstor(v); + + v->fpu_initialised = 1; + v->fpu_dirtied = 1; + + /* Xen doesn't need TS set, but the guest might. */ + if ( is_pv_vcpu(v) && (v->arch.pv_vcpu.ctrlreg[0] & X86_CR0_TS) ) + stts(); + } /* save the nonlazy extended state which is not tracked by CR0.TS bit */ - if ( v->arch.nonlazy_xstate_used ) + else if ( v->arch.nonlazy_xstate_used ) { /* Avoid recursion */ clts(); @@ -238,6 +255,8 @@ void vcpu_restore_fpu_lazy(struct vcpu *v) if ( v->fpu_dirtied ) return; + ASSERT(!v->arch.fully_eager_fpu); + if ( cpu_has_xsave ) fpu_xrstor(v, XSTATE_LAZY); else diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h index f917b46..20715e3 100644 --- a/xen/include/asm-x86/domain.h +++ b/xen/include/asm-x86/domain.h @@ -574,6 +574,9 @@ struct arch_vcpu * and thus should be saved/restored. */ bool_t nonlazy_xstate_used; + /* Restore all FPU state (lazy and non-lazy state) on context switch? */ + bool_t fully_eager_fpu; + /* * The SMAP check policy when updating runstate_guest(v) and the * secondary system time.
xsa267-4.6-2.patch
(application/octet-stream, 8 KB)
From: Andrew Cooper <[email protected]> Subject: x86/spec-ctrl: Mitigations for LazyFPU Intel Core processors since at least Nehalem speculate past #NM, which is the mechanism by which lazy FPU context switching is implemented. On affected processors, Xen must use fully eager FPU context switching to prevent guests from being able to read FPU state (SSE/AVX/etc) from previously scheduled vcpus. This is part of XSA-267 / CVE-2018-3665 Signed-off-by: Andrew Cooper <[email protected]> Reviewed-by: Jan Beulich <[email protected]> diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown index ece2cec..b742ee4 100644 --- a/docs/misc/xen-command-line.markdown +++ b/docs/misc/xen-command-line.markdown @@ -1423,7 +1423,7 @@ false disable the quirk workaround, which is also the default. ### spec-ctrl (x86) > `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>, -> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd}=<bool> ]` +> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd,eager-fpu}=<bool> ]` Controls for speculative execution sidechannel mitigations. By default, Xen will pick the most appropriate mitigations based on compiled in support, @@ -1473,6 +1473,11 @@ hardware, this is a global option applied at boot, and not virtualised for guest use. On Intel hardware, the feature is virtualised for guests, independently of Xen's choice of setting. +On all hardware, the `eager-fpu=` option can be used to force or prevent Xen +from using fully eager FPU context switches. This is currently implemented as +a global control. By default, Xen will choose to use fully eager context +switches on hardware believed to speculate past #NM exceptions. + ### sync\_console > `= <boolean>` diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index f9cf059..873eab8 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -16,6 +16,7 @@ #include <asm/i387.h> #include <asm/xstate.h> #include <asm/asm_defns.h> +#include <asm/spec_ctrl.h> /*******************************/ /* FPU Restore Functions */ @@ -309,6 +310,8 @@ int vcpu_init_fpu(struct vcpu *v) { int rc; + v->arch.fully_eager_fpu = opt_eager_fpu; + if ( (rc = xstate_alloc_save_area(v)) != 0 ) return rc; diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index 7900acd..8368972 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -44,6 +44,7 @@ static enum ind_thunk { static int8_t __initdata opt_ibrs = -1; bool_t __read_mostly opt_ibpb = 1; bool_t __read_mostly opt_ssbd = 0; +int8_t __read_mostly opt_eager_fpu = -1; bool_t __initdata bsp_delay_spec_ctrl; uint8_t __read_mostly default_xen_spec_ctrl; @@ -114,6 +115,7 @@ static int __init parse_spec_ctrl(char *s) opt_thunk = THUNK_JMP; opt_ibrs = 0; opt_ibpb = 0; + opt_eager_fpu = 0; } else if ( val > 0 ) rc = -EINVAL; @@ -167,6 +169,8 @@ static int __init parse_spec_ctrl(char *s) opt_ibpb = val; else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 ) opt_ssbd = val; + else if ( (val = parse_boolean("eager-fpu", s, ss)) >= 0 ) + opt_eager_fpu = val; else rc = -EINVAL; @@ -224,15 +228,19 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps) * Alternatives blocks for protecting against and/or virtualising * mitigation support for guests. */ - printk(" Support for VMs: PV:%s%s%s, HVM:%s%s%s\n", + printk(" Support for VMs: PV:%s%s%s%s, HVM:%s%s%s%s\n", (boot_cpu_has(X86_FEATURE_SC_MSR_PV) || - boot_cpu_has(X86_FEATURE_SC_RSB_PV)) ? "" : " None", + boot_cpu_has(X86_FEATURE_SC_RSB_PV) || + opt_eager_fpu) ? "" : " None", boot_cpu_has(X86_FEATURE_SC_MSR_PV) ? " MSR_SPEC_CTRL" : "", boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB" : "", + opt_eager_fpu ? " EAGER_FPU" : "", (boot_cpu_has(X86_FEATURE_SC_MSR_HVM) || - boot_cpu_has(X86_FEATURE_SC_RSB_HVM)) ? "" : " None", + boot_cpu_has(X86_FEATURE_SC_RSB_HVM) || + opt_eager_fpu) ? "" : " None", boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ? " MSR_SPEC_CTRL" : "", - boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : ""); + boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : "", + opt_eager_fpu ? " EAGER_FPU" : ""); printk(" XPTI (64-bit PV only): Dom0 %s, DomU %s\n", opt_xpti & OPT_XPTI_DOM0 ? "enabled" : "disabled", @@ -322,6 +330,82 @@ static bool_t __init __maybe_unused retpoline_safe(uint64_t caps) } } +/* Calculate whether this CPU speculates past #NM */ +static bool_t __init should_use_eager_fpu(void) +{ + /* + * Assume all unrecognised processors are ok. This is only known to + * affect Intel Family 6 processors. + */ + if ( boot_cpu_data.x86_vendor != X86_VENDOR_INTEL || + boot_cpu_data.x86 != 6 ) + return 0; + + switch ( boot_cpu_data.x86_model ) + { + /* + * Core processors since at least Nehalem are vulnerable. + */ + case 0x1e: /* Nehalem */ + case 0x1f: /* Auburndale / Havendale */ + case 0x1a: /* Nehalem EP */ + case 0x2e: /* Nehalem EX */ + case 0x25: /* Westmere */ + case 0x2c: /* Westmere EP */ + case 0x2f: /* Westmere EX */ + case 0x2a: /* SandyBridge */ + case 0x2d: /* SandyBridge EP/EX */ + case 0x3a: /* IvyBridge */ + case 0x3e: /* IvyBridge EP/EX */ + case 0x3c: /* Haswell */ + case 0x3f: /* Haswell EX/EP */ + case 0x45: /* Haswell D */ + case 0x46: /* Haswell H */ + case 0x3d: /* Broadwell */ + case 0x47: /* Broadwell H */ + case 0x4f: /* Broadwell EP/EX */ + case 0x56: /* Broadwell D */ + case 0x4e: /* Skylake M */ + case 0x55: /* Skylake X */ + case 0x5e: /* Skylake D */ + case 0x66: /* Cannonlake */ + case 0x67: /* Cannonlake? */ + case 0x8e: /* Kabylake M */ + case 0x9e: /* Kabylake D */ + return 1; + + /* + * Atom processors are not vulnerable. + */ + case 0x1c: /* Pineview */ + case 0x26: /* Lincroft */ + case 0x27: /* Penwell */ + case 0x35: /* Cloverview */ + case 0x36: /* Cedarview */ + case 0x37: /* Baytrail / Valleyview (Silvermont) */ + case 0x4d: /* Avaton / Rangely (Silvermont) */ + case 0x4c: /* Cherrytrail / Brasswell */ + case 0x4a: /* Merrifield */ + case 0x5a: /* Moorefield */ + case 0x5c: /* Goldmont */ + case 0x5f: /* Denverton */ + case 0x7a: /* Gemini Lake */ + return 0; + + /* + * Knights processors are not vulnerable. + */ + case 0x57: /* Knights Landing */ + case 0x85: /* Knights Mill */ + return 0; + + default: + printk("Unrecognised CPU model %#x - assuming vulnerable to LazyFPU\n", + boot_cpu_data.x86_model); + return 1; + } +} + #define OPT_XPTI_DEFAULT 0xff uint8_t __read_mostly opt_xpti = OPT_XPTI_DEFAULT; @@ -511,6 +595,10 @@ void __init init_speculation_mitigations(void) if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) ) opt_ibpb = 0; + /* Check whether Eager FPU should be enabled by default. */ + if ( opt_eager_fpu == -1 ) + opt_eager_fpu = should_use_eager_fpu(); + /* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */ init_shadow_spec_ctrl_state(); diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h index 45ac200..d81c855 100644 --- a/xen/include/asm-x86/spec_ctrl.h +++ b/xen/include/asm-x86/spec_ctrl.h @@ -28,6 +28,7 @@ void init_speculation_mitigations(void); extern bool_t opt_ibpb; extern bool_t opt_ssbd; +extern int8_t opt_eager_fpu; extern bool_t bsp_delay_spec_ctrl; extern uint8_t default_xen_spec_ctrl;
xsa267-4.7-1.patch
(application/octet-stream, 2.2 KB)
From: Andrew Cooper <[email protected]> Subject: x86: Support fully eager FPU context switching This is controlled on a per-vcpu bases for flexibility. This is part of XSA-267 / CVE-2018-3665 Signed-off-by: Andrew Cooper <[email protected]> Reviewed-by: Jan Beulich <[email protected]> diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index 86b098d..4110555 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -211,7 +211,7 @@ void vcpu_restore_fpu_eager(struct vcpu *v) ASSERT(!is_idle_vcpu(v)); /* Restore nonlazy extended state (i.e. parts not tracked by CR0.TS). */ - if ( !v->arch.nonlazy_xstate_used ) + if ( !v->arch.fully_eager_fpu && !v->arch.nonlazy_xstate_used ) return; /* Avoid recursion */ @@ -222,11 +222,19 @@ void vcpu_restore_fpu_eager(struct vcpu *v) * above) we also need to restore full state, to prevent subsequently * saving state belonging to another vCPU. */ - if ( xstate_all(v) ) + if ( v->arch.fully_eager_fpu || (v->arch.xsave_area && xstate_all(v)) ) { - fpu_xrstor(v, XSTATE_ALL); + if ( cpu_has_xsave ) + fpu_xrstor(v, XSTATE_ALL); + else + fpu_fxrstor(v); + v->fpu_initialised = 1; v->fpu_dirtied = 1; + + /* Xen doesn't need TS set, but the guest might. */ + if ( is_pv_vcpu(v) && (v->arch.pv_vcpu.ctrlreg[0] & X86_CR0_TS) ) + stts(); } else { @@ -248,6 +256,8 @@ void vcpu_restore_fpu_lazy(struct vcpu *v) if ( v->fpu_dirtied ) return; + ASSERT(!v->arch.fully_eager_fpu); + if ( cpu_has_xsave ) fpu_xrstor(v, XSTATE_LAZY); else diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h index 619a98d..e79af55 100644 --- a/xen/include/asm-x86/domain.h +++ b/xen/include/asm-x86/domain.h @@ -595,6 +595,9 @@ struct arch_vcpu * and thus should be saved/restored. */ bool_t nonlazy_xstate_used; + /* Restore all FPU state (lazy and non-lazy state) on context switch? */ + bool_t fully_eager_fpu; + /* * The SMAP check policy when updating runstate_guest(v) and the * secondary system time.
xsa267-4.7-2.patch
(application/octet-stream, 8 KB)
From: Andrew Cooper <[email protected]> Subject: x86/spec-ctrl: Mitigations for LazyFPU Intel Core processors since at least Nehalem speculate past #NM, which is the mechanism by which lazy FPU context switching is implemented. On affected processors, Xen must use fully eager FPU context switching to prevent guests from being able to read FPU state (SSE/AVX/etc) from previously scheduled vcpus. This is part of XSA-267 / CVE-2018-3665 Signed-off-by: Andrew Cooper <[email protected]> Reviewed-by: Jan Beulich <[email protected]> diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown index 039bc6b..0c24440 100644 --- a/docs/misc/xen-command-line.markdown +++ b/docs/misc/xen-command-line.markdown @@ -1472,7 +1472,7 @@ false disable the quirk workaround, which is also the default. ### spec-ctrl (x86) > `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>, -> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd}=<bool> ]` +> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd,eager-fpu}=<bool> ]` Controls for speculative execution sidechannel mitigations. By default, Xen will pick the most appropriate mitigations based on compiled in support, @@ -1522,6 +1522,11 @@ hardware, this is a global option applied at boot, and not virtualised for guest use. On Intel hardware, the feature is virtualised for guests, independently of Xen's choice of setting. +On all hardware, the `eager-fpu=` option can be used to force or prevent Xen +from using fully eager FPU context switches. This is currently implemented as +a global control. By default, Xen will choose to use fully eager context +switches on hardware believed to speculate past #NM exceptions. + ### sync\_console > `= <boolean>` diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index 4110555..279878f 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -16,6 +16,7 @@ #include <asm/i387.h> #include <asm/xstate.h> #include <asm/asm_defns.h> +#include <asm/spec_ctrl.h> /*******************************/ /* FPU Restore Functions */ @@ -308,6 +309,8 @@ int vcpu_init_fpu(struct vcpu *v) { int rc; + v->arch.fully_eager_fpu = opt_eager_fpu; + if ( (rc = xstate_alloc_save_area(v)) != 0 ) return rc; diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index d8528e3..7f3fd26 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -44,6 +44,7 @@ static enum ind_thunk { static int8_t __initdata opt_ibrs = -1; bool_t __read_mostly opt_ibpb = 1; bool_t __read_mostly opt_ssbd = 0; +int8_t __read_mostly opt_eager_fpu = -1; bool_t __initdata bsp_delay_spec_ctrl; uint8_t __read_mostly default_xen_spec_ctrl; @@ -114,6 +115,7 @@ static int __init parse_spec_ctrl(char *s) opt_thunk = THUNK_JMP; opt_ibrs = 0; opt_ibpb = 0; + opt_eager_fpu = 0; } else if ( val > 0 ) rc = -EINVAL; @@ -167,6 +169,8 @@ static int __init parse_spec_ctrl(char *s) opt_ibpb = val; else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 ) opt_ssbd = val; + else if ( (val = parse_boolean("eager-fpu", s, ss)) >= 0 ) + opt_eager_fpu = val; else rc = -EINVAL; @@ -223,15 +227,19 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps) * Alternatives blocks for protecting against and/or virtualising * mitigation support for guests. */ - printk(" Support for VMs: PV:%s%s%s, HVM:%s%s%s\n", + printk(" Support for VMs: PV:%s%s%s%s, HVM:%s%s%s%s\n", (boot_cpu_has(X86_FEATURE_SC_MSR_PV) || - boot_cpu_has(X86_FEATURE_SC_RSB_PV)) ? "" : " None", + boot_cpu_has(X86_FEATURE_SC_RSB_PV) || + opt_eager_fpu) ? "" : " None", boot_cpu_has(X86_FEATURE_SC_MSR_PV) ? " MSR_SPEC_CTRL" : "", boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB" : "", + opt_eager_fpu ? " EAGER_FPU" : "", (boot_cpu_has(X86_FEATURE_SC_MSR_HVM) || - boot_cpu_has(X86_FEATURE_SC_RSB_HVM)) ? "" : " None", + boot_cpu_has(X86_FEATURE_SC_RSB_HVM) || + opt_eager_fpu) ? "" : " None", boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ? " MSR_SPEC_CTRL" : "", - boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : ""); + boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : "", + opt_eager_fpu ? " EAGER_FPU" : ""); printk(" XPTI (64-bit PV only): Dom0 %s, DomU %s\n", opt_xpti & OPT_XPTI_DOM0 ? "enabled" : "disabled", @@ -321,6 +329,82 @@ static bool_t __init retpoline_safe(uint64_t caps) } } +/* Calculate whether this CPU speculates past #NM */ +static bool_t __init should_use_eager_fpu(void) +{ + /* + * Assume all unrecognised processors are ok. This is only known to + * affect Intel Family 6 processors. + */ + if ( boot_cpu_data.x86_vendor != X86_VENDOR_INTEL || + boot_cpu_data.x86 != 6 ) + return 0; + + switch ( boot_cpu_data.x86_model ) + { + /* + * Core processors since at least Nehalem are vulnerable. + */ + case 0x1e: /* Nehalem */ + case 0x1f: /* Auburndale / Havendale */ + case 0x1a: /* Nehalem EP */ + case 0x2e: /* Nehalem EX */ + case 0x25: /* Westmere */ + case 0x2c: /* Westmere EP */ + case 0x2f: /* Westmere EX */ + case 0x2a: /* SandyBridge */ + case 0x2d: /* SandyBridge EP/EX */ + case 0x3a: /* IvyBridge */ + case 0x3e: /* IvyBridge EP/EX */ + case 0x3c: /* Haswell */ + case 0x3f: /* Haswell EX/EP */ + case 0x45: /* Haswell D */ + case 0x46: /* Haswell H */ + case 0x3d: /* Broadwell */ + case 0x47: /* Broadwell H */ + case 0x4f: /* Broadwell EP/EX */ + case 0x56: /* Broadwell D */ + case 0x4e: /* Skylake M */ + case 0x55: /* Skylake X */ + case 0x5e: /* Skylake D */ + case 0x66: /* Cannonlake */ + case 0x67: /* Cannonlake? */ + case 0x8e: /* Kabylake M */ + case 0x9e: /* Kabylake D */ + return 1; + + /* + * Atom processors are not vulnerable. + */ + case 0x1c: /* Pineview */ + case 0x26: /* Lincroft */ + case 0x27: /* Penwell */ + case 0x35: /* Cloverview */ + case 0x36: /* Cedarview */ + case 0x37: /* Baytrail / Valleyview (Silvermont) */ + case 0x4d: /* Avaton / Rangely (Silvermont) */ + case 0x4c: /* Cherrytrail / Brasswell */ + case 0x4a: /* Merrifield */ + case 0x5a: /* Moorefield */ + case 0x5c: /* Goldmont */ + case 0x5f: /* Denverton */ + case 0x7a: /* Gemini Lake */ + return 0; + + /* + * Knights processors are not vulnerable. + */ + case 0x57: /* Knights Landing */ + case 0x85: /* Knights Mill */ + return 0; + + default: + printk("Unrecognised CPU model %#x - assuming vulnerable to LazyFPU\n", + boot_cpu_data.x86_model); + return 1; + } +} + #define OPT_XPTI_DEFAULT 0xff uint8_t __read_mostly opt_xpti = OPT_XPTI_DEFAULT; @@ -519,6 +603,10 @@ void __init init_speculation_mitigations(void) if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) ) opt_ibpb = 0; + /* Check whether Eager FPU should be enabled by default. */ + if ( opt_eager_fpu == -1 ) + opt_eager_fpu = should_use_eager_fpu(); + /* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */ init_shadow_spec_ctrl_state(); diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h index 45ac200..d81c855 100644 --- a/xen/include/asm-x86/spec_ctrl.h +++ b/xen/include/asm-x86/spec_ctrl.h @@ -28,6 +28,7 @@ void init_speculation_mitigations(void); extern bool_t opt_ibpb; extern bool_t opt_ssbd; +extern int8_t opt_eager_fpu; extern bool_t bsp_delay_spec_ctrl; extern uint8_t default_xen_spec_ctrl;
xsa267-4.8-livepatch.patch
(application/octet-stream, 4.3 KB)
From: Andrew Cooper <[email protected]> Subject: x86: Livepatch mitigations for LazyFPU This is part of XSA-267 / CVE-2018-3665 Signed-off-by: Andrew Cooper <[email protected]> diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index 86b098d..859be4d 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -10,6 +10,7 @@ #include <xen/config.h> #include <xen/sched.h> +#include <xen/livepatch_payload.h> #include <asm/current.h> #include <asm/processor.h> #include <asm/hvm/support.h> @@ -202,6 +203,94 @@ static inline void fpu_fxsave(struct vcpu *v) fpu_ctxt->x[FPU_WORD_SIZE_OFFSET] = fip_width; } +/* Calculate whether this CPU speculates past #NM */ +static bool should_use_eager_fpu(void) +{ + /* + * Assume all unrecognised processors are ok. This is only known to + * affect Intel Family 6 processors. + */ + if ( boot_cpu_data.x86_vendor != X86_VENDOR_INTEL || + boot_cpu_data.x86 != 6 ) + return false; + + switch ( boot_cpu_data.x86_model ) + { + /* + * Core processors since at least Nehalem are vulnerable. + */ + case 0x1e: /* Nehalem */ + case 0x1f: /* Auburndale / Havendale */ + case 0x1a: /* Nehalem EP */ + case 0x2e: /* Nehalem EX */ + case 0x25: /* Westmere */ + case 0x2c: /* Westmere EP */ + case 0x2f: /* Westmere EX */ + case 0x2a: /* SandyBridge */ + case 0x2d: /* SandyBridge EP/EX */ + case 0x3a: /* IvyBridge */ + case 0x3e: /* IvyBridge EP/EX */ + case 0x3c: /* Haswell */ + case 0x3f: /* Haswell EX/EP */ + case 0x45: /* Haswell D */ + case 0x46: /* Haswell H */ + case 0x3d: /* Broadwell */ + case 0x47: /* Broadwell H */ + case 0x4f: /* Broadwell EP/EX */ + case 0x56: /* Broadwell D */ + case 0x4e: /* Skylake M */ + case 0x55: /* Skylake X */ + case 0x5e: /* Skylake D */ + case 0x66: /* Cannonlake */ + case 0x67: /* Cannonlake? */ + case 0x8e: /* Kabylake M */ + case 0x9e: /* Kabylake D */ + return true; + + /* + * Atom processors are not vulnerable. + */ + case 0x1c: /* Pineview */ + case 0x26: /* Lincroft */ + case 0x27: /* Penwell */ + case 0x35: /* Cloverview */ + case 0x36: /* Cedarview */ + case 0x37: /* Baytrail / Valleyview (Silvermont) */ + case 0x4d: /* Avaton / Rangely (Silvermont) */ + case 0x4c: /* Cherrytrail / Brasswell */ + case 0x4a: /* Merrifield */ + case 0x5a: /* Moorefield */ + case 0x5c: /* Goldmont */ + case 0x5f: /* Denverton */ + case 0x7a: /* Gemini Lake */ + return false; + + /* + * Knights processors are not vulnerable. + */ + case 0x57: /* Knights Landing */ + case 0x85: /* Knights Mill */ + return false; + + default: + printk("Unrecognised CPU model %#x - assuming vulnerable to LazyFPU\n", + boot_cpu_data.x86_model); + return true; + } +} + +static bool fully_eager_fpu; + +void lazyfpu_apply_hook(void) +{ + fully_eager_fpu = should_use_eager_fpu(); + + printk("LazyFPU Livepatch: Fam %u, Model %u => Fully Eager %u\n", + boot_cpu_data.x86, boot_cpu_data.x86_model, fully_eager_fpu); +} + +LIVEPATCH_LOAD_HOOK(lazyfpu_apply_hook); + /*******************************/ /* VCPU FPU Functions */ /*******************************/ @@ -211,7 +300,7 @@ void vcpu_restore_fpu_eager(struct vcpu *v) ASSERT(!is_idle_vcpu(v)); /* Restore nonlazy extended state (i.e. parts not tracked by CR0.TS). */ - if ( !v->arch.nonlazy_xstate_used ) + if ( !fully_eager_fpu && !v->arch.nonlazy_xstate_used ) return; /* Avoid recursion */ @@ -222,11 +311,19 @@ void vcpu_restore_fpu_eager(struct vcpu *v) * above) we also need to restore full state, to prevent subsequently * saving state belonging to another vCPU. */ - if ( xstate_all(v) ) + if ( fully_eager_fpu || (v->arch.xsave_area && xstate_all(v)) ) { - fpu_xrstor(v, XSTATE_ALL); + if ( cpu_has_xsave ) + fpu_xrstor(v, XSTATE_ALL); + else + fpu_fxrstor(v); + v->fpu_initialised = 1; v->fpu_dirtied = 1; + + /* Xen doesn't need TS set, but the guest might. */ + if ( is_pv_vcpu(v) && (v->arch.pv_vcpu.ctrlreg[0] & X86_CR0_TS) ) + stts(); } else {
xsa267-4.9-1.patch
(application/octet-stream, 2.1 KB)
From: Andrew Cooper <[email protected]> Subject: x86: Support fully eager FPU context switching This is controlled on a per-vcpu bases for flexibility. This is part of XSA-267 / CVE-2018-3665 Signed-off-by: Andrew Cooper <[email protected]> Reviewed-by: Jan Beulich <[email protected]> diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index 9a172db..1da31af 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -210,7 +210,7 @@ void vcpu_restore_fpu_eager(struct vcpu *v) ASSERT(!is_idle_vcpu(v)); /* Restore nonlazy extended state (i.e. parts not tracked by CR0.TS). */ - if ( !v->arch.nonlazy_xstate_used ) + if ( !v->arch.fully_eager_fpu && !v->arch.nonlazy_xstate_used ) return; /* Avoid recursion */ @@ -221,11 +221,19 @@ void vcpu_restore_fpu_eager(struct vcpu *v) * above) we also need to restore full state, to prevent subsequently * saving state belonging to another vCPU. */ - if ( xstate_all(v) ) + if ( v->arch.fully_eager_fpu || (v->arch.xsave_area && xstate_all(v)) ) { - fpu_xrstor(v, XSTATE_ALL); + if ( cpu_has_xsave ) + fpu_xrstor(v, XSTATE_ALL); + else + fpu_fxrstor(v); + v->fpu_initialised = 1; v->fpu_dirtied = 1; + + /* Xen doesn't need TS set, but the guest might. */ + if ( is_pv_vcpu(v) && (v->arch.pv_vcpu.ctrlreg[0] & X86_CR0_TS) ) + stts(); } else { @@ -247,6 +255,8 @@ void vcpu_restore_fpu_lazy(struct vcpu *v) if ( v->fpu_dirtied ) return; + ASSERT(!v->arch.fully_eager_fpu); + if ( cpu_has_xsave ) fpu_xrstor(v, XSTATE_LAZY); else diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h index c08ddc0..702ec64 100644 --- a/xen/include/asm-x86/domain.h +++ b/xen/include/asm-x86/domain.h @@ -593,6 +593,9 @@ struct arch_vcpu * and thus should be saved/restored. */ bool_t nonlazy_xstate_used; + /* Restore all FPU state (lazy and non-lazy state) on context switch? */ + bool fully_eager_fpu; + /* Has the guest enabled CPUID faulting? */ bool cpuid_faulting;
xsa267-4.9-2.patch
(application/octet-stream, 8 KB)
From: Andrew Cooper <[email protected]> Subject: x86/spec-ctrl: Mitigations for LazyFPU Intel Core processors since at least Nehalem speculate past #NM, which is the mechanism by which lazy FPU context switching is implemented. On affected processors, Xen must use fully eager FPU context switching to prevent guests from being able to read FPU state (SSE/AVX/etc) from previously scheduled vcpus. This is part of XSA-267 / CVE-2018-3665 Signed-off-by: Andrew Cooper <[email protected]> Reviewed-by: Jan Beulich <[email protected]> diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown index 6300f69..52ed051 100644 --- a/docs/misc/xen-command-line.markdown +++ b/docs/misc/xen-command-line.markdown @@ -1635,7 +1635,7 @@ false disable the quirk workaround, which is also the default. ### spec-ctrl (x86) > `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>, -> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd}=<bool> ]` +> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd,eager-fpu}=<bool> ]` Controls for speculative execution sidechannel mitigations. By default, Xen will pick the most appropriate mitigations based on compiled in support, @@ -1685,6 +1685,11 @@ hardware, this is a global option applied at boot, and not virtualised for guest use. On Intel hardware, the feature is virtualised for guests, independently of Xen's choice of setting. +On all hardware, the `eager-fpu=` option can be used to force or prevent Xen +from using fully eager FPU context switches. This is currently implemented as +a global control. By default, Xen will choose to use fully eager context +switches on hardware believed to speculate past #NM exceptions. + ### sync\_console > `= <boolean>` diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index 1da31af..c3fda87 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -15,6 +15,7 @@ #include <asm/i387.h> #include <asm/xstate.h> #include <asm/asm_defns.h> +#include <asm/spec_ctrl.h> /*******************************/ /* FPU Restore Functions */ @@ -307,6 +308,8 @@ int vcpu_init_fpu(struct vcpu *v) { int rc; + v->arch.fully_eager_fpu = opt_eager_fpu; + if ( (rc = xstate_alloc_save_area(v)) != 0 ) return rc; diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index 8077000..ada4aac 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -44,6 +44,7 @@ static enum ind_thunk { static int8_t __initdata opt_ibrs = -1; bool __read_mostly opt_ibpb = true; bool __read_mostly opt_ssbd = false; +int8_t __read_mostly opt_eager_fpu = -1; bool __initdata bsp_delay_spec_ctrl; uint8_t __read_mostly default_xen_spec_ctrl; @@ -114,6 +115,7 @@ static int __init parse_spec_ctrl(char *s) opt_thunk = THUNK_JMP; opt_ibrs = 0; opt_ibpb = false; + opt_eager_fpu = 0; } else if ( val > 0 ) rc = -EINVAL; @@ -167,6 +169,8 @@ static int __init parse_spec_ctrl(char *s) opt_ibpb = val; else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 ) opt_ssbd = val; + else if ( (val = parse_boolean("eager-fpu", s, ss)) >= 0 ) + opt_eager_fpu = val; else rc = -EINVAL; @@ -223,15 +227,19 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps) * Alternatives blocks for protecting against and/or virtualising * mitigation support for guests. */ - printk(" Support for VMs: PV:%s%s%s, HVM:%s%s%s\n", + printk(" Support for VMs: PV:%s%s%s%s, HVM:%s%s%s%s\n", (boot_cpu_has(X86_FEATURE_SC_MSR_PV) || - boot_cpu_has(X86_FEATURE_SC_RSB_PV)) ? "" : " None", + boot_cpu_has(X86_FEATURE_SC_RSB_PV) || + opt_eager_fpu) ? "" : " None", boot_cpu_has(X86_FEATURE_SC_MSR_PV) ? " MSR_SPEC_CTRL" : "", boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB" : "", + opt_eager_fpu ? " EAGER_FPU" : "", (boot_cpu_has(X86_FEATURE_SC_MSR_HVM) || - boot_cpu_has(X86_FEATURE_SC_RSB_HVM)) ? "" : " None", + boot_cpu_has(X86_FEATURE_SC_RSB_HVM) || + opt_eager_fpu) ? "" : " None", boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ? " MSR_SPEC_CTRL" : "", - boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : ""); + boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : "", + opt_eager_fpu ? " EAGER_FPU" : ""); printk(" XPTI (64-bit PV only): Dom0 %s, DomU %s\n", opt_xpti & OPT_XPTI_DOM0 ? "enabled" : "disabled", @@ -321,6 +329,82 @@ static bool __init retpoline_safe(uint64_t caps) } } +/* Calculate whether this CPU speculates past #NM */ +static bool __init should_use_eager_fpu(void) +{ + /* + * Assume all unrecognised processors are ok. This is only known to + * affect Intel Family 6 processors. + */ + if ( boot_cpu_data.x86_vendor != X86_VENDOR_INTEL || + boot_cpu_data.x86 != 6 ) + return false; + + switch ( boot_cpu_data.x86_model ) + { + /* + * Core processors since at least Nehalem are vulnerable. + */ + case 0x1e: /* Nehalem */ + case 0x1f: /* Auburndale / Havendale */ + case 0x1a: /* Nehalem EP */ + case 0x2e: /* Nehalem EX */ + case 0x25: /* Westmere */ + case 0x2c: /* Westmere EP */ + case 0x2f: /* Westmere EX */ + case 0x2a: /* SandyBridge */ + case 0x2d: /* SandyBridge EP/EX */ + case 0x3a: /* IvyBridge */ + case 0x3e: /* IvyBridge EP/EX */ + case 0x3c: /* Haswell */ + case 0x3f: /* Haswell EX/EP */ + case 0x45: /* Haswell D */ + case 0x46: /* Haswell H */ + case 0x3d: /* Broadwell */ + case 0x47: /* Broadwell H */ + case 0x4f: /* Broadwell EP/EX */ + case 0x56: /* Broadwell D */ + case 0x4e: /* Skylake M */ + case 0x55: /* Skylake X */ + case 0x5e: /* Skylake D */ + case 0x66: /* Cannonlake */ + case 0x67: /* Cannonlake? */ + case 0x8e: /* Kabylake M */ + case 0x9e: /* Kabylake D */ + return true; + + /* + * Atom processors are not vulnerable. + */ + case 0x1c: /* Pineview */ + case 0x26: /* Lincroft */ + case 0x27: /* Penwell */ + case 0x35: /* Cloverview */ + case 0x36: /* Cedarview */ + case 0x37: /* Baytrail / Valleyview (Silvermont) */ + case 0x4d: /* Avaton / Rangely (Silvermont) */ + case 0x4c: /* Cherrytrail / Brasswell */ + case 0x4a: /* Merrifield */ + case 0x5a: /* Moorefield */ + case 0x5c: /* Goldmont */ + case 0x5f: /* Denverton */ + case 0x7a: /* Gemini Lake */ + return false; + + /* + * Knights processors are not vulnerable. + */ + case 0x57: /* Knights Landing */ + case 0x85: /* Knights Mill */ + return false; + + default: + printk("Unrecognised CPU model %#x - assuming vulnerable to LazyFPU\n", + boot_cpu_data.x86_model); + return true; + } +} + #define OPT_XPTI_DEFAULT 0xff uint8_t __read_mostly opt_xpti = OPT_XPTI_DEFAULT; @@ -519,6 +603,10 @@ void __init init_speculation_mitigations(void) if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) ) opt_ibpb = false; + /* Check whether Eager FPU should be enabled by default. */ + if ( opt_eager_fpu == -1 ) + opt_eager_fpu = should_use_eager_fpu(); + /* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */ init_shadow_spec_ctrl_state(); diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h index 91bed1b..5b40afb 100644 --- a/xen/include/asm-x86/spec_ctrl.h +++ b/xen/include/asm-x86/spec_ctrl.h @@ -28,6 +28,7 @@ void init_speculation_mitigations(void); extern bool opt_ibpb; extern bool opt_ssbd; +extern int8_t opt_eager_fpu; extern bool bsp_delay_spec_ctrl; extern uint8_t default_xen_spec_ctrl;
xsa267-4.10-1.patch
(application/octet-stream, 2.2 KB)
From: Andrew Cooper <[email protected]> Subject: x86: Support fully eager FPU context switching This is controlled on a per-vcpu bases for flexibility. This is part of XSA-267 / CVE-2018-3665 Signed-off-by: Andrew Cooper <[email protected]> Reviewed-by: Jan Beulich <[email protected]> diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index 8845252..50116d5 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -210,7 +210,7 @@ void vcpu_restore_fpu_eager(struct vcpu *v) ASSERT(!is_idle_vcpu(v)); /* Restore nonlazy extended state (i.e. parts not tracked by CR0.TS). */ - if ( !v->arch.nonlazy_xstate_used ) + if ( !v->arch.fully_eager_fpu && !v->arch.nonlazy_xstate_used ) return; /* Avoid recursion */ @@ -221,11 +221,19 @@ void vcpu_restore_fpu_eager(struct vcpu *v) * above) we also need to restore full state, to prevent subsequently * saving state belonging to another vCPU. */ - if ( xstate_all(v) ) + if ( v->arch.fully_eager_fpu || (v->arch.xsave_area && xstate_all(v)) ) { - fpu_xrstor(v, XSTATE_ALL); + if ( cpu_has_xsave ) + fpu_xrstor(v, XSTATE_ALL); + else + fpu_fxrstor(v); + v->fpu_initialised = 1; v->fpu_dirtied = 1; + + /* Xen doesn't need TS set, but the guest might. */ + if ( is_pv_vcpu(v) && (v->arch.pv_vcpu.ctrlreg[0] & X86_CR0_TS) ) + stts(); } else { @@ -247,6 +255,8 @@ void vcpu_restore_fpu_lazy(struct vcpu *v) if ( v->fpu_dirtied ) return; + ASSERT(!v->arch.fully_eager_fpu); + if ( cpu_has_xsave ) fpu_xrstor(v, XSTATE_LAZY); else diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h index 2ba21e1..7759332 100644 --- a/xen/include/asm-x86/domain.h +++ b/xen/include/asm-x86/domain.h @@ -569,6 +569,9 @@ struct arch_vcpu * and thus should be saved/restored. */ bool_t nonlazy_xstate_used; + /* Restore all FPU state (lazy and non-lazy state) on context switch? */ + bool fully_eager_fpu; + /* * The SMAP check policy when updating runstate_guest(v) and the * secondary system time.
xsa267-4.10-2.patch
(application/octet-stream, 8.1 KB)
From: Andrew Cooper <[email protected]> Subject: x86/spec-ctrl: Mitigations for LazyFPU Intel Core processors since at least Nehalem speculate past #NM, which is the mechanism by which lazy FPU context switching is implemented. On affected processors, Xen must use fully eager FPU context switching to prevent guests from being able to read FPU state (SSE/AVX/etc) from previously scheduled vcpus. This is part of XSA-267 / CVE-2018-3665 Signed-off-by: Andrew Cooper <[email protected]> Reviewed-by: Jan Beulich <[email protected]> diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown index 5c13f6f..cb81a9c 100644 --- a/docs/misc/xen-command-line.markdown +++ b/docs/misc/xen-command-line.markdown @@ -1727,7 +1727,7 @@ false disable the quirk workaround, which is also the default. ### spec-ctrl (x86) > `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>, -> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd}=<bool> ]` +> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd,eager-fpu}=<bool> ]` Controls for speculative execution sidechannel mitigations. By default, Xen will pick the most appropriate mitigations based on compiled in support, @@ -1777,6 +1777,11 @@ hardware, this is a global option applied at boot, and not virtualised for guest use. On Intel hardware, the feature is virtualised for guests, independently of Xen's choice of setting. +On all hardware, the `eager-fpu=` option can be used to force or prevent Xen +from using fully eager FPU context switches. This is currently implemented as +a global control. By default, Xen will choose to use fully eager context +switches on hardware believed to speculate past #NM exceptions. + ### sync\_console > `= <boolean>` diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index 50116d5..dbdf1b4 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -15,6 +15,7 @@ #include <asm/i387.h> #include <asm/xstate.h> #include <asm/asm_defns.h> +#include <asm/spec_ctrl.h> /*******************************/ /* FPU Restore Functions */ @@ -307,6 +308,8 @@ int vcpu_init_fpu(struct vcpu *v) { int rc; + v->arch.fully_eager_fpu = opt_eager_fpu; + if ( (rc = xstate_alloc_save_area(v)) != 0 ) return rc; diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index b5a7f9c..cf255ac 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -44,6 +44,7 @@ static enum ind_thunk { static int8_t __initdata opt_ibrs = -1; bool __read_mostly opt_ibpb = true; bool __read_mostly opt_ssbd = false; +int8_t __read_mostly opt_eager_fpu = -1; bool __initdata bsp_delay_spec_ctrl; uint8_t __read_mostly default_xen_spec_ctrl; @@ -114,6 +115,7 @@ static int __init parse_spec_ctrl(const char *s) opt_thunk = THUNK_JMP; opt_ibrs = 0; opt_ibpb = false; + opt_eager_fpu = 0; } else if ( val > 0 ) rc = -EINVAL; @@ -167,6 +169,8 @@ static int __init parse_spec_ctrl(const char *s) opt_ibpb = val; else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 ) opt_ssbd = val; + else if ( (val = parse_boolean("eager-fpu", s, ss)) >= 0 ) + opt_eager_fpu = val; else rc = -EINVAL; @@ -223,15 +227,19 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps) * Alternatives blocks for protecting against and/or virtualising * mitigation support for guests. */ - printk(" Support for VMs: PV:%s%s%s, HVM:%s%s%s\n", + printk(" Support for VMs: PV:%s%s%s%s, HVM:%s%s%s%s\n", (boot_cpu_has(X86_FEATURE_SC_MSR_PV) || - boot_cpu_has(X86_FEATURE_SC_RSB_PV)) ? "" : " None", + boot_cpu_has(X86_FEATURE_SC_RSB_PV) || + opt_eager_fpu) ? "" : " None", boot_cpu_has(X86_FEATURE_SC_MSR_PV) ? " MSR_SPEC_CTRL" : "", boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB" : "", + opt_eager_fpu ? " EAGER_FPU" : "", (boot_cpu_has(X86_FEATURE_SC_MSR_HVM) || - boot_cpu_has(X86_FEATURE_SC_RSB_HVM)) ? "" : " None", + boot_cpu_has(X86_FEATURE_SC_RSB_HVM) || + opt_eager_fpu) ? "" : " None", boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ? " MSR_SPEC_CTRL" : "", - boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : ""); + boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : "", + opt_eager_fpu ? " EAGER_FPU" : ""); printk(" XPTI (64-bit PV only): Dom0 %s, DomU %s\n", opt_xpti & OPT_XPTI_DOM0 ? "enabled" : "disabled", @@ -321,6 +329,82 @@ static bool __init retpoline_safe(uint64_t caps) } } +/* Calculate whether this CPU speculates past #NM */ +static bool __init should_use_eager_fpu(void) +{ + /* + * Assume all unrecognised processors are ok. This is only known to + * affect Intel Family 6 processors. + */ + if ( boot_cpu_data.x86_vendor != X86_VENDOR_INTEL || + boot_cpu_data.x86 != 6 ) + return false; + + switch ( boot_cpu_data.x86_model ) + { + /* + * Core processors since at least Nehalem are vulnerable. + */ + case 0x1e: /* Nehalem */ + case 0x1f: /* Auburndale / Havendale */ + case 0x1a: /* Nehalem EP */ + case 0x2e: /* Nehalem EX */ + case 0x25: /* Westmere */ + case 0x2c: /* Westmere EP */ + case 0x2f: /* Westmere EX */ + case 0x2a: /* SandyBridge */ + case 0x2d: /* SandyBridge EP/EX */ + case 0x3a: /* IvyBridge */ + case 0x3e: /* IvyBridge EP/EX */ + case 0x3c: /* Haswell */ + case 0x3f: /* Haswell EX/EP */ + case 0x45: /* Haswell D */ + case 0x46: /* Haswell H */ + case 0x3d: /* Broadwell */ + case 0x47: /* Broadwell H */ + case 0x4f: /* Broadwell EP/EX */ + case 0x56: /* Broadwell D */ + case 0x4e: /* Skylake M */ + case 0x55: /* Skylake X */ + case 0x5e: /* Skylake D */ + case 0x66: /* Cannonlake */ + case 0x67: /* Cannonlake? */ + case 0x8e: /* Kabylake M */ + case 0x9e: /* Kabylake D */ + return true; + + /* + * Atom processors are not vulnerable. + */ + case 0x1c: /* Pineview */ + case 0x26: /* Lincroft */ + case 0x27: /* Penwell */ + case 0x35: /* Cloverview */ + case 0x36: /* Cedarview */ + case 0x37: /* Baytrail / Valleyview (Silvermont) */ + case 0x4d: /* Avaton / Rangely (Silvermont) */ + case 0x4c: /* Cherrytrail / Brasswell */ + case 0x4a: /* Merrifield */ + case 0x5a: /* Moorefield */ + case 0x5c: /* Goldmont */ + case 0x5f: /* Denverton */ + case 0x7a: /* Gemini Lake */ + return false; + + /* + * Knights processors are not vulnerable. + */ + case 0x57: /* Knights Landing */ + case 0x85: /* Knights Mill */ + return false; + + default: + printk("Unrecognised CPU model %#x - assuming vulnerable to LazyFPU\n", + boot_cpu_data.x86_model); + return true; + } +} + #define OPT_XPTI_DEFAULT 0xff uint8_t __read_mostly opt_xpti = OPT_XPTI_DEFAULT; @@ -519,6 +603,10 @@ void __init init_speculation_mitigations(void) if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) ) opt_ibpb = false; + /* Check whether Eager FPU should be enabled by default. */ + if ( opt_eager_fpu == -1 ) + opt_eager_fpu = should_use_eager_fpu(); + /* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */ init_shadow_spec_ctrl_state(); diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h index 91bed1b..5b40afb 100644 --- a/xen/include/asm-x86/spec_ctrl.h +++ b/xen/include/asm-x86/spec_ctrl.h @@ -28,6 +28,7 @@ void init_speculation_mitigations(void); extern bool opt_ibpb; extern bool opt_ssbd; +extern int8_t opt_eager_fpu; extern bool bsp_delay_spec_ctrl; extern uint8_t default_xen_spec_ctrl;
xsa267-livepatch.patch
(application/octet-stream, 4.3 KB)
From: Andrew Cooper <[email protected]> Subject: x86: Livepatch mitigations for LazyFPU This is part of XSA-267 / CVE-2018-3665 Signed-off-by: Andrew Cooper <[email protected]> diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index 8845252..0961bfe 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -9,6 +9,7 @@ */ #include <xen/sched.h> +#include <xen/livepatch_payload.h> #include <asm/current.h> #include <asm/processor.h> #include <asm/hvm/support.h> @@ -201,6 +202,94 @@ static inline void fpu_fxsave(struct vcpu *v) fpu_ctxt->x[FPU_WORD_SIZE_OFFSET] = fip_width; } +/* Calculate whether this CPU speculates past #NM */ +static bool should_use_eager_fpu(void) +{ + /* + * Assume all unrecognised processors are ok. This is only known to + * affect Intel Family 6 processors. + */ + if ( boot_cpu_data.x86_vendor != X86_VENDOR_INTEL || + boot_cpu_data.x86 != 6 ) + return false; + + switch ( boot_cpu_data.x86_model ) + { + /* + * Core processors since at least Nehalem are vulnerable. + */ + case 0x1e: /* Nehalem */ + case 0x1f: /* Auburndale / Havendale */ + case 0x1a: /* Nehalem EP */ + case 0x2e: /* Nehalem EX */ + case 0x25: /* Westmere */ + case 0x2c: /* Westmere EP */ + case 0x2f: /* Westmere EX */ + case 0x2a: /* SandyBridge */ + case 0x2d: /* SandyBridge EP/EX */ + case 0x3a: /* IvyBridge */ + case 0x3e: /* IvyBridge EP/EX */ + case 0x3c: /* Haswell */ + case 0x3f: /* Haswell EX/EP */ + case 0x45: /* Haswell D */ + case 0x46: /* Haswell H */ + case 0x3d: /* Broadwell */ + case 0x47: /* Broadwell H */ + case 0x4f: /* Broadwell EP/EX */ + case 0x56: /* Broadwell D */ + case 0x4e: /* Skylake M */ + case 0x55: /* Skylake X */ + case 0x5e: /* Skylake D */ + case 0x66: /* Cannonlake */ + case 0x67: /* Cannonlake? */ + case 0x8e: /* Kabylake M */ + case 0x9e: /* Kabylake D */ + return true; + + /* + * Atom processors are not vulnerable. + */ + case 0x1c: /* Pineview */ + case 0x26: /* Lincroft */ + case 0x27: /* Penwell */ + case 0x35: /* Cloverview */ + case 0x36: /* Cedarview */ + case 0x37: /* Baytrail / Valleyview (Silvermont) */ + case 0x4d: /* Avaton / Rangely (Silvermont) */ + case 0x4c: /* Cherrytrail / Brasswell */ + case 0x4a: /* Merrifield */ + case 0x5a: /* Moorefield */ + case 0x5c: /* Goldmont */ + case 0x5f: /* Denverton */ + case 0x7a: /* Gemini Lake */ + return false; + + /* + * Knights processors are not vulnerable. + */ + case 0x57: /* Knights Landing */ + case 0x85: /* Knights Mill */ + return false; + + default: + printk("Unrecognised CPU model %#x - assuming vulnerable to LazyFPU\n", + boot_cpu_data.x86_model); + return true; + } +} + +static bool fully_eager_fpu; + +void lazyfpu_apply_hook(void) +{ + fully_eager_fpu = should_use_eager_fpu(); + + printk("LazyFPU Livepatch: Fam %u, Model %u => Fully Eager %u\n", + boot_cpu_data.x86, boot_cpu_data.x86_model, fully_eager_fpu); +} + +LIVEPATCH_LOAD_HOOK(lazyfpu_apply_hook); + /*******************************/ /* VCPU FPU Functions */ /*******************************/ @@ -210,7 +299,7 @@ void vcpu_restore_fpu_eager(struct vcpu *v) ASSERT(!is_idle_vcpu(v)); /* Restore nonlazy extended state (i.e. parts not tracked by CR0.TS). */ - if ( !v->arch.nonlazy_xstate_used ) + if ( !fully_eager_fpu && !v->arch.nonlazy_xstate_used ) return; /* Avoid recursion */ @@ -221,11 +310,19 @@ void vcpu_restore_fpu_eager(struct vcpu *v) * above) we also need to restore full state, to prevent subsequently * saving state belonging to another vCPU. */ - if ( xstate_all(v) ) + if ( fully_eager_fpu || (v->arch.xsave_area && xstate_all(v)) ) { - fpu_xrstor(v, XSTATE_ALL); + if ( cpu_has_xsave ) + fpu_xrstor(v, XSTATE_ALL); + else + fpu_fxrstor(v); + v->fpu_initialised = 1; v->fpu_dirtied = 1; + + /* Xen doesn't need TS set, but the guest might. */ + if ( is_pv_vcpu(v) && (v->arch.pv_vcpu.ctrlreg[0] & X86_CR0_TS) ) + stts(); } else {