[REGRESSION][BISECTED] Long boot time with Xen HVM guests during PV spinlock initialization
Teddy Astie <[email protected]>
| Newsgroups | dev.linux.lists.regressions,org.kernel.vger.linux-kernel,org.xenproject.lists.xen-devel |
|---|---|
| Message-ID | <1780914594.8631fc262581453bbf619ec5b2062170.19ea6c8227b000701b@vates.tech> |
Hello,
In 6.12.5+ kernels on AMD CPUs, we observe abnormally long boot times
where the guest is struggling on PV spinlock initialization.
This occurs starting with 6.12.5, and also on more recent kernels on
Intel platforms, but that hasn't been fully investigated at this time
(but I assume it's a variant of the same issue).
This occurs since a backport of 76031d9 ("clocksource: Make negative
motion detection more robust").
Some (claude-based) analysis made appears to relate that to the lack of
proper max_raw_delta in the jiffies clocksource which appears to make
the clock fail to progress meaningfully.
Here is a raw summary of the analysis
> We tracked it down to a single stable backport in 6.12.5: commit
1a678f6829a8 ("clocksource: Make negative motion detection more robust",
upstream 76031d9536a0). It introduces a max_raw_delta field on struct
clocksource but never initializes it for the default boot timekeeper
(the jiffies clocksource), so clocksource_delta() clamps every delta to
0 and CLOCK_MONOTONIC freezes while that clocksource is active. On this
HVM guest, SMP bring-up runs while the jiffies clocksource is still the
timekeeper, and the Xen single shot (high resolution) tick then advances
jiffies far too slowly, so the secondary CPUs burn seconds in
calibrate_delay().
Teddy
OpenPGP_0x660FA9D102CBCFD0.asc
(application/pgp-keys, 2.4 KB)
-----BEGIN PGP PUBLIC KEY BLOCK----- xsDNBGn5sK8BDACuzSrrTjpVf4ay06OYB6yY0J1PqKffihoNMtrQRZjAHxoAPC7L TBVHV/XOZw5HJc+9R71z1JV+iYg6z3jPziGKzX8Fj3ZXlzJPmpf1PuETH3KdbvtJ T4ny+OGntnJntUoRKRPhTirr6yNeBk/637O3CQXjtqFUPZnko8OI/o1yawIBhJJA WicutjkkUgd28Bh6HV9EIumHtCBgn5/1A/fpm9624MMgYLsA8qjC4XsoovQvFCaO 8HEhvfzrrTZHjn/nPeB9SigxIxXW8YaTVqMdqul07o72m3eA2mf+LMu9a04FX/d4 wbxBLtELm+1jIrbtyaFZEMOLv/haSiS/Lj3btJH/EoucejoZ5SH49ksmVAmKOLkt OaTQ8b2gEvP7iaKiIiszCCtOSRohr+2GvDsDeLvVZnlR3I+SPhHar7TPKjFz0G3D PNolyjXywNqOAMpomSPi8lSwjAFsxOtQbcck/qRGRSNk4DAmH70pA+89MXfQXZ3q t1Q01B1+sU0I8xsAEQEAAc0kVGVkZHkgQXN0aWUgPHRlZGR5LmFzdGllQHZhdGVz LnRlY2g+wsENBBMBCAA3FiEEGAIew9LzHY3pdrqtZg+p0QLLz9AFAmn5sK8FCQWj moACGwMECwkIBwUVCAkKCwUWAgMBAAAKCRBmD6nRAsvP0ID6DACGOktArFbLKHNz uyOVCskwfUZPla6Zpd3GZ8r61SrAKePIr2BnpgPkd0hV3bSRkRLIrgjzR2NRCzfp 0x0HfuhcYfAYPR46XHTvjaJEv99sT/vGUG1BZguYDOScSEpgSNaNlYum3RKZbMuR OxdK8G+YHccJY8PvWSq2K2yiae2KGiAv1yjnZxug9/PtDfX8vQFUSg2w1ukRDf50 wvDohN1zUQfFtofOP2xCRsDZiHAlQ0pF+aUjXQhPeP3IdpfWc8cyRLXF06Rk46YM YCytweGtGdHcqAfrVthl84129ZPN422k/voW0sm14gjYlGcTUwgnYlFRk2FLq0Qe KEDcS0aj3o3EVAQCrayoGzi1pnlIKE3PRGUcUzjGVvzQ/po24gOjwba9Egr/Wmu3 MQlx/7A8zT5QBzF/n+RYdLNQ0Eu6YnUwf0Z1uieqNaon+olyIRFiLb/hCZHO6ekN f5vrm2clHUbQAYaPQebknujoKBo6ZLHg0WM1gZS01Gz+aUpKsUfOwM0EafmwsAEM AKiQiZa3yQMmc/h3sDbfVHPSiBA4IMI/NAB7IotzPHq1GzCpsoVILAhF/INbWjxJ 3DbVf+en3/FvdVZg2S38xtnth0njNdlVKpyxm054phKjbdoFDwaknWolS4hrddTm etSG5/52AjtmPFtlXAk0NmLvfJnW3seXVQbgM7sW/MNXPP5UKDpkGnLhnvej+GU0 s3109sJeXT5ImVdphFs9cvyZyBT9t1PbRowv58EgV0zE4hbAeVkULAbxFV5b/ExT jjGVHoX7CVhWxvCiTqCUoXZRkUE9C3FnkzEFRkKbYu6NCfiHfEyB3Xyg9hfdrRgj MRq907zCof+nDtWxGz1MSEuvTj1g9GZ049Bennqzjc/Q+0ovXoK4jm+Py0FiUGUa A6yhexficjH+kCR/xDbVnWrMhSLB4AuTBT9HjfZI6gk3uYLhoT8Pig4/eVtR2Q1w ZIJsFToR6ofGuyECwFcs+PUXN7fmGRSiPXgjAr/zIUBdW0VWCE3OGPNqtRk2E5s6 IQARAQABwsD8BBgBCAAmFiEEGAIew9LzHY3pdrqtZg+p0QLLz9AFAmn5sLAFCQWj moACGwwACgkQZg+p0QLLz9DncQwAg76IehTemLIfrB8T9WIBZrI4kUV7G7a4rjiV oUiHYN5QwhnbZnsaJDlt+Ezoqy/510eo2bCSzvW5xXYPgyjcuOPwgQo1Qp764Qxy X6rld2f2RcWkDuBHun55ZWXjby8o21ginPRwruBVYY5rVf3DV1iBu4NurUeHtyFk /dS0XTOQi2wVUb17sW/+ybCEokdVacZGzOqP/OmwHrF8ylXlXnhQq6e3r+J+T8fu oGJelm/CJiMwyP6cEWE8sxVqX/iqwjwUYkuOCpE+lOWSvdNHgoEkWR0RXBPQjnGm LKbfTl/QDXLk6NP2/r9uxm2HL6Ei3QJKSEdrp+XZaVnk/OffO485NOTKwGOxyWb0 06cTMh53xPkAJFQu4Tvdj+odsHz88jqw5wfPG0BYWx0I/FspYj7N9kZR8ULR9nX0 LvpzJ/kB4NgHIUt8YtIL6ZSfM2dbF7fKzvx1UqFfvozJZwFzfEieJLXa4nlGgR6D x9fhaZEsniw8/bYgC3igkk5YJiOa =lUIA -----END PGP PUBLIC KEY BLOCK-----
OpenPGP_signature.asc
(application/pgp-signature, 665 B)
-----BEGIN PGP SIGNATURE----- wsD5BAABCAAjFiEEGAIew9LzHY3pdrqtZg+p0QLLz9AFAmommaEFAwAAAAAACgkQZg+p0QLLz9Dp KQwAoMrJWZgNTP6C30II34gUK3f+q1u+It7m5LrXITvEAWiJKvyNOBIyy/RiLrqoVXvA8juoYVL5 k9Qp/A/2IJxDgiIDHOW2/GrnDWbOIpdmc6Ct8mCLH4ZNn5bGnqslFLFwrbCDqVamAv/rEZ5mkdpI 5XBfqxTOQVr7L0jKg/3nKGhqsYZAg9DvorJ+69wNhaGDH2lMpTdtRWIXf5IEP8kv+T1F0HRs5tsR ZSqNisOIw9JrMGGMoaLVtvAvWDQn/Yl8whrgZ8p/BTwS6VzlySpryepBFGpvGYDG/ej2BjHqCWIa kGqDAsqIFY7OBNLvoV+Teo2/rjD80ILZjP/GXC1zjPd+Z8M0k5vtMAtNoBPyACWVdviW9tflAwc4 ZE0ay+QvC2vvuBtvhNEk/ngtiSfLf60+E/jJubc/Jrzbm+OkCse/9N109/IgnOK1/coGnubC5GxK DmdY1eVQRfifngzc5QZL0WR/6d13oO9klxMJRxA+5n+RPDv7RAzhI9ZBV2IP =cR7M -----END PGP SIGNATURE-----