Re: BUG: unable to handle kernel paging request at 00007ffd7ce07800

"Douglas Renton" <[email protected]> Wed, 23 Nov 2016 12:54:00 -0500
Newsgroups gmane.linux.real-time.rtai
Message-ID <[email protected]>
And just for completeness, 12 cpu/cores back to the long(er) delays
So there does seem to be an issue with 12 CPUs


linux-36hs:/usr/realtime/testsuite/kern/latency # ./run
*
*
* Type ^C to stop this application.
*
*
RTAI Testsuite - KERNEL space latency test (output data in nanoseconds)

*** latency verification tool with RTAI own real time kernel tasks ***
***    period = 100000 (ns),  avrgtime = 1 (s)    ***

RTH|    lat min|    ovl min|    lat avg|    lat max|    ovl max|   overruns
RTD|        228|        228|       1211|      18565|      18565|       1317
RTD|        169|        169|       1202|      20514|      20514|       1317
RTD|        226|        169|       1210|      22343|      22343|       1317
RTD|        296|        169|       1207|      16967|      22343|       1317
RTD|        266|        169|       1207|      23541|      23541|       1317
^CRTD|        266|        169|       1207|      23541|      23541|       1317

linux-36hs:/usr/realtime/testsuite/kern/latency #

linux-36hs:/home/doug # dmesg
[  671.257925] I-pipe: head domain RTAI registered.
[  671.257928] RTAI[hal]: mounted. ISOL_CPUS_MASK: 0.
[  671.257930] SYSINFO - # CPUs: 12, TIMER NAME: 'lapic', TIMER IRQ: 4353,
TIMER FREQ: 6246329, CLOCK NAME: 'tsc', CLOCK FREQ: 3398027000, CPU FREQ:
3398027000, LINUX TIMER IRQ: 4353.
[  671.267951] RTAI[malloc]: global heap size = 4194304 bytes, <BSD>.
[  671.278691] , kstacks pool size = 524288 bytes.
[  671.278694] RTAI[sched]: hard timer type/freq = lapic/6246329(Hz);
timing: oneshot; linear timed lists.
[  671.278695] RTAI[sched]: Linux timer freq = 1000 (Hz), TimeBase freq =
3398027000 hz.
[  671.278696] RTAI[sched]: timer setup = 160 ns, resched latency = 0 ns.
[  671.289332] BUG: unable to handle kernel paging request at
00007ffe7b6c3aa0
[  671.289343] IP: [<ffffffffa0688631>] handle_lxrt_request+0x131/0x7d0
[rtai_sched]
[  671.289353] PGD 7f3fb9067 PUD 798a23067 PMD 7bae93067 PTE 80000007ce198867
[  671.289361] Oops: 0001 [#2] PREEMPT SMP
[  671.289367] Modules linked in: rtai_sched(O+) rtai_malloc(PO)
rtai_hal(O) st sr_mod cdrom fuse af_packet snd_hda_codec_hdmi
snd_hda_codec_realtek snd_hda_codec_generic xfs libcrc32c
x86_pkg_temp_thermal coretemp kvm iTCO_wdt iTCO_vendor_support mxm_wmi
snd_hda_intel snd_hda_controller snd_hda_codec crct10dif_pclmul
crc32_pclmul snd_hwdep ghash_clmulni_intel snd_pcm aesni_intel aes_x86_64
lrw gf128mul glue_helper ablk_helper cryptd snd_timer e1000e serio_raw snd
i2c_i801 xhci_pci ptp mei_me xhci_hcd lpc_ich mei mfd_core pps_core
soundcore shpchp dm_mod btrfs xor raid6_pq nouveau video i2c_algo_bit
crc32c_intel drm_kms_helper ttm drm wmi sg [last unloaded: rtai_hal]
[  671.289460] CPU: 3 PID: 4711 Comm: calibrate Tainted: P      D    O  
3.18.22-STRADM #12
[  671.289464] Hardware name: ASUS All Series/X99-A II, BIOS 0801 06/28/2016
[  671.289468] task: ffff8800cab40190 ti: ffff88078c53c000 task.ti:
ffff88078c53c000
[  671.289472] RIP: 0010:[<ffffffffa0688631>]  [<ffffffffa0688631>]
handle_lxrt_request+0x131/0x7d0 [rtai_sched]
[  671.289481] RSP: 0018:ffff88078c53fe78  EFLAGS: 00010206
[  671.289485] RAX: ffff88078c53c000 RBX: 00007ffe7b6c3aa0 RCX:
0000000000000407
[  671.289489] RDX: 0000000000000000 RSI: 00007ffe7b6c3aa0 RDI:
0000000000407818
[  671.289492] RBP: ffff88078c53fec8 R08: 00007ffe7b6c3f69 R09:
00007ffe00000001
[  671.289496] R10: 00007ffe7b6c3a68 R11: 0000000000000202 R12:
0000000000000000
[  671.289500] R13: 0000000000407818 R14: 0000000000000000 R15:
0000000000000000
[  671.289504] FS:  00007f20f4ac3700(0000) GS:ffff88081f2c0000(0000)
knlGS:0000000000000000
[  671.289508] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  671.289511] CR2: 00007ffe7b6c3aa0 CR3: 000000078c642000 CR4:
00000000003407e0
[  671.289515] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[  671.289519] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
[  671.289523] I-pipe domain Linux
[  671.289525] Stack:
[  671.289527]  00000000000002a6 ffff880700000000 0000000000000000
ffff8807e9565a60
[  671.289536]  00000000e9565800 ffff8807e9565a48 00007ffe7b6c3aa0
0000000000000000
[  671.289543]  0000000000407818 0000000000000000 ffff88078c53ff38
ffffffffa0688ec0
[  671.289550] Call Trace:
[  671.289556]  [<ffffffffa0688ec0>] rtai_lxrt_invoke+0x1f0/0x900
[rtai_sched]
[  671.290561]  [<ffffffff810ea996>] ? __audit_syscall_exit+0x236/0x2e0
[  671.291794]  [<ffffffffa06872b5>] lxrt_intercept_syscall+0xc5/0x190
[rtai_sched]
[  671.293022]  [<ffffffff81602a18>] ? int_check_syscall_exit_work+0x34/0x3d
[  671.294245]  [<ffffffff811005d9>] __ipipe_notify_syscall+0x19/0x20
[  671.295430]  [<ffffffff8160268e>] pipeline_syscall+0xa/0x17
[  671.296577] Code: 89 04 24 31 c0 41 ff 52 08 48 83 c4 30 5b 41 5c 41 5d
41 5e 5d c3 48 c7 c7 b8 16 69 a0 e8 e8 e3 03 00 48 c7 c0 da ff ff ff eb 96
<4c> 8b 2e 8d 91 18 fc ff ff 83 fa 20 4c 89 e8 0f 87 85 00 00 00
[  671.297767] RIP  [<ffffffffa0688631>] handle_lxrt_request+0x131/0x7d0
[rtai_sched]
[  671.298930]  RSP <ffff88078c53fe78>
[  671.300061] CR2: 00007ffe7b6c3aa0
[  671.301175] ---[ end trace 3f395e14704b000b ]---
[  671.301227] USERMODE CHECK: ERROR.
[  671.301228] USERMODE CHECK PROVIDED (ns): KernelLatency 0, UserLatency 0.
[  671.301229] FINAL CALIBRATION SUMMARY (ns): KernelLatency 0,
UserLatency 0.
[  678.091058]

CPU USE SUMMARY
[  678.091063] # 0 -> 67747
[  678.091064] # 1 -> 0
[  678.091065] # 2 -> 0
[  678.091066] # 3 -> 0
[  678.091066] # 4 -> 0
[  678.091067] # 5 -> 0
[  678.091068] # 6 -> 0
[  678.091068] # 7 -> 0
[  678.091069] # 8 -> 0
[  678.091070] # 9 -> 0
[  678.091070] # 10 -> 0
[  678.091071] # 11 -> 0
[  678.091072] END OF CPU USE SUMMARY

[  678.840460] RTAI[sched]: unloaded (forced hard/soft/hard transitions:
traps 0, syscalls 0).
[  678.846013] RTAI[malloc]: unloaded.
[  678.851227] I-pipe: head domain RTAI unregistered.
[  678.851235] RTAI[hal]: unmounted.
linux-36hs:/home/doug #





On Wed, November 23, 2016 12:13 pm, Douglas Renton wrote:
> OK, 8 Cores is the same as 4 as far as I can see, Much Better than 12! :(
>  *
> *
> * Type ^C to stop this application.
> *
> *
> RTAI Testsuite - KERNEL space latency test (output data in nanoseconds)
>
>
> *** latency verification tool with RTAI own real time kernel tasks ***
> ***    period = 100000 (ns),  avrgtime = 1 (s)    ***
>
>
> RTH|    lat min|    ovl min|    lat avg|    lat max|    ovl max|
> overruns RTD|        245|        245|        459|       2168|       2168|
> 0
> RTD|        198|        198|        468|       2985|       2985|
> 0
> RTD|        294|        198|        455|       2538|       2985|
> 0
> RTD|        236|        198|        455|       2443|       2985|
> 0
> RTD|        240|        198|        455|       2851|       2985|
> 0
> RTD|        311|        198|        455|       2559|       2985|
> 0
> ^CRTD|        311|        198|        455|       2559|       2985|
> 0
> ^C^C
> linux-36hs:/usr/realtime/testsuite/kern/latency #
>
>
>
> linux-36hs:/usr/realtime/testsuite/kern/latency # dmesg
> [  311.385854] I-pipe: head domain RTAI registered.
> [  311.385861] RTAI[hal]: mounted. ISOL_CPUS_MASK: 0.
> [  311.385863] SYSINFO - # CPUs: 8, TIMER NAME: 'lapic', TIMER IRQ: 4353,
> TIMER FREQ: 6246289, CLOCK NAME: 'tsc', CLOCK FREQ: 3398151000, CPU FREQ:
> 3398151000, LINUX TIMER IRQ: 4353.
> [  311.394919] RTAI[malloc]: global heap size = 4194304 bytes, <BSD>.
> [  311.404474] , kstacks pool size = 524288 bytes.
> [  311.404477] RTAI[sched]: hard timer type/freq = lapic/6246289(Hz);
> timing: oneshot; linear timed lists.
> [  311.404478] RTAI[sched]: Linux timer freq = 1000 (Hz), TimeBase freq =
> 3398151000 hz.
> [  311.404479] RTAI[sched]: timer setup = 160 ns, resched latency = 0 ns.
> [  313.153812] BUG: unable to handle kernel paging request at
> 00007ffc0b6f4520
> [  313.154290] IP: [<ffffffffa0581611>] handle_lxrt_request+0x131/0x7d0
> [rtai_sched]
> [  313.155398] PGD 7cbf2b067 PUD 7cc186067 PMD 7cc187067 PTE
> 80000007cb080867
> [  313.156513] Oops: 0001 [#3] PREEMPT SMP
> [  313.157627] Modules linked in: rtai_sched(O+) rtai_malloc(PO)
> rtai_hal(O) fuse af_packet snd_hda_codec_hdmi snd_hda_codec_realtek
> snd_hda_codec_generic xfs libcrc32c x86_pkg_temp_thermal coretemp kvm
> iTCO_wdt iTCO_vendor_support mxm_wmi snd_hda_intel snd_hda_controller
> crct10dif_pclmul snd_hda_codec crc32_pclmul ghash_clmulni_intel snd_hwdep
>  snd_pcm aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper
> cryptd snd_timer e1000e snd serio_raw xhci_pci mei_me i2c_i801 ptp
> xhci_hcd mei pps_core soundcore shpchp lpc_ich mfd_core dm_mod btrfs xor
> raid6_pq crc32c_intel nouveau video i2c_algo_bit drm_kms_helper ttm drm
> wmi sg [last unloaded: rtai_hal]
> [  313.161364] CPU: 4 PID: 2360 Comm: calibrate Tainted: P      D    O
> 3.18.22-STRADM #11
> [  313.162649] Hardware name: ASUS All Series/X99-A II, BIOS 0801
> 06/28/2016
> [  313.163927] task: ffff8807cbcd6650 ti: ffff8807cc0a0000 task.ti:
> ffff8807cc0a0000 [  313.165209] RIP: 0010:[<ffffffffa0581611>]
> [<ffffffffa0581611>]
> handle_lxrt_request+0x131/0x7d0 [rtai_sched] [  313.166509] RSP:
> 0018:ffff8807cc0a3e78  EFLAGS: 00010206
> [  313.167819] RAX: ffff8807cc0a0000 RBX: 00007ffc0b6f4520 RCX:
> 0000000000000407
> [  313.169141] RDX: 0000000000000000 RSI: 00007ffc0b6f4520 RDI:
> 0000000000407818
> [  313.170472] RBP: ffff8807cc0a3ec8 R08: 00007ffc0b6f4f69 R09:
> 00007ffc00000001
> [  313.171794] R10: 00007ffc0b6f44e8 R11: 0000000000000206 R12:
> 0000000000000000
> [  313.173109] R13: 0000000000407818 R14: 0000000000000000 R15:
> 0000000000000000
> [  313.174419] FS:  00007fbabb2b8700(0000) GS:ffff88081fd00000(0000)
> knlGS:0000000000000000
> [  313.175726] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [  313.177028] CR2: 00007ffc0b6f4520 CR3: 00000007cbf2a000 CR4:
> 00000000003407e0
> [  313.178342] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [  313.179655] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
> 0000000000000400
> [  313.180959] I-pipe domain Linux
> [  313.182245] Stack:
> [  313.183530]  00000000000002a6 ffff880700000000 0000000000000000
> ffff8807ec4aee60 [  313.184807]  00000000ec4aec00 ffff8807ec4aee48
> 00007ffc0b6f4520
> 0000000000000000
> [  313.186062]  0000000000407818 0000000000000000 ffff8807cc0a3f38
> ffffffffa0581ea0 [  313.187312] Call Trace:
> [  313.188563]  [<ffffffffa0581ea0>] rtai_lxrt_invoke+0x1f0/0x900
> [rtai_sched]
> [  313.189819]  [<ffffffff810ea7c6>] ? __audit_syscall_exit+0x236/0x2e0
> [  313.191062]  [<ffffffffa0580295>] lxrt_intercept_syscall+0xc5/0x190
> [rtai_sched]
> [  313.192307]  [<ffffffff816026d8>] ?
> int_check_syscall_exit_work+0x34/0x3d [  313.193546]  [<ffffffff811003a9>]
> __ipipe_notify_syscall+0x19/0x20
> [  313.194748]  [<ffffffff8160234e>] pipeline_syscall+0xa/0x17
> [  313.195908] Code: 89 04 24 31 c0 41 ff 52 08 48 83 c4 30 5b 41 5c 41 5d
>  41 5e 5d c3 48 c7 c7 b8 a6 58 a0 e8 08 c4 12 00 48 c7 c0 da ff ff ff eb
> 96
> <4c> 8b 2e 8d 91 18 fc ff ff 83 fa 20 4c 89 e8 0f 87 85 00 00 00
> [  313.197113] RIP  [<ffffffffa0581611>] handle_lxrt_request+0x131/0x7d0
> [rtai_sched]
> [  313.198282]  RSP <ffff8807cc0a3e78>
> [  313.199422] CR2: 00007ffc0b6f4520
> [  313.200555] ---[ end trace c6ad8498e3b44420 ]---
> [  313.200606] USERMODE CHECK: ERROR.
> [  313.200608] USERMODE CHECK PROVIDED (ns): KernelLatency 0, UserLatency
> 0.
> [  313.200609] FINAL CALIBRATION SUMMARY (ns): KernelLatency 0,
> UserLatency 0.
> [  321.189508]
>
>
> CPU USE SUMMARY
> [  321.189512] # 0 -> 79722
> [  321.189513] # 1 -> 0
> [  321.189514] # 2 -> 0
> [  321.189514] # 3 -> 0
> [  321.189515] # 4 -> 0
> [  321.189516] # 5 -> 0
> [  321.189516] # 6 -> 0
> [  321.189517] # 7 -> 0
> [  321.189518] END OF CPU USE SUMMARY
>
>
> [  326.447805] RTAI[sched]: unloaded (forced hard/soft/hard transitions:
> traps 0, syscalls 0). [  326.453228] RTAI[malloc]: unloaded.
> [  326.458443] I-pipe: head domain RTAI unregistered.
> [  326.458450] RTAI[hal]: unmounted.
> linux-36hs:/usr/realtime/testsuite/kern/latency #
>
>
>


_______________________________________________
Rtai mailing list
[email protected]
https://mail.rtai.org/cgi-bin/mailman/listinfo/rtai