Re: BUG: unable to handle kernel paging request at 00007ffd7ce07800

I've seen the same problem in a few instances, where it was due to the 
need of warming up large caches.
In your case there are a few things I find strange (unclear to me at least):
- I do not see why there should be any overrun for an ovl max of 26094 
nanos, within a period of 100000 nanos;
- a likely unharmful page fault
- (not so strange) it is possible that RTAI cannot see more than 8 CPUs 
correctly; you should try and see what happens with a number of cpus  < 
11,  e.g. up to 8 in step of 2.
Paolo

On 11/23/2016 08:09 PM, Doug Renton wrote:
> I should add I'm running rtai-5.0-test1.tar.bz2
> Have there been changes not yet included in this? Should I try the CVS 
> version?
>
> On 11/23/2016 1:38 PM, Doug Renton wrote:
>> Hi All,
>>   I'm trying to setup my nre computer from "Question about 
>> CPU/Motherboard"
>> Running the latency test has some startup overruns, but seems ok after
>> that. I've run it for hours, and it is OK.  dmesg shows a number of
>> errors. I've tried lots of various compile options, and this error 
>> stays.
>>   Also the calibrate test absolutely fails. The second dmesg is from 
>> that.
>>    Any ideas?
>> Thanks,
>> Doug
>>
>>
>> *
>> *
>> * Type ^C to stop this application.
>> *
>> *
>> RTAI Testsuite - KERNEL space latency test (output data in nanoseconds)
>>
>> *** latency verification tool with RTAI own real time kernel tasks ***
>> ***    period = 100000 (ns),  avrgtime = 1 (s)    ***
>>
>> RTH|    lat min|    ovl min|    lat avg|    lat max|    ovl max|   
>> overruns
>> RTD|        510|        510|       1205|      15184| 15184|       2576
>> RTD|        205|        205|       1213|      22673| 22673|       2576
>> RTD|        258|        205|       1198|      20373| 22673|       2576
>> RTD|        510|        205|       1196|      14014| 22673|       2576
>> RTD|        255|        205|       1197|      15051| 22673|       2576
>> RTD|        306|        205|       1195|       8734| 22673|       2576
>> RTD|        510|        205|       1201|      14104| 22673|       2576
>> RTD|        512|        205|       1209|      21560| 22673|       2576
>> RTD|        362|        205|       1195|      23104| 23104|       2576
>> RTD|        357|        205|       1201|      15228| 23104|       2576
>> RTD|        221|        205|       1204|      26094| 26094|       2576
>> ^CRTD|        221|        205|       1204|      26094| 26094| 2576
>>
>>
>>
>> linux-36hs:/usr/realtime # dmesg -C
>> linux-36hs:/usr/realtime # dmesg
>> [  842.342715] I-pipe: head domain RTAI registered.
>> [  842.342719] RTAI[hal]: mounted. ISOL_CPUS_MASK: 0.
>> [  842.342720] SYSINFO - # CPUs: 12, TIMER NAME: 'lapic', TIMER IRQ:
>> 4353, TIMER FREQ: 6246289, CLOCK NAME: 'tsc', CLOCK FREQ: 3398323000,
>> CPU FREQ: 3398323000, LINUX TIMER IRQ: 4353.
>> [  842.353665] RTAI[malloc]: global heap size = 4194304 bytes, <BSD>.
>> [  842.365013] , kstacks pool size = 524288 bytes.
>> [  842.365016] RTAI[sched]: hard timer type/freq = lapic/6246289(Hz);
>> timing: oneshot; linear timed lists.
>> [  842.365018] RTAI[sched]: Linux timer freq = 1000 (Hz), TimeBase freq
>> = 3398323000 hz.
>> [  842.365019] RTAI[sched]: timer setup = 160 ns, resched latency = 0 
>> ns.
>> [  842.365476] BUG: unable to handle kernel paging request at
>> 00007ffd7ce07800
>> [  842.365986] IP: [<ffffffffa04e8631>] handle_lxrt_request+0x131/0x7d0
>> [rtai_sched]
>> [  842.367090] PGD 7c85d0067 PUD 7e9a7d067 PMD 7e6c4a067 PTE
>> 80000007e3613867
>> [  842.368221] Oops: 0001 [#4] PREEMPT SMP
>> [  842.369361] Modules linked in: rtai_sched(O+) rtai_malloc(PO)
>> rtai_hal(O) fuse af_packet snd_hda_codec_hdmi snd_hda_codec_realtek
>> snd_hda_codec_generic xfs libcrc32c x86_pkg_temp_thermal coretemp kvm
>> iTCO_wdt iTCO_vendor_support mxm_wmi snd_hda_intel snd_hda_controller
>> snd_hda_codec snd_hwdep crct10dif_pclmul crc32_pclmul
>> ghash_clmulni_intel snd_pcm aesni_intel aes_x86_64 lrw gf128mul
>> glue_helper ablk_helper cryptd snd_timer serio_raw e1000e snd mei_me mei
>> xhci_pci i2c_i801 ptp xhci_hcd pps_core lpc_ich shpchp soundcore
>> mfd_core dm_mod btrfs xor raid6_pq nouveau crc32c_intel video
>> i2c_algo_bit drm_kms_helper ttm drm wmi sg [last unloaded: rtai_hal]
>> [  842.373165] CPU: 3 PID: 2632 Comm: calibrate Tainted: P D    O
>> 3.18.22-STRADM #9
>> [  842.374469] Hardware name: ASUS All Series/X99-A II, BIOS 0801
>> 06/28/2016
>> [  842.375776] task: ffff8807e75982d0 ti: ffff8807c8c84000 task.ti:
>> ffff8807c8c84000
>> [  842.377073] RIP: 0010:[<ffffffffa04e8631>] [<ffffffffa04e8631>]
>> handle_lxrt_request+0x131/0x7d0 [rtai_sched]
>> [  842.378397] RSP: 0018:ffff8807c8c87e78  EFLAGS: 00010206
>> [  842.379730] RAX: ffff8807c8c84000 RBX: 00007ffd7ce07800 RCX:
>> 0000000000000407
>> [  842.381076] RDX: 0000000000000000 RSI: 00007ffd7ce07800 RDI:
>> 0000000000407818
>> [  842.382429] RBP: ffff8807c8c87ec8 R08: 00007ffd7ce08f69 R09:
>> 00007ffd00000001
>> [  842.383766] R10: 00007ffd7ce077c8 R11: 0000000000000202 R12:
>> 0000000000000000
>> [  842.385102] R13: 0000000000407818 R14: 0000000000000000 R15:
>> 0000000000000000
>> [  842.386432] FS:  00007f49073ca700(0000) GS:ffff88081f2c0000(0000)
>> knlGS:0000000000000000
>> [  842.387762] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [  842.389087] CR2: 00007ffd7ce07800 CR3: 00000007f5679000 CR4:
>> 00000000003407e0
>> [  842.390421] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
>> 0000000000000000
>> [  842.391759] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
>> 0000000000000400
>> [  842.393081] I-pipe domain Linux
>> [  842.394389] Stack:
>> [  842.395694]  00000000000002a6 ffff880000000000 0000000000000000
>> ffff8807e71a6260
>> [  842.396992]  00000000e71a6000 ffff8807e71a6248 00007ffd7ce07800
>> 0000000000000000
>> [  842.398272]  0000000000407818 0000000000000000 ffff8807c8c87f38
>> ffffffffa04e8ec0
>> [  842.399537] Call Trace:
>> [  842.400804]  [<ffffffffa04e8ec0>] rtai_lxrt_invoke+0x1f0/0x900
>> [rtai_sched]
>> [  842.402074]  [<ffffffff810ea996>] ? __audit_syscall_exit+0x236/0x2e0
>> [  842.403334]  [<ffffffffa04e72b5>] lxrt_intercept_syscall+0xc5/0x190
>> [rtai_sched]
>> [  842.404596]  [<ffffffff81602a18>] ?
>> int_check_syscall_exit_work+0x34/0x3d
>> [  842.405852]  [<ffffffff811005d9>] __ipipe_notify_syscall+0x19/0x20
>> [  842.407065]  [<ffffffff8160268e>] pipeline_syscall+0xa/0x17
>> [  842.408238] Code: 89 04 24 31 c0 41 ff 52 08 48 83 c4 30 5b 41 5c 41
>> 5d 41 5e 5d c3 48 c7 c7 b8 16 4f a0 e8 e8 c3 2d 00 48 c7 c0 da ff ff ff
>> eb 96 <4c> 8b 2e 8d 91 18 fc ff ff 83 fa 20 4c 89 e8 0f 87 85 00 00 00
>> [  842.409455] RIP  [<ffffffffa04e8631>] handle_lxrt_request+0x131/0x7d0
>> [rtai_sched]
>> [  842.410647]  RSP <ffff8807c8c87e78>
>> [  842.411800] CR2: 00007ffd7ce07800
>> [  842.420356] ---[ end trace a536dac0772dd8ef ]---
>> [  842.420410] USERMODE CHECK: ERROR.
>> [  842.420412] USERMODE CHECK PROVIDED (ns): KernelLatency 0,
>> UserLatency 0.
>> [  842.420413] FINAL CALIBRATION SUMMARY (ns): KernelLatency 0,
>> UserLatency 0.
>> [  850.797255]
>>
>> CPU USE SUMMARY
>> [  850.797259] # 0 -> 83615
>> [  850.797260] # 1 -> 0
>> [  850.797261] # 2 -> 0
>> [  850.797262] # 3 -> 0
>> [  850.797262] # 4 -> 0
>> [  850.797263] # 5 -> 0
>> [  850.797264] # 6 -> 0
>> [  850.797264] # 7 -> 0
>> [  850.797265] # 8 -> 0
>> [  850.797266] # 9 -> 0
>> [  850.797266] # 10 -> 0
>> [  850.797267] # 11 -> 0
>> [  850.797268] END OF CPU USE SUMMARY
>>
>> [  851.287459] RTAI[sched]: unloaded (forced hard/soft/hard transitions:
>> traps 0, syscalls 0).
>> [  851.293033] RTAI[malloc]: unloaded.
>> [  851.298420] I-pipe: head domain RTAI unregistered.
>> [  851.298430] RTAI[hal]: unmounted.
>>
>>
>>
>>
>> linux-36hs:/usr/realtime/calibration # ./run
>> *
>> *
>> * Type ^C to stop this application.
>> *
>> *
>> /usr/realtime/bin/rtai-load: line 185:  2873 Killed $suflag $*
>> linux-36hs:/usr/realtime/calibration #
>>
>>
>> linux-36hs:/usr/realtime # dmesg
>> [ 1114.495362] I-pipe: head domain RTAI registered.
>> [ 1114.495365] RTAI[hal]: mounted. ISOL_CPUS_MASK: 0.
>> [ 1114.495367] SYSINFO - # CPUs: 12, TIMER NAME: 'lapic', TIMER IRQ:
>> 4353, TIMER FREQ: 6246289, CLOCK NAME: 'tsc', CLOCK FREQ: 3398323000,
>> CPU FREQ: 3398323000, LINUX TIMER IRQ: 4353.
>> [ 1114.509743] RTAI[malloc]: global heap size = 4194304 bytes, <BSD>.
>> [ 1114.520995] , kstacks pool size = 524288 bytes.
>> [ 1114.520998] RTAI[sched]: hard timer type/freq = lapic/6246289(Hz);
>> timing: oneshot; linear timed lists.
>> [ 1114.520999] RTAI[sched]: Linux timer freq = 1000 (Hz), TimeBase freq
>> = 3398323000 hz.
>> [ 1114.521000] RTAI[sched]: timer setup = 160 ns, resched latency = 0 
>> ns.
>> [ 1114.521417] BUG: unable to handle kernel paging request at
>> 00007ffd243033a0
>> [ 1114.521925] IP: [<ffffffffa04e8631>] handle_lxrt_request+0x131/0x7d0
>> [rtai_sched]
>> [ 1114.523053] PGD 7e6a93067 PUD 7c8c9b067 PMD 7e7964067 PTE
>> 80000007cec47867
>> [ 1114.524195] Oops: 0001 [#7] PREEMPT SMP
>> [ 1114.525342] Modules linked in: rtai_sched(O+) rtai_malloc(PO)
>> rtai_hal(O) fuse af_packet snd_hda_codec_hdmi snd_hda_codec_realtek
>> snd_hda_codec_generic xfs libcrc32c x86_pkg_temp_thermal coretemp kvm
>> iTCO_wdt iTCO_vendor_support mxm_wmi snd_hda_intel snd_hda_controller
>> snd_hda_codec snd_hwdep crct10dif_pclmul crc32_pclmul
>> ghash_clmulni_intel snd_pcm aesni_intel aes_x86_64 lrw gf128mul
>> glue_helper ablk_helper cryptd snd_timer serio_raw e1000e snd mei_me mei
>> xhci_pci i2c_i801 ptp xhci_hcd pps_core lpc_ich shpchp soundcore
>> mfd_core dm_mod btrfs xor raid6_pq nouveau crc32c_intel video
>> i2c_algo_bit drm_kms_helper ttm drm wmi sg [last unloaded: rtai_hal]
>> [ 1114.529176] CPU: 11 PID: 2981 Comm: calibrate Tainted: P D    O
>>  3.18.22-STRADM #9
>> [ 1114.530488] Hardware name: ASUS All Series/X99-A II, BIOS 0801
>> 06/28/2016
>> [ 1114.531800] task: ffff8807e795e350 ti: ffff8807e7424000 task.ti:
>> ffff8807e7424000
>> [ 1114.533111] RIP: 0010:[<ffffffffa04e8631>] [<ffffffffa04e8631>]
>> handle_lxrt_request+0x131/0x7d0 [rtai_sched]
>> [ 1114.534442] RSP: 0018:ffff8807e7427e78  EFLAGS: 00010206
>> [ 1114.535790] RAX: ffff8807e7424000 RBX: 00007ffd243033a0 RCX:
>> 0000000000000407
>> [ 1114.537134] RDX: 0000000000000000 RSI: 00007ffd243033a0 RDI:
>> 0000000000407818
>> [ 1114.538486] RBP: ffff8807e7427ec8 R08: 00007ffd24303f69 R09:
>> 00007ffd00000001
>> [ 1114.539832] R10: 00007ffd24303368 R11: 0000000000000202 R12:
>> 0000000000000000
>> [ 1114.541171] R13: 0000000000407818 R14: 0000000000000000 R15:
>> 0000000000000000
>> [ 1114.542506] FS:  00007f14aad8a700(0000) GS:ffff88081f4c0000(0000)
>> knlGS:0000000000000000
>> [ 1114.543836] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [ 1114.545169] CR2: 00007ffd243033a0 CR3: 00000007c851c000 CR4:
>> 00000000003407e0
>> [ 1114.546504] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
>> 0000000000000000
>> [ 1114.547839] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
>> 0000000000000400
>> [ 1114.549165] I-pipe domain Linux
>> [ 1114.550476] Stack:
>> [ 1114.551791]  00000000000002a6 ffff880700000000 0000000000000000
>> ffff8807f3a92e60
>> [ 1114.553086]  00000000f3a92c00 ffff8807f3a92e48 00007ffd243033a0
>> 0000000000000000
>> [ 1114.554362]  0000000000407818 0000000000000000 ffff8807e7427f38
>> ffffffffa04e8ec0
>> [ 1114.555631] Call Trace:
>> [ 1114.556900]  [<ffffffffa04e8ec0>] rtai_lxrt_invoke+0x1f0/0x900
>> [rtai_sched]
>> [ 1114.558181]  [<ffffffff810ea996>] ? __audit_syscall_exit+0x236/0x2e0
>> [ 1114.559442]  [<ffffffffa04e72b5>] lxrt_intercept_syscall+0xc5/0x190
>> [rtai_sched]
>> [ 1114.560707]  [<ffffffff81602a18>] ?
>> int_check_syscall_exit_work+0x34/0x3d
>> [ 1114.561970]  [<ffffffff811005d9>] __ipipe_notify_syscall+0x19/0x20
>> [ 1114.563188]  [<ffffffff8160268e>] pipeline_syscall+0xa/0x17
>> [ 1114.564365] Code: 89 04 24 31 c0 41 ff 52 08 48 83 c4 30 5b 41 5c 41
>> 5d 41 5e 5d c3 48 c7 c7 b8 16 4f a0 e8 e8 c3 2d 00 48 c7 c0 da ff ff ff
>> eb 96 <4c> 8b 2e 8d 91 18 fc ff ff 83 fa 20 4c 89 e8 0f 87 85 00 00 00
>> [ 1114.565584] RIP  [<ffffffffa04e8631>] handle_lxrt_request+0x131/0x7d0
>> [rtai_sched]
>> [ 1114.566781]  RSP <ffff8807e7427e78>
>> [ 1114.567935] CR2: 00007ffd243033a0
>> [ 1114.569088] ---[ end trace a536dac0772dd8f2 ]---
>> [ 1114.569171] USERMODE CHECK: ERROR.
>> [ 1114.569174] USERMODE CHECK PROVIDED (ns): KernelLatency 0,
>> UserLatency 0.
>> [ 1114.569175] FINAL CALIBRATION SUMMARY (ns): KernelLatency 0,
>> UserLatency 0.
>> [ 1114.579349] BUG: unable to handle kernel paging request at
>> 00007ffdc26b67e0
>> [ 1114.579842] IP: [<ffffffffa04e8631>] handle_lxrt_request+0x131/0x7d0
>> [rtai_sched]
>> [ 1114.580985] PGD 7cb615067 PUD 7eb676067 PMD 7cb63a067 PTE
>> 80000000d0f69867
>>
>>
>> [ 1114.582127] Oops: 0001 [#8] PREEMPT SMP
>> [ 1114.583281] Modules linked in: rtai_sched(O) rtai_malloc(PO)
>> rtai_hal(O) fuse af_packet snd_hda_codec_hdmi snd_hda_codec_realtek
>> snd_hda_codec_generic xfs libcrc32c x86_pkg_temp_thermal coretemp kvm
>> iTCO_wdt iTCO_vendor_support mxm_wmi snd_hda_intel snd_hda_controller
>> snd_hda_codec snd_hwdep crct10dif_pclmul crc32_pclmul
>> ghash_clmulni_intel snd_pcm aesni_intel aes_x86_64 lrw gf128mul
>> glue_helper ablk_helper cryptd snd_timer serio_raw e1000e snd mei_me mei
>> xhci_pci i2c_i801 ptp xhci_hcd pps_core lpc_ich shpchp soundcore
>> mfd_core dm_mod btrfs xor raid6_pq nouveau crc32c_intel video
>> i2c_algo_bit drm_kms_helper ttm drm wmi sg [last unloaded: rtai_hal]
>> [ 1114.587108] CPU: 11 PID: 2994 Comm: calibration_hel Tainted: P      D
>>    O   3.18.22-STRADM #9
>> [ 1114.588427] Hardware name: ASUS All Series/X99-A II, BIOS 0801
>> 06/28/2016
>> [ 1114.589739] task: ffff8807eb716650 ti: ffff8807e6ad0000 task.ti:
>> ffff8807e6ad0000
>> [ 1114.591049] RIP: 0010:[<ffffffffa04e8631>] [<ffffffffa04e8631>]
>> handle_lxrt_request+0x131/0x7d0 [rtai_sched]
>> [ 1114.592381] RSP: 0018:ffff8807e6ad3e78  EFLAGS: 00010202
>> [ 1114.593726] RAX: ffff8807e6ad0000 RBX: 00007ffdc26b67e0 RCX:
>> 00000000000003ea
>> [ 1114.595070] RDX: 0000000000000000 RSI: 00007ffdc26b67e0 RDI:
>> 00000000003ea828
>> [ 1114.596425] RBP: ffff8807e6ad3ec8 R08: 000000000000000f R09:
>> 0000000000000000
>> [ 1114.597770] R10: 00007ffdc26b6798 R11: 0000000000003202 R12:
>> 0000000000000000
>> [ 1114.599112] R13: 00000000003ea828 R14: 0000000000000000 R15:
>> 0000000000000000
>> [ 1114.600443] FS:  00007f5b26ace700(0000) GS:ffff88081f4c0000(0000)
>> knlGS:0000000000000000
>> [ 1114.601778] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>> [ 1114.603109] CR2: 00007ffdc26b67e0 CR3: 00000007cb6ef000 CR4:
>> 00000000003407e0
>> [ 1114.604446] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
>> 0000000000000000
>> [ 1114.605782] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
>> 0000000000000400
>> [ 1114.607104] I-pipe domain Linux
>> [ 1114.608420] Stack:
>> [ 1114.609727]  0000000000000000 00007ffdc2797000 ffff8807e6ad3e98
>> ffff8807f5679e60
>> [ 1114.611023]  00000000f5679c00 ffff8807f5679e48 00007ffdc26b67e0
>> 0000000000000000
>> [ 1114.612300]  00000000003ea828 0000000000000000 ffff8807e6ad3f38
>> ffffffffa04e8ec0
>> [ 1114.613572] Call Trace:
>> [ 1114.614841]  [<ffffffffa04e8ec0>] rtai_lxrt_invoke+0x1f0/0x900
>> [rtai_sched]
>> [ 1114.616122]  [<ffffffff810ea996>] ? __audit_syscall_exit+0x236/0x2e0
>> [ 1114.617389]  [<ffffffffa04e72b5>] lxrt_intercept_syscall+0xc5/0x190
>> [rtai_sched]
>> [ 1114.618660]  [<ffffffff81602a18>] ?
>> int_check_syscall_exit_work+0x34/0x3d
>> [ 1114.619921]  [<ffffffff811005d9>] __ipipe_notify_syscall+0x19/0x20
>> [ 1114.621143]  [<ffffffff8160268e>] pipeline_syscall+0xa/0x17
>> [ 1114.622320] Code: 89 04 24 31 c0 41 ff 52 08 48 83 c4 30 5b 41 5c 41
>> 5d 41 5e 5d c3 48 c7 c7 b8 16 4f a0 e8 e8 c3 2d 00 48 c7 c0 da ff ff ff
>> eb 96 <4c> 8b 2e 8d 91 18 fc ff ff 83 fa 20 4c 89 e8 0f 87 85 00 00 00
>> [ 1114.623540] RIP  [<ffffffffa04e8631>] handle_lxrt_request+0x131/0x7d0
>> [rtai_sched]
>> [ 1114.624736]  RSP <ffff8807e6ad3e78>
>> [ 1114.625895] CR2: 00007ffdc26b67e0
>> [ 1114.627043] ---[ end trace a536dac0772dd8f3 ]---
>> [ 1114.970918] RTAI[sched]: unloaded (forced hard/soft/hard transitions:
>> traps 0, syscalls 0).
>> [ 1114.979641] RTAI[malloc]: unloaded.
>> [ 1114.985289] I-pipe: head domain RTAI unregistered.
>> [ 1114.985299] RTAI[hal]: unmounted.
>> linux-36hs:/usr/realtime #
>>
>>
>>
>>
>>
>> _______________________________________________
>> Rtai mailing list
>> [email protected]
>> https://mail.rtai.org/cgi-bin/mailman/listinfo/rtai
>>
> _______________________________________________
> Rtai mailing list
> [email protected]
> https://mail.rtai.org/cgi-bin/mailman/listinfo/rtai


_______________________________________________
Rtai mailing list
[email protected]
https://mail.rtai.org/cgi-bin/mailman/listinfo/rtai