Re: RTAI 5.0.1 calibrate program: BUG: unable to handle kernel paging request at 00007ffe78e450f0 (Paolo Mantegazza)

An i3-6xxx was the cpu that first lead to the calibration problem. Which, in fact, was not a calibration problem but a memory protection that did not exist on previous generations, i.e for about more than 10 years of RTAI existence.
In any case, once the problem was solved, through the contribution release you mention, to my knowledge, the friendly user that helped me in the fix has not signaled any latency problem.
That said, can you tell us a few things more?. For example, what are the latency values you have, found with what test? Moreover are you sure they are due to SMI and not to other Linux config params (see the suggestions below)? Have you diagnosed SMI with the graphic tool found in RTAI? 

Paolo, 

REMARKS-SUGGESTIONS ABOUT CONFIGURING LINUX AND RTAI

< LINUX kernel related >
- Under SMP set the number of CPUs equal to the real ones and have it matched in RTAI,
  no hyperthreading intended (see below)
- Some peripherals, e.g. video cards, may stall CPUs attempting to access IO space.
  Verify "what ifs" related to graphic acceleration, likely better if disabled.
  Consider also if X term usage is really needed. If possible avoid it, especially
  in production work.
- LINUX use of DMA can add latency, especially when it is supported in burst mode.
- Cached memory disruption can add significant latencies, as experienced in a 
  controller, after a far jump in the code and data, till the cache becomes hot
  again.
- Power management, see CONFIG_CPU_FREQ and CONFIG_CPU_IDLE below; on portables 
  battery management too.
- Recent Intel SpeedStepping and Boosting.
- Disable AUDITSYSCALLS.
- Disable CPU_FREQ.
- Disable CPU_IDLE and INTEL_IDLE, or boot with "intel_idle.max_cstate=0". If you
  want to be sure to have a never sleeping CPU execute, at the lowest priority, 
  your own, per cpu, idle task, i.e. just doing "while(1);".
- Disable APM and ACPI_Processor, but not everything related to power management.
  Take also into account that without ACPI enabled you might not see more than 
  a single CPU.
- Do not enable IPIPE legacy support. As a safety measure against such a choice, 
  RTAI build is inhibited at the making of rtai_hal.ko.
- Do not disable USB, but just any legacy support, possibly in the BIOS also. Once
  upon a time USB was a source of high RTAI latencies. Now that should happen with
  just legacy support enabled. 
- Try to configure the CPU type to be as close as possible to the one you have.
- If unsure on the CPU to choose, care of setting one featuring a Time Stamp Clock 
  (TSC), which means no 486 and "false" i586, as generic INTEL i586 compatibles
  often do not have a TSC, while true INTEL ones do have it.
- If you are using a UniProcessor (UP) compile RTAI against a UP configured kernel.
  In fact RTAI compiled for SMP may not work when used on a UP machine. An issue 
  to be fixed, sooner or later. In any case having everything UP for UP is always
  the more efficient solution.
- If it is of no interest to you, disable any kernel debug support. Generally speaking,
  most RTAI users are not expected to debug the kernel, so it is suggested to always do 
  so for production work. In any case, even if there should be no trouble in keeping it,
  except for some overhead more and an increase of latency here and there, it may happen
  that tracing could affect RTAI parts having some commonality with the kernel, e.g.
  though trace points missing symbols. If that happens you should either disable, at 
  least, the trace option in "Kernel hacking", or add the exporting of the related
  symbols, which appear missing at insmoding any RTAI module.
- If you witness a stack protector crash, disable stack protector support, at the
  bottom of the kernel config General Setup. There are 3 choices, "None" will work
  with RTAI for sure. It is possible that the "Regular" one could also be OK, while
  "Strong" will fail, always.


< RTAI related >
- Even if RTAI can work with hyperthreading enabled, such an option is deprecated
  as a possible cause of latency; in any case try and verify if it is acceptable,
  with your hardware and for your applications.
- Any initialization of the device drivers, or anything related to the hardware,
  may lead to high latencies, e.g., but not always. doing "startx &" while a real
  time application is running. Once it is started there should be no major problems.
  If the truoble persists and you really need X, concurrently with you RTAI tasks, try
  disabling hardware graphic acceleration.  The best latencies usually come with no
  graphic application running.
- A sizable part of the latency you see in RTAI hard timed programs can mature as 
  an interrupt latency, little can be done to avoid that. Recall that, for shared pieces
  of hardware, e.g APIC, LINUX is left with the capability of using a few hard interrupts
  disable/enable. Moreover the processing of LINUX interrupts, during hard real time
  RTAI activities, are delayed but must be pended to be processed later on anyhow.
  A thing which requires keeping interrupts disabled for a, hopefully short, while.
- If SMI is enabled and latencies are high, often appearing periodic also, use the 
  RTAI tools to monitor and, possibly, fix it.


________________________________________
From: Rtai [[email protected]] on behalf of Vivian McPhail [[email protected]]
Sent: Thursday, October 5, 2017 7:14 PM
To: [email protected]
Subject: Re: [Rtai] RTAI 5.0.1 calibrate program: BUG: unable to handle kernel paging request at 00007ffe78e450f0 (Paolo Mantegazza)

Hi List,

With respect to the below problem.  We have used the CONTRIB branch which Paolo mentioned (I had thought he pointed to the same tarball as the release).

We are getting unacceptably high latencies with an i3 6100 7th generation CPU and are wondering whether we should downgrade to a CPU that doesn't have SMI.

Has anyone else encountered latency issues like this?

We can provide more information if required.

Thanks,

Vivian


Message: 1
Date: Tue, 19 Sep 2017 17:42:43 +0200
From: Paolo Mantegazza <[email protected]<mailto:[email protected]>>
To: Rtai <[email protected]<mailto:[email protected]>>
Subject: Re: [Rtai] RTAI 5.0.1 calibrate program: BUG: unable to
        handle kernel paging request at 00007ffe78e450f0
Message-ID: <[email protected]<mailto:[email protected]>>
Content-Type: text/plain; charset="utf-8"; Format="flowed"

Ciao,
please, beware of trolls. They are amusing, at times, but often they do
not know what they are talking about.

The problem of calibration faults has been solved by the end of
May-2017, thanks to a cooperation with Josef (A?falg).
The fix is
in"https://www.rtai.org/userfiles/downloads/RTAICONTRIB/rtai-5.0.1.tar.bz2",
to be 5.1, when we'll get a new RTAI repository, hopefully soon.

If you want to know more about it have a look at the following May-2017
threads on the RTAI mailing list:
https://mail.rtai.org/pipermail/rtai/2017-May/027461.html
https://mail.rtai.org/pipermail/rtai/2017-May/027502.html

Paolo

On 09/19/2017 05:01 AM, Alec Ari wrote:
> I'm not sure why Paolo treats his RTAI 5.0.1 release like gold. It's got tons of issues, everything from hard-locks, overruns, and extreme latency spikes. Not sure where he got the idea that it's so brilliant and to always grab the latest versions of his (ripped-off) code. RTAI 4.0.1 which I've based my stable tree of off has absolutely no issues.
>
> Myself and many others (LinuxCNC) have been using this tree for the past few years and it's fine:
>
>
> https://github.com/NTULINUX/RTAI
>
> Don't use the drop-legacy branch, it's not ready and I'm still ironing out all the bugs that Paolo introduced into the scheduler and self-calibration. The old calibration code worked so much better, literally "just works." RTAI 5 is not ready for production and is not stable by any means. I highly suggest avoiding anything above 5.0 until there's some more confirmed cases of it working in upcoming releases.
>
> I've been doing kernel development for over a decade and this is not user-error.
> _______________________________________________
> Rtai mailing list
> [email protected]<mailto:[email protected]>
> https://mail.rtai.org/cgi-bin/mailman/listinfo/rtai
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.rtai.org/pipermail/rtai/attachments/20170919/5adf2dc1/attachment-0001.html>

------------------------------

Message: 2
Date: Tue, 19 Sep 2017 22:04:56 +0000 (UTC)
From: Alec Ari <[email protected]<mailto:[email protected]>>
To: Rtai <[email protected]<mailto:[email protected]>>
Subject: Re: [Rtai] RTAI 5.0.1 calibrate program: BUG: unable to
        handle kernel paging request at 00007ffe78e450f0
Message-ID: <[email protected]<mailto:[email protected]>>
Content-Type: text/plain; charset=UTF-8

I'm lost for words, Paolo actually replied to something I said.


So use the RTAI 5.0.1 tarball from the users repository instead of the one in the releases, you say? The one with "ifdef CONFIG_RTAI_USE_STACK_ARGS" scattered about? You do realize that CONFIG_RTAI_USE_STACK_ARGS is an undefined config option that has no meaning, not in Kconfig or configure.in<http://configure.in> anywhere?


If you think I'm trolling, you should look at yourself. Stealing my code and writing the sloppiest crap to almost purposely make it confusing.

Like what the hell is this, seriously:

{ if (ipipe_notifier_enabled_p(current)) return ((int (*)(unsigned long, void *))ipipe_root_domain->legacy.handlers[exception])(exception, regs); } {

I've said it before and I'll say it again; your work really belongs in the obfuscated C code contest. This project would be dead without me and ShabbyX, but you're too arrogant to give anyone else credit. Everything that hasn't been done by myself or Shahbaz doesn't work. He left the project and moved on to other things but I remember the countless hours we spent into revamping the entire tree. You tell me who the troll is, or just go hide off in your corner again like you always do.


Alec Ari


------------------------------

Subject: Digest Footer

_______________________________________________
Rtai mailing list
[email protected]<mailto:[email protected]>
https://mail.rtai.org/cgi-bin/mailman/listinfo/rtai

------------------------------

End of Rtai Digest, Vol 117, Issue 8
************************************



--
Vivian McPhail
PhD Candidate
Department of Physics
University of Auckland
_______________________________________________
Rtai mailing list
[email protected]
https://mail.rtai.org/cgi-bin/mailman/listinfo/rtai