Re: RTAI in debian9

Paolo Mantegazza <[email protected]> Wed, 28 Feb 2018 20:46:21 +0000
Newsgroups	gmane.linux.real-time.rtai
Message-ID	<HE1PR0601MB24581923663B733B4920E5F090C70@HE1PR0601MB2458.eurprd06.prod.outlook.com>
The third number is the assumed default period used for the calibration, if=
 it succeeded and you did not set it to a different one when configuring RT=
AI.
Clearly, when set by hands, it has little meaning, albeit being needed when=
 rtai_sched is insmoded.

It is strange that, once you ran latency, the preempt test do not run. Reca=
ll that you are assumed to run those tests from the RTAI install dir. Be su=
re of doing it in such a way and let me know.

Now that you ran rtai_sched using the latencies file, the call_userhelper s=
upport seems to be working also on your system. So I've even less clues tha=
n before on the why it did not calibrate. In fact the automatic calibration=
 is nothing but the latency test in disguise, coupled to a trivial Kalman f=
ilter to estimate the expected latency on the fly.

Thanks for your feedback, Paolo.

_______________________________________
From: Paul Pfeiffer [[email protected]]
Sent: Wednesday, February 28, 2018 7:14 PM
To: Paolo Mantegazza; [email protected]
Subject: Re: [Rtai] RTAI in debian9

Thanks for the feedback.

I added the latencies file and now the rtai_sched module can be loaded
without specifiying user or kernel latency. In the kernel output, I see
that the kernel latency is now 440ns and the user latency 880ns. What is
the meaning of the third number?

I tried running the other tests, but they do not work, because of a
missing .runinfo file. Do I have to create this? What should be in this
file?

We will use the RTAI patched kernel to perform recordings from neurons
with real time feedback loops. I will update you, when we have tried the
new RTAI patch within this application.

Best, Paul


On 02/27/2018 10:59 PM, Paolo Mantegazza wrote:
> Your system seems OK. If you edit a file, named "latencies", with a singl=
e line containing:
> 440 880 10000
> and copy it into "rtai_installdir/calibration/latencies.
> you'll have what the calibration should have done and there will be non n=
eed to set those latencies at rtai_sched.ko insmod.
> In any case, once you have done what above, may test both latency and pre=
empt, in user, kern and kthread directories of yout rtai_installdir and the=
n report what happened?
> Do not test switches, It locks the cpu for a bit of time and may be risky=
. I'll not be surprised of a muted system.
> As for why it does not work on your system, without a diagnose hands on, =
I've no clue.
> Moreover I'll be glad to hear of some further checks with a production co=
de of yours, if any.
>
> Thanks, Paolo
>
> ________________________________________the
> From: Rtai [[email protected]] on behalf of Paul Pfeiffer [pfeifferpa=
[email protected]]
> Sent: Tuesday, February 27, 2018 4:41 PM
> To: [email protected]
> Subject: Re: [Rtai] RTAI in debian9
>
> Hi,
>
> thanks for the nice discussion in this thread. I have a similar issue to
> the one described by josu, but on a real machine.
>
> Setup: Linux kernel 4.9.76 and RTAI 5.1
>
> For patching the kernel and compilation of the kernel modules, I
> followed the instructions from Jan Benda, here
> http://relacs.sourceforge.net/plugins/rtaicomedi/
>
> Insertion of the rtai_hal module works, but insertion of rtai_sched
> kills the system.
>
> Like Paolo suggested, I also tried loading the rtai_sched with
> kernel_latency and user_latency set to 1000. This works. Does this mean
> that the calibration is the problem? How could I solve this for my setup?
>
> Also, I ran the suggested modified latency test. However, like josu, I
> had to change the insertion of the rtai_sched module to
>
> ```
>
> insmod $MODULES/rtai_sched.ko kernel_latency=3D1 user_latency=3D1
>
> ```
>
> otherwise the system would stop responding.
>
> Please find below the output of the test and the corresponding kernel
> messages.
>
> Thank you, Paul
>
> ________________
>
> $ sudo ./testsuite/user/latency/run
>
> RTAI Testsuite - USER space latency test (output data in nanoseconds)
>
> *** latency verification tool with real time hardened user space
> processes/threads ***
> ***    period =3D 100000 (ns),  avrgtime =3D 1 (s)    ***
>
> RTAI Testsuite - USER latency (all data in nanoseconds)
> 2018/02/27 16:18:13
> RTH|=A0=A0=A0 lat min|=A0=A0=A0 ovl min|=A0=A0=A0 lat avg|=A0=A0=A0 lat m=
ax|=A0=A0=A0 ovl max| overruns
> RTD|=A0=A0=A0=A0=A0=A0=A0 726|=A0=A0=A0=A0=A0=A0=A0 726|=A0=A0=A0=A0=A0=
=A0=A0 887|=A0=A0=A0=A0=A0=A0 5216| 5216|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 705|=A0=A0=A0=A0=A0=A0=A0 705|=A0=A0=A0=A0=A0=
=A0=A0 884|=A0=A0=A0=A0=A0=A0 2667| 5216|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 884|=A0=A0=A0=A0=A0=A0 3267| 5216|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 599|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 890|=A0=A0=A0=A0=A0=A0 8122| 8122|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 711|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 884|=A0=A0=A0=A0=A0 15637| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 638|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 882|=A0=A0=A0=A0=A0=A0 6058| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 722|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 877|=A0=A0=A0=A0=A0=A0 3233| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 741|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 881|=A0=A0=A0=A0=A0=A0 2186| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 733|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 874|=A0=A0=A0=A0=A0=A0 3213| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 734|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 874|=A0=A0=A0=A0=A0=A0 2666| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 718|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 874|=A0=A0=A0=A0=A0=A0 1823| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 729|=A0=A0=A0=A0    545|=A0=A0=A0=A0=A0=A0=A0 8=
74|=A0=A0=A0=A0=A0=A0 2813| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 709|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 883|=A0=A0=A0=A0=A0=A0 5861| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 722|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 897|=A0=A0=A0=A0=A0 12125| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 721|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 895|=A0=A0=A0=A0=A0=A0 3858| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 598|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 891|=A0=A0=A0=A0=A0=A0 2384| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 701|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 877|=A0=A0=A0=A0=A0=A0 2205| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 642|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 878|=A0=A0=A0=A0=A0=A0 3895| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 590|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 880|=A0=A0=A0=A0=A0=A0 1844| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 725|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 877|=A0=A0=A0=A0=A0=A0 3841| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 549|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 884|=A0=A0=A0=A0=A0=A0 2105| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> 2018/02/27 16:18:35
> RTH|=A0=A0=A0 lat min|=A0=A0=A0 ovl min|=A0=A0=A0 lat avg|=A0=A0=A0 lat m=
ax|=A0=A0=A0 ovl max| overruns
> RTD|=A0=A0=A0=A0=A0=A0=A0 708|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 880|=A0=A0=A0=A0=A0=A0 3730| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 717|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 875|=A0=A0=A0=A0=A0=A0 3596| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> RTD|=A0=A0=A0=A0=A0=A0=A0 721|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 878|=A0=A0=A0=A0=A0=A0 1470| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
> ^CRTD|=A0=A0=A0=A0=A0=A0=A0 721|=A0=A0=A0=A0=A0=A0=A0 545|=A0=A0=A0=A0=A0=
=A0=A0 878|=A0=A0=A0=A0=A0=A0 1470| 15637|=A0=A0=A0=A0=A0=A0=A0=A0=A0 0
>
>   >>> S =3D 98.696, EXECTIME =3D 0.0105475
>
>
> $sudo dmesg -T -w
>
> [Tue Feb 27 16:18:12 2018] I-pipe: head domain RTAI registered.
> [Tue Feb 27 16:18:12 2018] RTAI[hal]: mounted. ISOL_CPUS_MASK: 0.
> [Tue Feb 27 16:18:12 2018] SYSINFO - # CPUs: 4, TIMER NAME: 'lapic',
> TIMER IRQ: 4354, TIMER FREQ: 437999993, CLOCK NAME: 'tsc', CLOCK FREQ:
> 3500000000, CPU FREQ: 3500000000, LINUX TIMER IRQ: 4354.
> [Tue Feb 27 16:18:12 2018]
>                              REMARK: RTAI WILL NOT ACCESS USER SPACE ON
> STACKS ARGS ITS WAY.
>
> [Tue Feb 27 16:18:12 2018] RTAI[malloc]: global heap size =3D 2097152
> bytes, <BSD>.
> [Tue Feb 27 16:18:12 2018] , kstacks pool size =3D 524288 bytes
> [Tue Feb 27 16:18:12 2018] .
> [Tue Feb 27 16:18:12 2018] RTAI[sched]: hard timer type/freq =3D
> lapic/437999993(Hz); timing: ONESHOT;
> [Tue Feb 27 16:18:12 2018] linear timed lists.
> [Tue Feb 27 16:18:12 2018] RTAI[sched]: Linux timer freq =3D 250 (Hz),
> TimeBase freq =3D 3500000000 hz.
> [Tue Feb 27 16:18:12 2018] RTAI[sched]: timer setup =3D 66 ns, resched
> latency =3D 0 ns.
> [Tue Feb 27 16:18:12 2018] FINAL CALIBRATION SUMMARY (ns): KernelLatency
> 0, UserLatency 0.
> [Tue Feb 27 16:18:39 2018] RTAI[malloc]: unloaded.
> [Tue Feb 27 16:18:39 2018] RTAI[sched]: unloaded (forced hard/soft/hard
> transitions: traps 0, syscalls 0).
> [Tue Feb 27 16:18:39 2018] I-pipe: head domain RTAI unregistered.
> [Tue Feb 27 16:18:39 2018] RTAI[hal]: unmounted.
>
>
> On 02/24/2018 12:20 AM, josu wrote:
>> Yes, of course, I can=92t get real time (much less, 100 us period), only=
 test the procedure. However, setting deadline to 1 (and not 0), the progra=
ms run, without hanging the VM.
>>
>> Although the meaning is not at all the same that in a real system, the l=
atency test are able to run with your run script  (setting a number greater=
 than 0). This is the result:
>>
>> root@debianRB:/usr/realtime/testsuite/user/latency# ./run
>> RTAI Testsuite - USER space latency test (output data in nanoseconds)
>>
>> *** latency verification tool with real time hardened user space process=
es/threads ***
>> ***    period =3D 100000 (ns),  avrgtime =3D 1 (s)    ***
>>
>> RTAI Testsuite - USER latency (all data in nanoseconds)
>> 2018/02/23 22:47:06
>> RTH|    lat min|    ovl min|    lat avg|    lat max|    ovl max|   overr=
uns
>> RTD|          0|          0|      25068|     870103|     870103|        =
 67
>> RTD|          0|          0|      25608|    1007163|    1007163|        =
 95
>> RTD|          0|          0|      25288|     886190|    1007163|        =
124
>> RTD|          0|          0|      25275|     985017|    1007163|        =
155
>> RTD|          0|          0|      24564|     832423|    1007163|        =
179
>> RTD|          0|          0|      25620|     199742|    1007163|        =
194
>> RTD|          0|          0|      25393|     881237|    1007163|        =
226
>> RTD|          0|          0|      25067|     874862|    1007163|        =
253
>> RTD|          0|          0|      25247|     827829|    1007163|        =
274
>> RTD|          0|          0|      25566|     855516|    1007163|        =
310
>> RTD|          0|          0|      24899|    1035597|    1035597|        =
360
>> RTD|          0|          0|      25403|     202992|    1035597|        =
376
>> RTD|          0|          0|      25288|     771628|    1035597|        =
398
>> RTD|          0|          0|      25288|     771628|    1035597|        =
398
>> ^C
>>>>> S =3D 98.696, EXECTIME =3D 0.0663255
>>> On 24 Feb 2018, at 00:11, Paolo Mantegazza <[email protected]>=
 wrote:
>>>
>>> VM changes the matter a lot.  Anticipating the scheduling deadline by 1=
 ns does not change anything with respect to 0.
>>> In any case it served to indicate that you cannot run the RTAI tests, s=
imply because VM cannot provide true real time. In particular it cannot sup=
port a 100 us period.
>>> I dare saying that if you configure RTAI to use a lower calibrating fre=
quency, say 2000 Hz , in place of the default 10000 Hz, you might succeed i=
n trying the RTAI latency tests, along with the hidden latency calibration.
>>> In any case hard real time will likely be a no hope matter.
>>>
>>> Paolo
>>>
>>> ________________________________________
>>> From: josu [[email protected]]
>>> Sent: Friday, February 23, 2018 10:57 PM
>>> To: Paolo Mantegazza
>>> Cc: RTAI RTAI
>>> Subject: Re: [Rtai] RTAI in debian9
>>>
>>> Hi again:
>>>
>>> First, I want to clarify than I am testing with a VM. Sorry for the pos=
sible misunderstanding. I have been very busy and I have forgot to say this=
 important point.
>>>
>>> In any case, I have test your script and the system hangs. However, mak=
ing the next change:
>>>
>>> insmod $MODULES/rtai_sched.ko kernel_latency=3D1 user_latency=3D1
>>>
>>> I have obtained the result attached below.
>>>
>>> josu
>>>
>>> [cid:[email protected]]
>>>
>>> On 23 Feb 2018, at 18:56, Paolo Mantegazza <[email protected]<=
mailto:[email protected]>> wrote:
>>>
>>> May you try the "testsuite/user/latency" test, in your RTAI install dir=
, after saving and substituting the existing "run" script with the one belo=
w and then let me know what happened?
>>> Do it on an unloaded machined and, if possible, after seeing a stabiliz=
ed average latency, attach to your answer also a short snapshot of what app=
ears on the screen.
>>> Thanks, Paolo
>>> -----------------------------------------------------------------------=
------------------------------------
>>> #!/bin/sh
>>>
>>> prefix=3D`rtai-config --prefix`
>>>
>>> if test "$prefix" =3D ""; then
>>> echo "ERROR: please set your PATH variable to <rtai-install>/bin"
>>> exit
>>> fi
>>>
>>> MODULES=3D$prefix/modules
>>>
>>> libpath=3D`rtai-config --library-dir`
>>>
>>> if test "$libpath" =3D ""; then
>>> echo "ERROR: please set your PATH variable to <rtai-install>/bin"
>>> exit
>>> fi
>>>
>>> export LD_LIBRARY_PATH=3D$libpath:$LD_LIBRARY_PATH
>>>
>>> cleanup () {
>>> rmmod rtai_msg
>>> rmmod rtai_mbx
>>> rmmod rtai_sem
>>> rmmod rtai_sched
>>> rmmod rtai_hal
>>> }
>>>
>>> trap cleanup INT ALRM TERM HUP KILL
>>>
>>> insmod $MODULES/rtai_hal.ko
>>> insmod $MODULES/rtai_sched.ko kernel_latency=3D0 user_latency=3D0
>>> insmod $MODULES/rtai_sem.ko;
>>> insmod $MODULES/rtai_mbx.ko;
>>> insmod $MODULES/rtai_msg.ko;
>>>
>>> ./latency &
>>> until(`ps -a | grep -q latency`); do sleep 1; done
>>> ./display
>>>
>>> cleanup
>>> -----------------------------------------------------------------------=
------------------------
>>>
>>> ________________________________________
>>> From: josu jugo [[email protected]<mailto:[email protected]>]
>>> Sent: Friday, February 23, 2018 5:37 PM
>>> To: Paolo Mantegazza
>>> Cc: RTAI RTAI
>>> Subject: Re: [Rtai] RTAI in debian9
>>>
>>> Hi again:
>>>
>>> I have test a simple program obtained with rtai-lab  (scicoslab version=
), monitoring with qrtailab and the system is working.
>>>
>>> I have load rtai_sched with kernel_latency=3D1000 and user_latency=3D10=
00
>>>
>>> josu
>>>
>>> On 23 Feb 2018, at 12:51, josu jugo <[email protected]<mailto:josu.jugo=
@ehu.eus><mailto:[email protected]>> wrote:
>>>
>>>
>>>
>>>
>>> <Screen Shot 2018-02-23 at 22.47.37.png>
>> _______________________________________________
>> Rtai mailing list
>> [email protected]
>> https://mail.rtai.org/cgi-bin/mailman/listinfo/rtai
> _______________________________________________
> Rtai mailing list
> [email protected]
> https://mail.rtai.org/cgi-bin/mailman/listinfo/rtai

_______________________________________________
Rtai mailing list
[email protected]
https://mail.rtai.org/cgi-bin/mailman/listinfo/rtai