Re: Question about LTTng Kernel Probe Tracking
Amir Najafi Zadeh via lttng-dev <[email protected]> Tue, 7 Apr 2026 11:14:25 -0400
| Newsgroups | org.lttng.lists.lttng-dev |
|---|---|
| Message-ID | <CAOQX81e-YGCktrjYwdwDAe_5uRmP5=GaYPBbWm1EiMDnuG0aaA@mail.gmail.com> |
--000000000000c289fc064ee03d7e
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Added LTTng dev to BCC to avoid spamming.
Hi Kienan. Thanks for your response. I tried specifying the session name,
but nothing changed.
Here is the output I get after adding the tracker:
Added 1002 to the group ID tracker inclusion set
I don=E2=80=99t think the session name is the issue, because I also specifi=
ed the
session name when starting tracking. This is the session state after
creating it:
$ sudo lttng list lttng-gid-test
Recording session lttng-gid-test: [active]
Trace output: /tmp/lttng/lttng-gid-test
=3D=3D=3D Domain: Linux kernel =3D=3D=3D
Tracked process attributes
Process IDs: none
Virtual process IDs: none
User IDs: none
Virtual user IDs: none
Group IDs: 1002
Virtual group IDs: none
And these are the logs I get:
[11:01:51.222277079] (+0.000004846) dhcp15 ext4_get_inode_loc: {
cpu_id =3D 3 }, { pid =3D 403653, tid =3D 403660, procname =3D
"lttng-consumerd", gid =3D 0, uid =3D 0, vpid =3D 403653, vuid =3D 0, vgid =
=3D 0
}, { ip =3D 0xFFFFFFFFAC804940 }
...
There are many logs like this from systemd, kworker, and processes
belonging to other groups. Note that the GID 1002 that I want to trace
currently has no active subjects.
Do you suggest switching to the recent version of LTTng (current version
2.13.11)?
Best,
Amir
On Tue, Apr 7, 2026 at 10:57=E2=80=AFAM Kienan Stewart <[email protected]=
om>
wrote:
> Hi Amir,
>
> On 2026-04-04 21:02, Amir Najafi Zadeh via lttng-dev wrote:
> > Hello all,
> >
> > I hope you are doing well.
> >
> > I have a quick question about LTTng tracking when using kernel probes. =
I
> > mainly use LTTng to trace file system functions in the Linux kernel, an=
d
> > for that I use the LTTng kernel probe feature, which works very well.
> > However, when I tried filtering logs for a specific group ID, I noticed
> > that filters are not supported for kernel probes.
> >
> > Then I attempted to use the tracking feature to trace only a specific
> > GID, but this also did not work. From what I see in my logs, it appears
> > that for kernel probes all events are captured and recorded, regardless
> > of the tracking options.
> >
> > I wanted to know if this is a known limitation, and whether there is an=
y
> > in-progress work or reason why this is not currently supported.
> >
>
> I just tried on the master branch (since I had easy access to it), and
> wasn't able to reproduce a situation where I added the gid tracking and
> had results for a different gid in lttng view.
>
> > For reference, here is my code. I am using LTTng 2.13.11 with kernel
> > 6.8.0-101-generic:
> >
> > ``` #!/bin/sh
> > SESSION_NAME=3D"lttng-gid-test"
> > OUTPUT_DIR=3D"/tmp/lttng/${SESSION_NAME}"
> > GROUP_ID=3D1002
> >
> > # create the lttng session
> > lttng create "$SESSION_NAME" -o "$OUTPUT_DIR"
> >
> > # create the lttng ring buffer channel
> > lttng enable-channel --session=3D"$SESSION_NAME" --kernel channel0 \
> > --subbuf-size=3D16M \
> > --num-subbuf=3D2
> >
> > # add pid, tid, proc, and gid for tracing collection
> > lttng add-context --session=3D"$SESSION_NAME" --channel=3Dchannel0 --ke=
rnel
> > --type pid
> > lttng add-context --session=3D"$SESSION_NAME" --channel=3Dchannel0 --ke=
rnel
> > --type gid
> >
> > # enable the target probe
> > lttng enable-event --session=3D"$SESSION_NAME" --channel=3Dchannel0 --k=
ernel
> > --probe=3Dext4_get_inode_loc ext4_get_inode_loc
> >
> > # disable all tracks # NOTE: untracking doesn't make a difference
> > # lttng untrack --session=3D"$SESSION_NAME" --kernel --all --gid
> > # lttng untrack --session=3D"$SESSION_NAME" --kernel --all --pid
> > # lttng untrack --session=3D"$SESSION_NAME" --kernel --all --vpid
> > # lttng untrack --session=3D"$SESSION_NAME" --kernel --all --uid
> > # lttng untrack --session=3D"$SESSION_NAME" --kernel --all --vuid
> > # lttng untrack --session=3D"$SESSION_NAME" --kernel --all --vgid
> > # only enable gid 1002 track
> > lttng track --kernel --gid=3D1002
> >
>
> Could you check the output of your script after "lttng track --kernel
> --gid=3D1002"?
>
> The rest of your commands specify a session name, but this one does not.
> I wonder if it's selecting a different default session that the one
> you're expecting?
>
> > # start lttng lttng start $SESSION_NAME
> >
> > ```
> >
> > Thank you for your time and guidance.
> >
> > Best regards,
> > Amir
> >
> > --
> > *Amirhossein Najafizadeh*
> > *PhD Student, Computer Science Department, Stony Brook University, N.Y.
> > File systems and Storage Lab (FSL)
> >
> > *
> > [email protected] <mailto:[email protected]>
> > https://amirhnajafiz.github.io/ <https://amirhnajafiz.github.io/>
>
>
--000000000000c289fc064ee03d7e
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><p class=3D"gmail-isSelectedEnd">Added <span class=3D"gmai=
l-hover:entity-accent entity-underline gmail-inline gmail-cursor-pointer gm=
ail-align-baseline"><span class=3D"gmail-whitespace-normal">LTTng</span></s=
pan> dev to <span class=3D"gmail-hover:entity-accent entity-underline gmail=
-inline gmail-cursor-pointer gmail-align-baseline"><span class=3D"gmail-whi=
tespace-normal">BCC</span></span> to avoid spamming.<br><br>Hi Kienan. Than=
ks for your response. I tried specifying the session name, but nothing chan=
ged.</p><p class=3D"gmail-isSelectedEnd">Here is the output I get after add=
ing the tracker:</p><pre dir=3D"ltr"><code dir=3D"ltr">Added 1002 to the gr=
oup ID tracker inclusion set</code></pre><p class=3D"gmail-isSelectedEnd">I=
don=E2=80=99t think the session name is the issue, because I also specifie=
d the session name when starting tracking. This is the session state after =
creating it:</p><pre dir=3D"ltr"><code dir=3D"ltr">$ sudo lttng list lttng-=
gid-test
Recording session lttng-gid-test: <span class=3D"gmail-text-token-text-prim=
ary gmail-cursor-text gmail-rounded-sm">[active]</span>
Trace output: /tmp/lttng/lttng-gid-test
=3D=3D=3D Domain: Linux kernel =3D=3D=3D
Tracked process attributes
Process IDs: none
Virtual process IDs: none
User IDs: none
Virtual user IDs: none
Group IDs: 1002
Virtual group IDs: none</code></pre><p class=3D"gmail-isSelectedEnd">A=
nd these are the logs I get:</p><pre dir=3D"ltr"><code dir=3D"ltr"><span cl=
ass=3D"gmail-text-token-text-primary gmail-cursor-text gmail-rounded-sm">[1=
1:01:51.222277079]</span> (+0.000004846) dhcp15 ext4_get_inode_loc: { cpu_i=
d =3D 3 }, { pid =3D 403653, tid =3D 403660, procname =3D "lttng-consu=
merd", gid =3D 0, uid =3D 0, vpid =3D 403653, vuid =3D 0, vgid =3D 0 }=
, { ip =3D 0xFFFFFFFFAC804940 }
...</code></pre><p>There are many logs like this from systemd, kworker, and=
processes belonging to other groups. Note that the GID 1002 that I want to=
trace currently has no active subjects.<br><br>Do you suggest switching to=
the recent version of LTTng (current version 2.13.11)?=C2=A0<br><br>Best,<=
br>Amir</p></div><br><div class=3D"gmail_quote gmail_quote_container"><div =
dir=3D"ltr" class=3D"gmail_attr">On Tue, Apr 7, 2026 at 10:57=E2=80=AFAM Ki=
enan Stewart <<a href=3D"mailto:[email protected]">kstewart@efficios=
.com</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"mar=
gin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1=
ex">Hi Amir,<br>
<br>
On 2026-04-04 21:02, Amir Najafi Zadeh via lttng-dev wrote:<br>
> Hello all,<br>
> <br>
> I hope you are doing well.<br>
> <br>
> I have a quick question about LTTng tracking when using kernel probes.=
I <br>
> mainly use LTTng to trace file system functions in the Linux kernel, a=
nd <br>
> for that I use the LTTng kernel probe feature, which works very well. =
<br>
> However, when I tried filtering logs for a specific group ID, I notice=
d <br>
> that filters are not supported for kernel probes.<br>
> <br>
> Then I attempted to use the tracking feature to trace only a specific =
<br>
> GID, but this also did not work. From what I see in my logs, it appear=
s <br>
> that for kernel probes all events are captured and recorded, regardles=
s <br>
> of the tracking options.<br>
> <br>
> I wanted to know if this is a known limitation, and whether there is a=
ny <br>
> in-progress work or reason why this is not currently supported.<br>
> <br>
<br>
I just tried on the master branch (since I had easy access to it), and <br>
wasn't able to reproduce a situation where I added the gid tracking and=
<br>
had results for a different gid in lttng view.<br>
<br>
> For reference, here is my code. I am using LTTng 2.13.11 with kernel <=
br>
> 6.8.0-101-generic:<br>
> <br>
> ``` #!/bin/sh<br>
> SESSION_NAME=3D"lttng-gid-test"<br>
> OUTPUT_DIR=3D"/tmp/lttng/${SESSION_NAME}"<br>
> GROUP_ID=3D1002<br>
> <br>
> # create the lttng session<br>
> lttng create "$SESSION_NAME" -o "$OUTPUT_DIR"<br>
> <br>
> # create the lttng ring buffer channel<br>
> lttng enable-channel --session=3D"$SESSION_NAME" --kernel ch=
annel0 \<br>
> --subbuf-size=3D16M \<br>
> --num-subbuf=3D2<br>
> <br>
> # add pid, tid, proc, and gid for tracing collection<br>
> lttng add-context --session=3D"$SESSION_NAME" --channel=3Dch=
annel0 --kernel <br>
> --type pid<br>
> lttng add-context --session=3D"$SESSION_NAME" --channel=3Dch=
annel0 --kernel <br>
> --type gid<br>
> <br>
> # enable the target probe<br>
> lttng enable-event --session=3D"$SESSION_NAME" --channel=3Dc=
hannel0 --kernel <br>
> --probe=3Dext4_get_inode_loc ext4_get_inode_loc<br>
> <br>
> # disable all tracks # NOTE: untracking doesn't make a difference<=
br>
> # lttng untrack --session=3D"$SESSION_NAME" --kernel --all -=
-gid<br>
> # lttng untrack --session=3D"$SESSION_NAME" --kernel --all -=
-pid<br>
> # lttng untrack --session=3D"$SESSION_NAME" --kernel --all -=
-vpid<br>
> # lttng untrack --session=3D"$SESSION_NAME" --kernel --all -=
-uid<br>
> # lttng untrack --session=3D"$SESSION_NAME" --kernel --all -=
-vuid<br>
> # lttng untrack --session=3D"$SESSION_NAME" --kernel --all -=
-vgid<br>
> # only enable gid 1002 track<br>
> lttng track --kernel --gid=3D1002<br>
> <br>
<br>
Could you check the output of your script after "lttng track --kernel =
<br>
--gid=3D1002"?<br>
<br>
The rest of your commands specify a session name, but this one does not. <b=
r>
I wonder if=C2=A0 it's selecting a different default session that the o=
ne <br>
you're expecting?<br>
<br>
> # start lttng lttng start $SESSION_NAME<br>
> <br>
> ```<br>
> <br>
> Thank you for your time and guidance.<br>
> <br>
> Best regards,<br>
> Amir<br>
> <br>
> -- <br>
> *Amirhossein Najafizadeh*<br>
> *PhD Student, Computer Science Department, Stony Brook University, N.Y=
.<br>
> File systems and Storage Lab (FSL)<br>
> <br>
> *<br>
> <a href=3D"mailto:[email protected]" target=3D"_blank">najafizad=
[email protected]</a> <mailto:<a href=3D"mailto:[email protected]" ta=
rget=3D"_blank">[email protected]</a>><br>
> <a href=3D"https://amirhnajafiz.github.io/" rel=3D"noreferrer" target=
=3D"_blank">https://amirhnajafiz.github.io/</a> <<a href=3D"https://amir=
hnajafiz.github.io/" rel=3D"noreferrer" target=3D"_blank">https://amirhnaja=
fiz.github.io/</a>><br>
<br>
</blockquote></div>
--000000000000c289fc064ee03d7e--