[PATCH v5 9/9] documentation: update stapsdt docs to describe wildcard support

We now have a limited form of wildcard support; document it.

Signed-off-by: Alan Maguire <[email protected]>
---
 .../reference/dtrace_providers_stapsdt.md     | 91 ++++++++++++++++++-
 1 file changed, 86 insertions(+), 5 deletions(-)

diff --git a/doc/userguide/reference/dtrace_providers_stapsdt.md b/doc/userguide/reference/dtrace_providers_stapsdt.md
index cb6ce71f..9bbeb060 100644
--- a/doc/userguide/reference/dtrace_providers_stapsdt.md
+++ b/doc/userguide/reference/dtrace_providers_stapsdt.md
@@ -29,9 +29,29 @@ library source code as follows:
 #define _SDT_HAS_SEMAPHORES 1
 #include <sys/sdt.h>
 
+/* This complicated declaration of semaphores is needed to ensure that
+ * they work correctly; the kernel manages the semaphore count.
+ * The first three definitions simplify semaphore definiton and use;
+ * after these are defined, new semaphores are added by adding
+ *
+ * SEMA_DEFINE(provider_probename_semaphore);
+ *
+ * and used to guard probes with
+ *
+ * if (SEMA_ENABLED(provider_probename_semaphore)) ...
+ */
+#define SEC(name) __attribute__((section(name), used))
+#define SEMA_DEFINE(name)						\
+	__extension__ unsigned short name __attribute__((unused))	\
+	SEC(".probes")
+#define SEMA_ENABLED(name)                                      \
+        __builtin_expect (name, 0)
+
+SEMA_DEFINE(myprovider_myprobe_semaphore);
+
 unsigned short myprovider_myprobe_semaphore = 0;
 
-        if (myprovider_myprobe_semaphore) {
+        if (SEMA_ENABED(myprovider_myprobe)) {
             STAP_PROBEn(myprovider, myprobe, ...);
         }
 ```
@@ -52,12 +72,25 @@ DTrace will pass semaphore details to the kernel for reference counting.)
 The `stapsdt` provider also supports probes created dynamically via `libstapsdt`.
 See [https://github.com/linux-usdt/libstapsdt](https://github.com/linux-usdt/libstapsdt).
 
-In your D scripts, the provider name must have the pid for your process
+In your D scripts, two forms of provider specification are supported.
+
+In the first, the provider name must have the pid for your process
 appended.  For the provider shown above, you might have `myprovider12345` for
-pid 12345.  No wildcard may be used, but a symbolic value like `$target` may.
+pid 12345.  A symbolic value like `$target` may be used also.
+
+In the second approach, a wildcard may be used, but it must be used
+in combination with the path or name of binary or library.  In the case
+where only a binary or library name is specified, DTrace will use
+PATH or LD_LIBRARY_PATH to find the absolute path and add instrumentation.
+Probes are added on a file basis, so it is vital to specify the correct
+file that contains the probe;  unlike in the per-process case we need to
+know if a probe is delivered in a library or in a program for example.
 
 In your D scripts, the `stapsdt` module name may be `a.out`, just as with
 the [`pid` provider](dtrace_providers_pid.md#dt_ref_pidprobes_prov).
+However as noted above, for a wildcarded pid a filename discoverable
+via PATH or LD_LIBRARY_PATH must be specified so that DTrace can find
+the file to instrument.
 
 In probe names, two consecutive underscores \(`__`\) become a dash \(`-`\)
 just as with USDT probe names.  For example, if a probe is defined in source
@@ -73,12 +106,19 @@ might place in application or library source code:
 #define _SDT_HAS_SEMAPHORES 1
 #include <sys/sdt.h>
 
-unsigned short myprovider_myprobe_semaphore = 0;
+#define SEC(name) __attribute__((section(name), used))
+#define SEMA_DEFINE(name)                                               \
+	__extension__ unsigned short name __attribute__((unused))       \
+	SEC(".probes")
+#define SEMA_ENABLED(name)                                      \
+	__builtin_expect (name, 0)
+
+SEMA_DEFINE(myprovider_myprobe_semaphore);
 
 int
 main(int argc, char **argv)
 {
-    if (myprovider_myprobe_semaphore) {
+    if (SEMA_ENABLED(myprovider_myprobe_semaphore)) {
         printf("the probe is enabled\n");
         [... do something expensive to prepare probe arguments? ...]
         STAP_PROBE3(myprovider, myprobe, argc, argv[0], 18);
@@ -147,6 +187,13 @@ Displaying notes found in: .note.stapsdt
     Arguments: -4@-4(%rbp) 8@%rax -4@$18
 ```
 
+Alternatively we can list the probes in the object via DTrace
+provided we know the provider name:
+
+```
+$ dtrace -lm 'myprovider*:/path2/a.out'
+```
+
 Notice that the instrumented source code includes the header file
 `<sys/sdt.h>`.  On Oracle Linux, this file is installed
 by the RPM package `systemtap-sdt-devel`, but the package also
@@ -154,6 +201,40 @@ installs `/usr/bin/dtrace`.  So be sure to have `/usr/sbin`
 in front of `/usr/bin` in your path, or explicitly specify
 `/usr/sbin/dtrace` whenever you want to use `dtrace`.
 
+We can also specify systemwide probes for tracing; for example when
+Python 3 is built `--with-dtrace` we can do the following to show the top 5
+function-entry calls system-wide:
+
+```
+$ sudo dtrace -qn 'python*:libpython3.6m.so::function-entry {
+	@c[execname,stringof(arg0),stringof(arg1)] = count();
+}
+END {
+	trunc(@c, 5);
+	printf("%10s %35s %20s %10s\n", "PROG", "FILE", "FUNC", "COUNT");
+	printa("%10s %35s %20s %@10d\n", @c);
+}'
+^C
+      PROG                                FILE                 FUNC      COUNT
+     tuned   /usr/lib64/python3.6/threading.py             __exit__          5
+     tuned   /usr/lib64/python3.6/threading.py     _acquire_restore          5
+     tuned   /usr/lib64/python3.6/threading.py            _is_owned          5
+     tuned   /usr/lib64/python3.6/threading.py        _release_save          5
+     tuned   /usr/lib64/python3.6/threading.py                 wait         10
+```
+
+So we see the top 5 functions called were all called by `tuned`.
+
+We specified `libpython.3.6m.so` because that is where the probes are
+defined; in some cases they are in the python3 program itself, and cavets
+mentioned above about identifying whether probes are in a library or a
+program (in this case libypthon3.6.m.so or python3.6) apply.  To determine
+probe location we can use `dtrace -lm` as described above.
+
+DTrace will expand the paths of programs and libraries using
+PATH and LD_LIBRARY_PATH respectively; the absolute path can
+also be used if needed.
+
 ## stapsdt Stability <a id="dt_ref_stapsdtstab_prov">
 
 The `stapsdt` provider uses DTrace's stability mechanism to describe its stabilities.
-- 
2.43.5