[PATCH 00/15] ceph: add binary logging (BLOG) for CephFS

Alex Markuze <[email protected]>
Newsgroups org.kernel.vger.ceph-devel,org.kernel.vger.linux-kernel
Message-ID <[email protected]>
This series adds a binary logging subsystem for the CephFS kernel
client.  On hot I/O paths every dout() call formats a printk-style
string even when nobody is reading the log.  BLOG sidesteps that cost
by storing compact binary records keyed by a source-ID; the records
are decoded offline or via debugfs.

All new code lives under fs/ceph/ and include/linux/ceph/.  There
are no changes outside the Ceph subsystem apart from the kselftests.

Design
------

Allocation.  A page-fragment allocator hands out buffer slices without
taking any global lock.  Freed fragments are recycled through per-CPU
magazine batches so the common-case log path is allocation-free.
Filesystem-reachable allocation uses GFP_NOFS to avoid reclaim
recursion.

Source-ID caching.  Each (file, func, line, fmt) tuple is registered
once in a per-logger rhashtable and assigned a u32 source-ID.  A
per-callsite cache with smp_store_release / smp_load_acquire avoids
the rhashtable lookup after the first hit.  A generation counter on
the logger prevents stale cache hits across mount/unmount cycles.

Per-superblock ownership.  Every mounted CephFS instance (including
-o noshare mounts) gets its own blog_module_context, so enable/disable
and debugfs are fully isolated between mounts.

Context propagation.  ceph_blog_enter() stashes the owning
ceph_fs_client and an optional mds_request pointer in a
ceph_journal_info that lives in current->journal_info.  Nested VFS
re-entry inherits the context only when the owning fsc matches,
preventing cross-mount leaks.  Task pointers used as hash keys are
not pinned; stale entries are retired lazily via PID-mismatch
detection on reuse.

debugfs.  Per-superblock files appear under
  /sys/kernel/debug/ceph/<fsid>/blog/{entries,stats,sources,clients}
plus a bool "enable" knob and a write-only "clear" trigger.

Logging macros.  bout() and boutc() are drop-in replacements for
dout() and doutc(), gated on fsc->blog_enabled.  The serializer
supports all kernel printf specifiers used by Ceph debug logs
including %o.

Patch breakdown
---------------

  01-06  BLOG infrastructure, one patch per compilation unit:
         headers, deserializer, pagefrag allocator, magazine batcher,
         logger core, per-module context manager.

  07     Ceph BLOG scaffolding -- ceph_blog.h, blog_client.c,
         super.h / libceph.h additions, Makefile wiring.

  08     bout/boutc macro definitions in ceph_debug.h.

  09     MDS request plumbing -- switch __do_request() to
         ceph_blog_enter() / ceph_blog_exit().

  10     debugfs interface -- blog_debugfs.c and lifecycle hooks.

  11-14  Callsite conversions -- dout/doutc -> bout/boutc, grouped
         by subsystem: inodes+dirs, data I/O, caps+snaps, helpers.

  15     Selftests and MAINTAINERS update.

Alex Markuze (15):
  ceph: add BLOG public headers
  ceph: add BLOG deserialization support
  ceph: add BLOG page-fragment allocator
  ceph: add BLOG magazine batch allocator
  ceph: add BLOG logger core
  ceph: add BLOG per-module context management
  ceph: add Ceph BLOG scaffolding
  ceph: add bout and boutc wrappers for BLOG
  ceph: switch MDS request plumbing to struct ceph_journal_info
  ceph: add BLOG debugfs interface
  ceph: convert VFS inode and directory paths to bout
  ceph: convert VFS data I/O paths to bout
  ceph: convert capability and snapshot paths to bout
  ceph: convert remaining helper paths to bout
  selftests: add Ceph BLOG smoke test and collection helpers

 MAINTAINERS                                   |   1 +
 fs/ceph/Makefile                              |   6 +
 fs/ceph/addr.c                                | 168 ++---
 fs/ceph/blog_batch.c                          | 312 ++++++++++
 fs/ceph/blog_client.c                         | 266 ++++++++
 fs/ceph/blog_core.c                           | 424 +++++++++++++
 fs/ceph/blog_debugfs.c                        | 434 +++++++++++++
 fs/ceph/blog_des.c                            | 342 ++++++++++
 fs/ceph/blog_module.c                         | 584 ++++++++++++++++++
 fs/ceph/blog_pagefrag.c                       |  95 +++
 fs/ceph/caps.c                                | 252 ++++----
 fs/ceph/crypto.c                              |  14 +-
 fs/ceph/debugfs.c                             |  23 +-
 fs/ceph/dir.c                                 | 290 ++++++---
 fs/ceph/export.c                              |  83 ++-
 fs/ceph/file.c                                | 274 +++++---
 fs/ceph/inode.c                               | 233 ++++---
 fs/ceph/ioctl.c                               |   6 +-
 fs/ceph/locks.c                               |  66 +-
 fs/ceph/mds_client.c                          | 293 ++++-----
 fs/ceph/mdsmap.c                              |   8 +-
 fs/ceph/quota.c                               |   2 +-
 fs/ceph/snap.c                                |  66 +-
 fs/ceph/super.c                               |  58 +-
 fs/ceph/super.h                               |   5 +
 fs/ceph/xattr.c                               |  88 ++-
 include/linux/ceph/blog.h                     |   0
 include/linux/ceph/blog_batch.h               |   0
 include/linux/ceph/blog_des.h                 |   0
 include/linux/ceph/blog_module.h              |   0
 include/linux/ceph/blog_pagefrag.h            |   0
 include/linux/ceph/blog_ser.h                 |   0
 include/linux/ceph/ceph_blog.h                | 284 +++++++++
 include/linux/ceph/ceph_debug.h               |  44 +-
 include/linux/ceph/libceph.h                  |   2 +
 tools/testing/selftests/Makefile              |   1 +
 tools/testing/selftests/ceph_blog/Makefile    |   7 +
 .../selftests/ceph_blog/blog_analyze.sh       | 124 ++++
 .../selftests/ceph_blog/blog_collect.sh       | 116 ++++
 .../selftests/ceph_blog/blog_smoke_test.sh    | 363 +++++++++++
 tools/testing/selftests/ceph_blog/config      |   2 +
 41 files changed, 4583 insertions(+), 753 deletions(-)
 create mode 100644 fs/ceph/blog_batch.c
 create mode 100644 fs/ceph/blog_client.c
 create mode 100644 fs/ceph/blog_core.c
 create mode 100644 fs/ceph/blog_debugfs.c
 create mode 100644 fs/ceph/blog_des.c
 create mode 100644 fs/ceph/blog_module.c
 create mode 100644 fs/ceph/blog_pagefrag.c
 create mode 100644 include/linux/ceph/blog.h
 create mode 100644 include/linux/ceph/blog_batch.h
 create mode 100644 include/linux/ceph/blog_des.h
 create mode 100644 include/linux/ceph/blog_module.h
 create mode 100644 include/linux/ceph/blog_pagefrag.h
 create mode 100644 include/linux/ceph/blog_ser.h
 create mode 100644 include/linux/ceph/ceph_blog.h
 create mode 100644 tools/testing/selftests/ceph_blog/Makefile
 create mode 100644 tools/testing/selftests/ceph_blog/blog_analyze.sh
 create mode 100644 tools/testing/selftests/ceph_blog/blog_collect.sh
 create mode 100644 tools/testing/selftests/ceph_blog/blog_smoke_test.sh
 create mode 100644 tools/testing/selftests/ceph_blog/config

-- 
2.34.1
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.