[PATCH 3/7] btrfs: do an initial lockless committed log transaction check during log syncing

From: Filipe Manana <[email protected]>

When calling btrfs_sync_log() one of the things we do first is to check if
the log transaction the given context belongs to was already committed
and return immediately if it was committed before. However we take the
root's log_mutex before doing the check, which adds lock contention in
case the log transaction was already committed, as this is a heavily used
mutex in case we have multiple tasks doing fsyncs or renaming and adding
hard links to files that were fsynced before in the current transaction.

So add an initial lockless check in btrfs_sync_log() to reduce lock
contention.

The following fio test was used to benchmark:

  $ cat fio-buffered-fsync.sh
  DEV=/dev/sdj
  MNT=/mnt/sdj
  MOUNT_OPTIONS=""
  MKFS_OPTIONS=""

  if [ $# -ne 6 ]; then
     echo "Use $0 NUM_JOBS FILE_SIZE IO_SIZE FSYNC_FREQ BLOCK_SIZE [write|randwrite]"
     exit 1
  fi

  NUM_JOBS=$1
  FILE_SIZE=$2
  IO_SIZE=$3
  FSYNC_FREQ=$4
  BLOCK_SIZE=$5
  WRITE_MODE=$6

  if [ "$WRITE_MODE" != "write" ] && [ "$WRITE_MODE" != "randwrite" ]; then
     echo "Invalid WRITE_MODE, must be 'write' or 'randwrite'"
     exit 1
  fi

  cat <<EOF > /tmp/fio-job.ini
  [writers]
  rw=$WRITE_MODE
  fsync=$FSYNC_FREQ
  fallocate=none
  group_reporting=1
  direct=0
  bs=$BLOCK_SIZE
  ioengine=psync
  filesize=$FILE_SIZE
  io_size=$IO_SIZE
  directory=$MNT
  numjobs=$NUM_JOBS
  EOF

  echo
  echo "Using config:"
  echo
  cat /tmp/fio-job.ini
  echo

  umount $MNT &> /dev/null
  mkfs.btrfs -f $MKFS_OPTIONS $DEV
  mount $MOUNT_OPTIONS $DEV $MNT

  fio /tmp/fio-job.ini
  umount $MNT

Running the script as:  ./fio-buffered-fsync.sh 12 64M 64M 1 4K randwrite

Before patch:

  WRITE: bw=16.6MiB/s (17.4MB/s), 16.6MiB/s-16.6MiB/s (17.4MB/s-17.4MB/s), io=768MiB (805MB), run=46235-46235msec

After patch:

  WRITE: bw=18.2MiB/s (19.0MB/s), 18.2MiB/s-18.2MiB/s (19.0MB/s-19.0MB/s), io=768MiB (805MB), run=42277-42277msec

Signed-off-by: Filipe Manana <[email protected]>
---
 fs/btrfs/tree-log.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c
index 580f2aabc065..191308982e9c 100644
--- a/fs/btrfs/tree-log.c
+++ b/fs/btrfs/tree-log.c
@@ -3306,15 +3306,21 @@ int btrfs_sync_log(struct btrfs_trans_handle *trans,
 	struct btrfs_root *log = root->log_root;
 	struct btrfs_root *log_root_tree = fs_info->log_root_tree;
 	struct btrfs_root_item new_root_item;
-	int log_transid = 0;
+	int log_transid = ctx->log_transid;
 	struct btrfs_log_ctx root_log_ctx;
 	struct blk_plug plug;
 	u64 log_root_start;
 	u64 log_root_level;
 
+	/* Avoid fist taking root->log_mutex, reduce lock contention. */
+	if (data_race(root->log_transid_committed) >= log_transid) {
+		trace_btrfs_sync_log_enter(trans, root, ctx);
+		trace_btrfs_sync_log_exit(trans, root, ctx, ctx->log_ret);
+		return ctx->log_ret;
+	}
+
 	mutex_lock(&root->log_mutex);
 	trace_btrfs_sync_log_enter(trans, root, ctx);
-	log_transid = ctx->log_transid;
 	if (root->log_transid_committed >= log_transid) {
 		trace_btrfs_sync_log_exit(trans, root, ctx, ctx->log_ret);
 		mutex_unlock(&root->log_mutex);
-- 
2.47.2