[PATCH] rust_binder: add TF_DEFER_COMPLETE flag for avoiding userspace roundtrip

Outgoing transactions are able to send a message and wait for its reply
in a single ioctl. Why not avoid a userspace roundtrip by applying the
same logic for replying to incoming messages and waiting for the next
incoming message?

Generally, when you send a reply using BC_REPLY, the kernel sends
BR_TRANSACTION_COMPLETE as a reply to BC_REPLY right away. The
BR_TRANSACTION_COMPLETE command indicates that it's safe for userspace
to free any resources associated with this message (such as embedded fds
or Binder nodes). However, the BR_TRANSACTION_COMPLETE message is
problematic because after BC_REPLY is issued, there will be a pending
message for userspace. The kernel will refuse to sleep for incoming
messages in this scenario.

The way this is handled for outgoing transaction is through a mechanism
known as deferred delivery of BR_TRANSACTION_COMPLETE. The idea is that
when you send an outgoing transaction, then we do not return to
userspace right away if BR_TRANSACTION_COMPLETE is the only pending
message. This patch adds a new flag called TF_DEFER_COMPLETE that lets
userspace opt-in to the same deferred delivery mechanism for
BR_TRANSACTION_COMPLETE when using BC_REPLY.

Given this new uapi, we can adjust sendReply in userspace libbinder
so that it writes the BC_REPLY command into mOut but does not flush the
buffer to the kernel. Then, userspace simply continues running until it
returns all the way out to the top-level joinThreadPool() loop, which
calls into the kernel to get the next incoming transaction. At this
point, mOut is flushed, sending the reply. The same ioctl then proceeds
to sleep for an incoming message.

Userspace only actually specifies TF_DEFER_COMPLETE when the Parcel does
not contain fds or refcounts on binder objects. This is because
otherwise said fd or binder node will not be freed until the binder
thread receives another incoming transaction, which could be a long
time. In the case of fds, this is especially important because delaying
fclose() can result in processes hanging because they read from a pipe
that isn't being closed due to fclose() not getting called. Note that
even if TF_DEFER_COMPLETE is not specified for this transaction, it can
still be useful to defer the BC_REPLY command, as it can still avoid a
userspace roundtrip when a new incoming transaction is available right
away.

Observing the cuttlefish logs while booting with this change shows that
there were 4297 opportunities for this optimization to kick in (that is,
boot invoked BC_REPLY 4297 times). Out of those, 3441 binder ioctls sent
and received a transaction in the same ioctl. This indicates that we
successfully eliminated a syscall on the server side for 80% of incoming
transactions. Generally, this means that a server is now able to handle
incoming messages using one syscall per incoming message (for each
incoming transaction, the syscall handles one BC_FREE_BUFFER and
BC_REPLY command, and then waits for the next incoming transaction).

Signed-off-by: Alice Ryhl <[email protected]>
---
 drivers/android/binder/defs.rs      |  3 ++-
 drivers/android/binder/thread.rs    | 10 +++++++++-
 include/uapi/linux/android/binder.h |  1 +
 3 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/drivers/android/binder/defs.rs b/drivers/android/binder/defs.rs
index 33f51b4139c7..52e4540a0dc0 100644
--- a/drivers/android/binder/defs.rs
+++ b/drivers/android/binder/defs.rs
@@ -75,7 +75,8 @@ macro_rules! pub_no_prefix {
     TF_ONE_WAY,
     TF_ACCEPT_FDS,
     TF_CLEAR_BUF,
-    TF_UPDATE_TXN
+    TF_UPDATE_TXN,
+    TF_DEFER_COMPLETE,
 );
 
 pub(crate) use uapi::{
diff --git a/drivers/android/binder/thread.rs b/drivers/android/binder/thread.rs
index 3b8520813941..d58a9168d908 100644
--- a/drivers/android/binder/thread.rs
+++ b/drivers/android/binder/thread.rs
@@ -1340,7 +1340,15 @@ fn reply_inner(self: &Arc<Self>, info: &mut TransactionInfo) -> BinderResult {
             let process = orig.from.process.clone();
             let allow_fds = orig.flags & TF_ACCEPT_FDS != 0;
             let reply = Transaction::new_reply(self, process, info, allow_fds)?;
-            self.inner.lock().push_work(completion);
+            {
+                // This performs a deferred push so that `read` can wait for the next incoming
+                // transaction without a userspace roundtrip.
+                let mut inner = self.inner.lock();
+                inner.push_work_deferred(completion);
+                // However, if `TF_DEFER_COMPLETE` is not set, then set `process_work_list` to make
+                // the push non-deferred. This forces a userspace roundtrip.
+                inner.process_work_list |= info.flags & TF_DEFER_COMPLETE == 0;
+            }
             orig.from.deliver_reply(Ok(reply), &orig, None);
             Ok(())
         })()
diff --git a/include/uapi/linux/android/binder.h b/include/uapi/linux/android/binder.h
index 701cad36de43..96e5b0184a1b 100644
--- a/include/uapi/linux/android/binder.h
+++ b/include/uapi/linux/android/binder.h
@@ -296,6 +296,7 @@ enum transaction_flags {
 	TF_ACCEPT_FDS	= 0x10,	/* allow replies with file descriptors */
 	TF_CLEAR_BUF	= 0x20,	/* clear buffer on txn complete */
 	TF_UPDATE_TXN	= 0x40,	/* update the outdated pending async txn */
+	TF_DEFER_COMPLETE = 0x80,	/* defer transaction complete to userspace */
 };
 
 struct binder_transaction_data {

---
base-commit: a13c140cc289c0b7b3770bce5b3ad42ab35074aa
change-id: 20260715-defer-complete-f1dea9af13a8

Best regards,
-- 
Alice Ryhl <[email protected]>