[RFC] Do manifest_put in a background task.

Hi all,

This isn't meant to be a final patch, but more of a starting point up 
for discussion. Before it's a finished patch it'll need some autoconf 
work, some adjustment to the test-cases, and probably a config option to 
turn it off. (Another question is whether to do it on cache miss, as in 
the patch, or in all cases to take the stats update into the background?)

The idea is simple: don't spend foreground time doing work that isn't on 
the critical path.

My profiling (using valgrind --tool=callgrind) suggests that ccache 
spends approximately 10% of its run time running manifest_put, and this 
happens *after* ccache has already acquired all the results the calling 
build needs to continue.

The time isn't wasted, it's an investment in the future, but it's still 
slowing down the current build, and a build with cache-misses is already 
a slow build.

I've done some basic wall-clock benchmarking, and without this patch 
binutils 2.22 takes 2m:39s to build with an empty cache, on my machine, 
compared to 2m:25s if I use this patch. (The measurements were not 
terribly scientific, and repeated runs gave fairly noisy results, but 
those times are fairly representative.)

In addition to just being nice-to-have on its own, this might also open 
up some previously-too-expensive possibilities for fixing the flaws in 
direct-mode caching, or even making improvements.

Thoughts?

Andrew

_______________________________________________
ccache mailing list
[email protected]
https://lists.samba.org/mailman/listinfo/ccache

From 6522b925ae2631c50e7c6dd841f172377a19239f Mon Sep 17 00:00:00 2001
From: Andrew Stubbs <[email protected]>
Date: Tue, 20 Nov 2012 21:30:38 +0000
Subject: [PATCH] Do manifest_put in a background task.

---
 ccache.c |   10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/ccache.c b/ccache.c
index f899bf9..6cfd261 100644
--- a/ccache.c
+++ b/ccache.c
@@ -1324,6 +1324,16 @@ from_cache(enum fromcache_call_mode mode, bool put_object_in_manifest)
 	    && !conf->read_only) {
 		struct stat st;
 		size_t old_size = 0; /* in bytes */
+
+		/* We now have all the data we need to exit, but we've yet to
+		   create any manifest files. However, the user isn't
+		   interested in that, so we fork into the background to let
+		   them get on with their build. */
+		int pid = fork();
+		if (pid > 0)
+		  /* This is the exit point for the foreground process. */
+		  _exit(0);
+
 		if (stat(manifest_path, &st) == 0) {
 			old_size = file_size(&st);
 		}
-- 
1.7.10.4