libdar polling, KDar forking...

I talked with a programmer friend last night (Mark) about interrupting 
threads. A real programming challenge it seems. The apparent way to go is 
to poll. In fact, the pthread_cancel() function, and the ENABLE and 
DISABLE flags are a type of polling. The libdar would have to have polling 
points to make it work. Mark explained that the stigma of polling comes 
from asynchronous communication, as an example. But the real problem is to 
ascertain how long you are willing to wait between polls. 

As an example, suppose you have a process that loops through a list of 
files, compressing each in turn. Let's say the program polls the outside 
world once per iteration to see if it should cancel the operation. If the 
compression time is small, then the polling interval is small, and the 
cancel operation takes effect almost immediately. 

If, on the other hand, the compression takes a long time, say 30 seconds to 
possibly several minutes for a large file or a slow computer, then the 
user will have to wait that time before the execution polls again. This 
might be an unacceptably long delay to cancel the operation.

In either case, most of the loop is involved in doing real work (the 
compression in this case) with a very small part devoted to polling. 

So a tradeoff must be made between how complicated one wants to get in the 
code --- perhaps by implementing the compression algorithm directly and 
adding new polling points --- and how much delay is acceptable for things 
like cancelling the file compression operation. If users typically 
compress only small files, then the delay will be minimal and acceptable. 
But if they compress large files, the delay associated with cancelling the 
operation can be a problem. 

For most KDar users, we might get away with a quick response most of the 
time, and then occasionally one tries to cancel during an extended period 
without a poll, and a few minute wait happens. This might be a trade-off 
that we could live with, in which case I would continue to call the libdar 
operations in separate threads.

One way to get around the problem is to call the thread_cancel function, 
and immediately return control to the user, ignoring the fact that the 
thread may still be running, and just let it finish on its own time. This 
can be undesirable because the cancelled thread may take some considerable 
time to stop, and will still potentially waste a lot of CPU cycles. 
However, it may be the case that most of the time the operation stops 
quickly and we're okay.

Another way would be to fork the process and either exec a new program 
dedicated to the task, or just run the copy of the original process. 
Communications between the parent and child could be established through a 
pipe or shared memory. Then if the user cancels the operation, we just 
kill the process, and don't worry a bit about dangling threads, locked 
mutexes, and the lot. The advantage of this technique is that we are 
guaranteed to kill the process quickly and get control back to the user. 
At least as quickly as a <Ctrl>-C on the command-line.

The issue of whether threads are better than forked processes depends on 
the application. In the case of the backup program, we might typically 
just run a backup, or a restore, by itself. I could see where we are even 
testing one archive while diffing another one. But there won't be more 
than several libdar operations going on at once. So the extra overhead 
associated with forking a kdar is not really a problem.

If I decided to go with the fork technique, I wouldn't have to worry about 
how you implement the cancelling operation in libdar, and so our lines of 
development could in principle be carried out independently from that 
point of view.

I have to get educated on forking before I make the plunge into those 
waters, but it may be a few weeks as I have some jobs to apply for and 
other work-related priorities that will eat at my spare time. But I will 
try to get the cancel operation implemented because it is important not to 
leave the user "hanging" and frustrated when they want to stop an 
operation.

JB
-- 
Johnathan K. Burchill, Ph.D.
[email protected]

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQBA4SI7rXMSOtiWrO4RApsOAJ4wBEDfYYEREy6MKFiW1CszXUtQGACfakJ2
ct5pHPtZ2leggTjycvd5f88=
=/6c9
-----END PGP SIGNATURE-----