Re: Re: file locking

A couple of points I would like to throw out.

In the long term I would like to see the afsd_service.exe support both
the IFS and SMB/CIFS interfaces at the same time.   Therefore, I would
like to see locking implemented within the cache manager.  Given the
current demand I want to see locking available for the 1.4.1 release
which will still not support the IFS.   The IFS will be targeted either
at 1.6 or 2.0 or whatever comes next.

I am in no way committed to the existing cm_Lock / cm_Unlock
implementations.   If there is no good reason to have timed locks in the
cache manager then we should re-write that code.   Although, it would be
worth spending some time to understand why the timed locks were added to
that code in the first place.   Perhaps it is due to some SMB/CIFS
behavior that really should be implemented at SMB layer and not in the
CM layer.

I have asked Asanka Herath to develop a test application to exercise the
various locking combinations.   We can use this application to observe
the current behavior of the AFS Client Service via the SMB/CIFS
interface and test the expected behavior of new code.

Jeffrey Altman


Matt Benjamin wrote:

> Eric,
> 
>>jeff suggests:
> 
> 
>>> How about we take this discussion to the openafs-win32-devel list?
>>  
>>
> Happy to do this on-list, if you both agree.  Forgive me for putting
> some snippets in from different mails.
> 
>>> Eric,
>>>
>>> Excellent.  This is the fun part of it, I assume for IFS only?  If I
>>> understand correctly, that effectively puts IFS client and traditional
>>> client on a par wrt locking?
> 
>> yes, ifs only.  the traditional client may not have functional byte-range
>> locks, but, being of the advisory type, they are not extremely useful.
> 
> I think I need some help parsing that...
> 
> I had written some trivial code to see what Win32 LockFile did in various scenarios.  Didn't attempt ones with conflicting sessions.  Running out of AFS (recent client) on a single machine, it certainly appears you can lock ranges as a reader of the Win32 API docs would expect, but of course these don't correspond to anything in AFS.  I have assumed that these are IFS locks managed by windows file sharing, and that enforcement would also be what you would expect...  (Please correct here, if I have this all wrong.)
> 
> In the Linux client, I used the Trond-enhanced posix_lock_file_wait to
> do more or less what you do for IFS in your patch.  It wasn't hard to do
> there either.  A lock request fails if, after getting the posix lock,
> the client doesn't already have, and cannot get, a corresponding
> whole-file lock in AFS.  Of course, this does not require an upcall in
> that context, etc.
> 
> If I follow the "alternatives" discussion correctly, the issues raised
> _here_
> 
> if the cache manager were to start enforcing mandatory locks (just as a
>>scenario), there would still be problems.  mapped file (or, paging i/o)
>>requests should never be rejected because of lock conflicts.  it is
>>possible to bypass the lock on a region by using mem-mapped i/o.  this is
>>an expected and necessary behaviour; not completing such a request
>>successfully is bad.  so, if all instances have locks on them, and a
>>paging request is received, the cache manager would reject the upcall
>>read/write request.  of course, in this case, the read and write calls
>>could also be modified...
> 
> could be addressed if the code responsible for doing the upcall knew
> what the RsRtl routines know about IO operations?  I'm missing how doing
> our own range reader-writer locks would address that, but, you don't
> advocate that.
> 
> In particular, it seems to me if the FsRtl routines exist, provide a
> locking implementation for free that is likely the default implmentation
> for everyone else, too ("**Lock* 'Em Up - Byte Range Locking")*, then it
> seems the more difficult argument is for _not_ using them. 
> 
> Now, when you say the protocols are incompatible, it appears that the only issues unaccounted for are related to the time of validity in the cm locking interface?  
> 
>>i found the locking protocols to be incompatible.
>>
>>the cache manager allows locks to be placed and removed using the four
>>parameters (instance, lock_type, offset, length).  additionally, locks
>>require a time-period of validity.  these are advisory locks.
>>
> 
> The time-period of validity throws me--seems to be a Windows CM-ism?  I didn't run into this in afs_lockctl, nor do I see it in afs.h, afsint.xg, and so forth.
> 
>>the ifs layer only receives read-write locks on regions:  the caller is
>>guaranteed exclusive access to the locked region.  the lock request
>>requires the four parameters (instance, offset, length, key).  however,
>>unlock requests have one of three forms:  (instance, offset, length, key),
>>(instance, key), (instance).  it is possible to remove all file locks on a
>>specified handle with a single call.  if granted, these are mandatory
>>locks.
> 
> This is not too dissimilar to what you see on the various Unixes, right?
> 
>>a simpler solution is to use the kernel's FsRtl functions to manage locks,
>>and to send the whole-file-lock request up to a daemon which spins on
>>making lock requests as often as necessary (because of expiry).  this is
>>similar to smb_WaitingLocksDaemon.  even if the lock is dropped by the cm,
>>for one of any number of reasons, the mandatory lock is still enforced
>>correctly in the kernel.  this is what i would like to do.
> 
> Aside from the the lock expiry "spinning" behavior, the solution you propose does sound to me like the analog of the behavior I implemented, Nathan Neulinger described, etc.
> 
> Would we need a lock-re-upping daemon if the Windows CM worked like the Unix cms in this regard? 
> 
> Matt
> 
> my work-up of locking:
> 
> 
>>
>>>
>>>i could modify the cache manager's locking functions to conform to the ifs
>>>model for lock requests.  the ifs redirector would pass the lock requests
>>>to the cache manager.  for a read request, the cm would have to be
>>>consulted to make sure there is no lock conflict (granted locks are
>>>mandatory), and then the i/o request would be processed.  if the lock is
>>>destroyed by the cm, then improper i/o could take place.  this is a
>>>serious problem (ms office, for instance, may crash or fail), as i'm sure
>>>you know.
>>>
>>>if the cache manager were to start enforcing mandatory locks (just as a
>>>scenario), there would still be problems.  mapped file (or, paging i/o)
>>>requests should never be rejected because of lock conflicts.  it is
>>>possible to bypass the lock on a region by using mem-mapped i/o.  this is
>>>an expected and necessary behaviour; not completing such a request
>>>successfully is bad.  so, if all instances have locks on them, and a
>>>paging request is received, the cache manager would reject the upcall
>>>read/write request.  of course, in this case, the read and write calls
>>>could also be modified...
>>>
>>>another possibility is to not change the cache manager's code, and to keep
>>>a list of locks in the kernel.  then, an appropriate number of unlock
>>>requests could be sent via upcalls.  this is redundant at best.
>>>
>>>a simpler solution is to use the kernel's FsRtl functions to manage locks,
>>>and to send the whole-file-lock request up to a daemon which spins on
>>>making lock requests as often as necessary (because of expiry).  this is
>>>similar to smb_WaitingLocksDaemon.  even if the lock is dropped by the cm,
>>>for one of any number of reasons, the mandatory lock is still enforced
>>>correctly in the kernel.  this is what i would like to do.
>>
> 
> 
> 
> 
> Eric Williams wrote:
> 
>>On Wed, 20 Jul 2005, Matt Benjamin wrote:
>>
>>  
>>
>>>Eric,
>>>
>>>Excellent.  This is the fun part of it, I assume for IFS only?  If I
>>>understand correctly, that effectively puts IFS client and traditional
>>>client on a par wrt locking?
>>>    
>>>
>>
>>yes, ifs only.  the traditional client may not have functional byte-range
>>locks, but, being of the advisory type, they are not extremely useful.
>>
>>  
>>
> 
> -- 
> 
> Matt Benjamin
> 
> The Linux Box
> 206 South Fifth Ave. Suite 150
> Ann Arbor, MI  48104
> 
> http://linuxbox.com
> 
> tel. 734-761-4689
> fax. 734-769-8938
> cel. 734-216-5309
> 
>