More: Re: Corrupted file names

Hello, I have further documented the file corruption problems I 
described in my postings on 2024-02-20, 2024-03-05 and 2024-03-07.

Two separate but similar ASUS notebook computers were involved in these 
cases: (both syncing with the same profile under unison 2.53.4 between a 
Ubuntu 22.04 ntfs partition on an internal SSD and a ntfs partition on a 
new Samsung external SSD [the same for both computers]): one computer 
six months old (called ASUS14) and the other five years old (called ASUS15).

1. In the first event reported (ASUS14) on 2024-02-20 a spurious file 
with the name "[old-filename].ntfs-3g-0000000001" was detected by unison 
on the internal SSD and skipped (Error in querying file information) 
then detected and copied from internal to external disk on 2024-02-18, 
and finally and detected but skipped on 2024-02-20 (Error in querying 
file information). The original file was not damaged and was copied 
correctly. This problem could only be resolved by running "chkdsk /f" in 
Windows.

2. In the second event reported [but chronologically the first] (ASUS15) 
66 spurious files were definitely created by unison on 2024-02-12 (but 
only reported on 2024-02-20). The filenames had the form 
".unison.[old-filename][32-char hexcode][.unison.tmp]", and the original 
files were not damaged. I tried to re-check the log file but found that 
the log of this run had disappeared (could someone inform on the default 
maximum size of the log file and how to change it if possible?). I note 
however, that the profile at that time had the attributes set to 
"dontchmod = true" and "perms = 0" which Tõivo recommended that I remove 
on 2024-02-26 (and which I subsequently removed prior to the third event 
below). I have the 66 files in trash folders, and could inform on them 
if useful; they are all apparently well formed copies of either the 
previous or final versions synced in the unison run.

3. In the third event (ASUS15), a spurious file 
"[old-filename].ntfs-3g-0000000001" was detected on the internal SSD by 
unison on 2024-03-07 and skipped (Error in querying file information). 
The original file was not damaged and was updated in the same run. This 
problem could only be resolved by running chkdsk in Windows.

The external SSD shows clean with chkdsk under Windows. There is, 
however, a Windows trash entry dated 2024-02-18 for the file of the 
first event (which had apparently been copied by unison then found by 
the ntfs driver to be improper).

So in summary we have two examples of a single spurious defective file 
having been introduced by the ngfs-3g driver; although these files, 
which had to be removed in Windows, were detected by unison, there is no 
evidence for now that unison had anything to do with their creation. 
Regarding the 66 spurious files created by unison in a single run, I do 
not have any concrete information on how this happened; it would be nice 
if we knew in order to prevent it from happening again.

Subsequently I have run unison three times on the same profile without 
any errors, and am continuing to maintain a log.

Thanks, will get back as appropriate but any comments always welcome,

     John



-------- Message transféré --------
Sujet : 	Re: [unison-users] Corrupted file names
Date : 	Sat, 9 Mar 2024 11:56:42 +0100
De : 	John Rose <[email protected]>
Pour : 	unison-users <[email protected]>



Thanks Greg (and also to Dale for his subsequent posting on perceived 
Linux failings concerning NTFS management).

There are zillions of test tools available, and I really don't know 
where to start. My hunch is that the corruption is occurring only on my 
five-year old internal Asus SSD and not on the new external Samsung SSD. 
A quick search on the web shows that users seem to be more worried about 
performance indicators than about occasional corruption. I propose to 
try to find expert advice on whether tweaking any of the FSTAB 
parameters could help to stabilize the system, and also to start a 
rigorous log to try to understand the problem better. The other options 
like an all-Linux or all-Windows system are not very appealing for the 
moment.

Best regards,

     John

Le 07/03/2024 à 19:40, Greg Troxel a écrit :
> John Rose <[email protected]> writes:
>
>> So I went into Windows and ran chkdisk and found that there was indeed
>> and index problem with the above file. Once fixed everything syncs
>> OK. I am not sure whether this file corruption problem happened at the
>> same time as the similar ones reported on 20 February or later.
> Thanks for continuing to look into this and figure out more about what
> happened.
>
> It is basically a 100% conclusion that user programs cannot cause file
> system inconsistencies.
>
>> Quite disturbing and annoying, still no clear idea about why it
>> happened and what to do? Even if there is no direct incrimination of
>> unison, perhaps it is possible that the high volume IO is enough to
>> push the Ubuntu management of ntfs beyond its limit? I will post
>> further news if the problem continues.
>
> I think it is plausible that high-volume IO is troublesome while low
> loads are not. Any simllar sync program will neessarily call stat(2) on
> efery file, and when actually syncing will read and write a lot.
>
> My only thought (other than "Don't use windows at all." :-) is that
> there must be filesystem exerciser/tester programs that do a large
> number of operations, from multiple threads, and check the results.
> You might find and run such a tool.
>
> Greg

-- 
************

                John B. Rose
                1 Bis rue des Châtre-Sacs
                92310 Sèvres, France

                Email:[email protected]

-- 
To unsubscribe from this group and stop receiving emails from it, send an email [email protected].

-- 
************

                John B. Rose
                1 Bis rue des Châtre-Sacs
                92310 Sèvres, France

                Email:[email protected]

-- 
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].