Re: Backups not working after running out of space

Hi,

too late for Peter's issue, but I created an example on how to avoid the 
issue (or at least reduce the risk):
https://github.com/rdiff-backup/rdiff-backup/pull/947

Review welcome.

BTW, check also 
https://rdiff-backup.net/FAQ.html#how_to_handle_a_no_space_left_on_device_message_while_doing_a_backup

KR, Eric

On 15.11.2023 20:52, [email protected] wrote:
> Anyone have any suggestions?
> 
> If nothing else, is there some way I could remove appropriate files
> from the rdiff-backup-data directory to get my backups working again?
> 
> thanks,
> Peter
> 
> On 2023-11-13 13:01, Peter Canning wrote:
>> Here's the command I've been using, with -v 6 added, followed by the 
>> output it produced:
>>> C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe -v 6 backup 
>>> --print-statistics `
>>>             --exclude "H:/Lightroom/Processing Catalog/Backups" `
>>>             --exclude "H:/Lightroom/Processing Catalog/Processing 
>>> Catalog Previews.lrdata" `
>>>             --exclude "H:/Lightroom/Processing Catalog/Processing 
>>> Catalog-v12 Previews.lrdata" `
>>>             "H:/Lightroom/Processing Catalog" `
>>> \\nas4free.local\lightroom-catalogs-backups\Lightroom-Processing-Catalog.rdiff-backup
>> 
>>> C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe : WARNING: 
>>> Previous backup seems to have failed, regressing destination now
>>> At line:1 char:1
>>> + C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe -v 6 
>>> backup  ...
>>> + 
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>     + CategoryInfo          : NotSpecified: (WARNING: 
>>> Previo...destination now:String) [], RemoteException
>>>     + FullyQualifiedErrorId : NativeCommandError
>>> 
>>> WARNING: Could not find mirror metadata file. Metadata will be read 
>>> from filesystem instead
>>> Fatal Error: No metadata for time Thu Nov  9 02:35:42 2023 
>>> (1699526142) found, cannot regress
>>> * Using repository 
>>> '//nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup'
>>> * Hardlinks disabled by default on Windows
>>> * Unable to import module (py)xattr. Extended attributes not 
>>> supported on filesystem at path H:/Lightroom/Processing Catalog
>>> * Unable to import module posix1e from pylibacl package. POSIX ACLs 
>>> not supported on filesystem at path H:/Lightroom/Processing Catalog
>>> * -----------------------------------------------------------------
>>> Detected abilities for source (read only) file system:
>>>   Access control lists                         Off
>>>   Extended attributes                          Off
>>>   Windows access control lists                 On
>>>   Case sensitivity                             Off
>>>   Escape DOS devices                           On
>>>   Escape trailing spaces                       On
>>>   Mac OS X style resource forks                Off
>>>   Mac OS X Finder information                  Off
>>> -----------------------------------------------------------------
>>> * Directories on file system at path 
>>> //nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup/rdiff-backup-data/rdiff-backup.tmp.0 
>>> are not fsyncable. Assuming it's unnecessary.
>>> * Unable to import module (py)xattr. Extended attributes not 
>>> supported on filesystem at path 
>>> //nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup/rdiff-backup-data/rdiff-ba
>>> ckup.tmp.0
>>> * Unable to import module posix1e from pylibacl package. POSIX ACLs 
>>> not supported on filesystem at path 
>>> //nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup/rdiff-backup-da
>>> ta/rdiff-backup.tmp.0
>>> * -----------------------------------------------------------------
>>> Detected abilities for destination (read/write) file system:
>>>   Ownership changing                           Off
>>>   Hard linking                                 On
>>>   fsync() directories                          Off
>>>   Directory inc permissions                    Off
>>>   High-bit permissions                         On
>>>   Symlink permissions                          Off
>>>   Extended filenames                           On
>>>   Windows reserved filenames                   On
>>>   Access control lists                         Off
>>>   Extended attributes                          Off
>>>   Windows access control lists                 On
>>>   Case sensitivity                             Off
>>>   Escape DOS devices                           On
>>>   Escape trailing spaces                       On
>>>   Mac OS X style resource forks                Off
>>>   Mac OS X Finder information                  Off
>>> -----------------------------------------------------------------
>>> * Backup: escape_dos_devices = False
>>> * Backup: escape_trailing_spaces = False
>>> * Enabled use_compatible_timestamps
>>> NOTE: Symbolic links excluded by default on Windows
>>> NOTE: Regressing to date/time Thu Nov  9 02:35:42 2023
>>> * Cleaning up
>> 
>> 
>> On 2023-11-13 04:57, Patrik Dufresne patrik-at-ikus-soft.com 
>> |rdiff-backup-users| wrote:
>>> Hello Peter,
>>> 
>>> I apologize for the inconvenience you've experienced in recovering 
>>> your
>>> backup after encountering space issues. Typically, rdiff-backup can 
>>> handle
>>> recovery seamlessly following an increase in disk space. However, it 
>>> seems
>>> that the manual changes you made might have caused interference.
>>> 
>>> To assist you more effectively, could you please share the backup 
>>> command
>>> line you are currently using? Additionally, I recommend executing the
>>> command with higher verbosity by adding "-v 6." This will provide 
>>> more
>>> detailed information that can help us pinpoint the issue.
>>> 
>>> In the expected scenario, rdiff-backup should detect the interruption 
>>> in
>>> the previous backup and initiate a regression of the repository to
>>> eliminate any intermediate changes. This process is designed to 
>>> restore the
>>> health of your repository. Subsequently, the backup will resume its 
>>> normal
>>> operation.
>>> 
>>> Your cooperation in providing the requested information will greatly 
>>> aid us
>>> in resolving this issue promptly. If you have any questions or 
>>> concerns,
>>> please don't hesitate to reach out.
>>> 
>>> Thank you for your understanding and cooperation.
>>> 
>>> 
>>> Le lun. 13 nov. 2023, à 00 h 12,<[email protected]>  a écrit 
>>> :
>>> 
>>>> Using rdiff-backup 2.2.2 on Windows 10 Pro 22H2, I have a scheduled 
>>>> task
>>>> that runs my system backup PowerShell script at 2 AM every morning.  
>>>> The
>>>> script runs rdiff-backup several times to backup several different
>>>> directories.  A few days ago one of the backups failed because the 
>>>> NAS
>>>> volume containing targetdir filled up.  I took some remedial steps 
>>>> that
>>>> didn't help, but at this point I don't remember exactly what I did.  
>>>> I do
>>>> know that this first thing I tried, not thinking very careful, and
>>>> realizing that I had recently created some large files in sourcedir 
>>>> that I
>>>> didn't need any more, I deleted those files and tried doing a backup
>>>> running rdiff-backup manually (incorrectly thinking rdiff-backup 
>>>> would
>>>> remove some files from targetdir).  Of course since rdiff-backup is
>>>> creating incremental backups, and targetdir was still full, that 
>>>> backup
>>>> failed too.  At some point I increased the size of the NAS volume, 
>>>> and here
>>>> is where my memory is failing me. I probably tried running 
>>>> rdiff-backup
>>>> manually again.   Either then, or shortly thereafter, I started 
>>>> getting
>>>> "regressing destination" and then rdiff-backup failing whenever I 
>>>> tried to
>>>> backup or verify, so I attempted to follow the instructions in FAQ 
>>>> item
>>>> 15.  I did some other troubleshooting that I don't remember, but the
>>>> backups are still failing.
>>>> 
>>>> Now, running "rdiff-backup.exe backup --print-statistics  ..." 
>>>> doesn't
>>>> print the expected statistics (so I assume it didn't backup 
>>>> anything) and
>>>> produces the following output:
>>>> 
>>>> C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe : WARNING:
>>>> Previous backup seems to have failed, regressing destination now
>>>> At line:1 char:1
>>>> + C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe backup 
>>>> --pri ...
>>>> + 
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>      + CategoryInfo          : NotSpecified: (WARNING: 
>>>> Previo...destination
>>>> now:String) [], RemoteException
>>>>      + FullyQualifiedErrorId : NativeCommandError
>>>> 
>>>> WARNING: Could not find mirror metadata file. Metadata will be read 
>>>> from
>>>> filesystem instead
>>>> Fatal Error: No metadata for time Thu Nov  9 02:35:42 2023 
>>>> (1699526142)
>>>> found, cannot regress
>>>> NOTE: Symbolic links excluded by default on Windows
>>>> NOTE: Regressing to date/time Thu Nov  9 02:35:42 2023
>>>> 
>>>> Running "rdiff-backup.exe verify ..." produces the following output:
>>>> 
>>>> C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe : WARNING: 
>>>> Two
>>>> different times for current mirror found
>>>> At line:1 char:1
>>>> + C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe verify 
>>>> \\nas ...
>>>> + 
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>      + CategoryInfo          : NotSpecified: (WARNING: Two di...nt 
>>>> mirror
>>>> found:String) [], RemoteException
>>>>      + FullyQualifiedErrorId : NativeCommandError
>>>> 
>>>> WARNING: Could not find mirror metadata file. Metadata will be read 
>>>> from
>>>> filesystem instead
>>>> WARNING: Mirror metadata not found, reading from directory
>>>> 
>>>> followed by many lines saying
>>>> 
>>>> WARNING: Cannot find SHA1 digest for file ..., perhaps because this
>>>> feature was added in v1.1.1
>>>> 
>>>> and ending with the line:
>>>> NOTE: Verification found 1428 files without hash, all others could 
>>>> be
>>>> verified successfully
>>>> 
>>>> Any suggestions for how to get my backups (and verifies) working 
>>>> again?
>>>> 
>>>> thanks,
>>>> Peter
>>>> 
>>>> 
>>>> 
>>