Re: Backups not working after running out of space

FWIW, my script(s) run "df -h" on the volume (as suggested in the FAQ 
entry) before and after so I can see the effect of my "remove increments 
--older-than 1Y" and "backup --print-statistics" invocations, so I knew 
was down to 5% free, but I also (thought I) had a good idea of the rate 
of growth, so I wasn't tooworried. Unfortunately, a software update (to 
Adobe Lightroom) caused the size of my sourcedir to increase way more 
than usual, resulting in a much larger than usual increment and 
targetdir running out of space.   The frustrating part is that the new 
version of Lightroom has a bug 
(https://community.adobe.com/t5/lightroom-classic-bugs/p-local-adjustement-preset-incorrectly-listed-in-the-history-panel-as-mask1-update-point-colors/idc-p/14222414) 
so I reverted to the previous version after a couple of days.

Your tools/misc/verify_disk_space.sh script is a great idea.  I'm going 
to incorporate it into my backup script(s).

Thanks for adding it and announcing it here.

- Peter

On 2023-11-16 23:00, Eric L. ewl+rdiffbackup-at-lavar.de 
|rdiff-backup-users| wrote:
> Hi,
>
> too late for Peter's issue, but I created an example on how to avoid 
> the issue (or at least reduce the risk):
> https://github.com/rdiff-backup/rdiff-backup/pull/947
>
> Review welcome.
>
> BTW, check also 
> https://rdiff-backup.net/FAQ.html#how_to_handle_a_no_space_left_on_device_message_while_doing_a_backup
>
> KR, Eric
>
> On 15.11.2023 20:52, [email protected] wrote:
>> Anyone have any suggestions?
>>
>> If nothing else, is there some way I could remove appropriate files
>> from the rdiff-backup-data directory to get my backups working again?
>>
>> thanks,
>> Peter
>>
>> On 2023-11-13 13:01, Peter Canning wrote:
>>> Here's the command I've been using, with -v 6 added, followed by the 
>>> output it produced:
>>>> C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe -v 6 
>>>> backup --print-statistics `
>>>>             --exclude "H:/Lightroom/Processing Catalog/Backups" `
>>>>             --exclude "H:/Lightroom/Processing Catalog/Processing 
>>>> Catalog Previews.lrdata" `
>>>>             --exclude "H:/Lightroom/Processing Catalog/Processing 
>>>> Catalog-v12 Previews.lrdata" `
>>>>             "H:/Lightroom/Processing Catalog" `
>>>> \\nas4free.local\lightroom-catalogs-backups\Lightroom-Processing-Catalog.rdiff-backup 
>>>>
>>>
>>>> C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe : WARNING: 
>>>> Previous backup seems to have failed, regressing destination now
>>>> At line:1 char:1
>>>> + C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe -v 6 
>>>> backup  ...
>>>> + 
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>     + CategoryInfo          : NotSpecified: (WARNING: 
>>>> Previo...destination now:String) [], RemoteException
>>>>     + FullyQualifiedErrorId : NativeCommandError
>>>>
>>>> WARNING: Could not find mirror metadata file. Metadata will be read 
>>>> from filesystem instead
>>>> Fatal Error: No metadata for time Thu Nov  9 02:35:42 2023 
>>>> (1699526142) found, cannot regress
>>>> * Using repository 
>>>> '//nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup'
>>>> * Hardlinks disabled by default on Windows
>>>> * Unable to import module (py)xattr. Extended attributes not 
>>>> supported on filesystem at path H:/Lightroom/Processing Catalog
>>>> * Unable to import module posix1e from pylibacl package. POSIX ACLs 
>>>> not supported on filesystem at path H:/Lightroom/Processing Catalog
>>>> * -----------------------------------------------------------------
>>>> Detected abilities for source (read only) file system:
>>>>   Access control lists                         Off
>>>>   Extended attributes                          Off
>>>>   Windows access control lists                 On
>>>>   Case sensitivity                             Off
>>>>   Escape DOS devices                           On
>>>>   Escape trailing spaces                       On
>>>>   Mac OS X style resource forks                Off
>>>>   Mac OS X Finder information                  Off
>>>> -----------------------------------------------------------------
>>>> * Directories on file system at path 
>>>> //nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup/rdiff-backup-data/rdiff-backup.tmp.0 
>>>> are not fsyncable. Assuming it's unnecessary.
>>>> * Unable to import module (py)xattr. Extended attributes not 
>>>> supported on filesystem at path 
>>>> //nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup/rdiff-backup-data/rdiff-ba
>>>> ckup.tmp.0
>>>> * Unable to import module posix1e from pylibacl package. POSIX ACLs 
>>>> not supported on filesystem at path 
>>>> //nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup/rdiff-backup-da
>>>> ta/rdiff-backup.tmp.0
>>>> * -----------------------------------------------------------------
>>>> Detected abilities for destination (read/write) file system:
>>>>   Ownership changing                           Off
>>>>   Hard linking                                 On
>>>>   fsync() directories                          Off
>>>>   Directory inc permissions                    Off
>>>>   High-bit permissions                         On
>>>>   Symlink permissions                          Off
>>>>   Extended filenames                           On
>>>>   Windows reserved filenames                   On
>>>>   Access control lists                         Off
>>>>   Extended attributes                          Off
>>>>   Windows access control lists                 On
>>>>   Case sensitivity                             Off
>>>>   Escape DOS devices                           On
>>>>   Escape trailing spaces                       On
>>>>   Mac OS X style resource forks                Off
>>>>   Mac OS X Finder information                  Off
>>>> -----------------------------------------------------------------
>>>> * Backup: escape_dos_devices = False
>>>> * Backup: escape_trailing_spaces = False
>>>> * Enabled use_compatible_timestamps
>>>> NOTE: Symbolic links excluded by default on Windows
>>>> NOTE: Regressing to date/time Thu Nov  9 02:35:42 2023
>>>> * Cleaning up
>>>
>>>
>>> On 2023-11-13 04:57, Patrik Dufresne patrik-at-ikus-soft.com 
>>> |rdiff-backup-users| wrote:
>>>> Hello Peter,
>>>>
>>>> I apologize for the inconvenience you've experienced in recovering 
>>>> your
>>>> backup after encountering space issues. Typically, rdiff-backup can 
>>>> handle
>>>> recovery seamlessly following an increase in disk space. However, 
>>>> it seems
>>>> that the manual changes you made might have caused interference.
>>>>
>>>> To assist you more effectively, could you please share the backup 
>>>> command
>>>> line you are currently using? Additionally, I recommend executing the
>>>> command with higher verbosity by adding "-v 6." This will provide more
>>>> detailed information that can help us pinpoint the issue.
>>>>
>>>> In the expected scenario, rdiff-backup should detect the 
>>>> interruption in
>>>> the previous backup and initiate a regression of the repository to
>>>> eliminate any intermediate changes. This process is designed to 
>>>> restore the
>>>> health of your repository. Subsequently, the backup will resume its 
>>>> normal
>>>> operation.
>>>>
>>>> Your cooperation in providing the requested information will 
>>>> greatly aid us
>>>> in resolving this issue promptly. If you have any questions or 
>>>> concerns,
>>>> please don't hesitate to reach out.
>>>>
>>>> Thank you for your understanding and cooperation.
>>>>
>>>>
>>>> Le lun. 13 nov. 2023, à 00 h 12,<[email protected]>  a écrit :
>>>>
>>>>> Using rdiff-backup 2.2.2 on Windows 10 Pro 22H2, I have a 
>>>>> scheduled task
>>>>> that runs my system backup PowerShell script at 2 AM every 
>>>>> morning.  The
>>>>> script runs rdiff-backup several times to backup several different
>>>>> directories.  A few days ago one of the backups failed because the 
>>>>> NAS
>>>>> volume containing targetdir filled up.  I took some remedial steps 
>>>>> that
>>>>> didn't help, but at this point I don't remember exactly what I 
>>>>> did.  I do
>>>>> know that this first thing I tried, not thinking very careful, and
>>>>> realizing that I had recently created some large files in 
>>>>> sourcedir that I
>>>>> didn't need any more, I deleted those files and tried doing a backup
>>>>> running rdiff-backup manually (incorrectly thinking rdiff-backup 
>>>>> would
>>>>> remove some files from targetdir).  Of course since rdiff-backup is
>>>>> creating incremental backups, and targetdir was still full, that 
>>>>> backup
>>>>> failed too.  At some point I increased the size of the NAS volume, 
>>>>> and here
>>>>> is where my memory is failing me. I probably tried running 
>>>>> rdiff-backup
>>>>> manually again.   Either then, or shortly thereafter, I started 
>>>>> getting
>>>>> "regressing destination" and then rdiff-backup failing whenever I 
>>>>> tried to
>>>>> backup or verify, so I attempted to follow the instructions in FAQ 
>>>>> item
>>>>> 15.  I did some other troubleshooting that I don't remember, but the
>>>>> backups are still failing.
>>>>>
>>>>> Now, running "rdiff-backup.exe backup --print-statistics ..." doesn't
>>>>> print the expected statistics (so I assume it didn't backup 
>>>>> anything) and
>>>>> produces the following output:
>>>>>
>>>>> C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe : WARNING:
>>>>> Previous backup seems to have failed, regressing destination now
>>>>> At line:1 char:1
>>>>> + C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe backup 
>>>>> --pri ...
>>>>> + 
>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>      + CategoryInfo          : NotSpecified: (WARNING: 
>>>>> Previo...destination
>>>>> now:String) [], RemoteException
>>>>>      + FullyQualifiedErrorId : NativeCommandError
>>>>>
>>>>> WARNING: Could not find mirror metadata file. Metadata will be 
>>>>> read from
>>>>> filesystem instead
>>>>> Fatal Error: No metadata for time Thu Nov  9 02:35:42 2023 
>>>>> (1699526142)
>>>>> found, cannot regress
>>>>> NOTE: Symbolic links excluded by default on Windows
>>>>> NOTE: Regressing to date/time Thu Nov  9 02:35:42 2023
>>>>>
>>>>> Running "rdiff-backup.exe verify ..." produces the following output:
>>>>>
>>>>> C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe : 
>>>>> WARNING: Two
>>>>> different times for current mirror found
>>>>> At line:1 char:1
>>>>> + C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe verify 
>>>>> \\nas ...
>>>>> + 
>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>      + CategoryInfo          : NotSpecified: (WARNING: Two di...nt 
>>>>> mirror
>>>>> found:String) [], RemoteException
>>>>>      + FullyQualifiedErrorId : NativeCommandError
>>>>>
>>>>> WARNING: Could not find mirror metadata file. Metadata will be 
>>>>> read from
>>>>> filesystem instead
>>>>> WARNING: Mirror metadata not found, reading from directory
>>>>>
>>>>> followed by many lines saying
>>>>>
>>>>> WARNING: Cannot find SHA1 digest for file ..., perhaps because this
>>>>> feature was added in v1.1.1
>>>>>
>>>>> and ending with the line:
>>>>> NOTE: Verification found 1428 files without hash, all others could be
>>>>> verified successfully
>>>>>
>>>>> Any suggestions for how to get my backups (and verifies) working 
>>>>> again?
>>>>>
>>>>> thanks,
>>>>> Peter
>>>>>
>>>>>
>>>>>
>>>
>
>