Re: Amazon cloud driver fails with large part sizes
Robert Heller <[email protected]>
| Newsgroups | gmane.comp.bacula.user |
|---|---|
| Organization | Deepwoods Software |
| Message-ID | <[email protected]> |
I am using 9.6.7-7 (adapted from the Debian 12 distributed version) on both Debian 12/ARM64 (Raspberry Pi 5) and Debian 12/AMD64 (a VPS), with a max part size of 1G and a max vol size of 5G. This works well for me. It would not supprize me if the Amazon S3 API has some upper limit on file size. I know that Amazon Glazier has a limit on the transfer size for large files (they must be uploaded in parts). At Tue, 29 Oct 2024 21:35:35 +0100 Justin Case <[email protected]> wrote: > > Hi there, I am struggling with the Amazon cloud driver. I have defined the Max vol. bytes as 350G and not defined Max Part Size, but due to volume size of 350G, no part will be bigger than 350G, I suppose. Whether this makes sense when transferring over the Internet is another story⦠what would be a recommended part size? > > No problems occur with small backups / part sizes (like say smaller than a 1G). > > The problem I am encountering is with a backup of part size slightly over 200G. In the cache folder the part file is called âpart.3â, but during transfer I can see on the S3 backend (minIO, self-hosted) that the filename of the part is âpart<timestamp>.3â where timestamp consists of year, month, day, time, all concatenated with out any separators. After the transfer I find the following error message in the log: > > "An error occurred (InvalidArgument) when calling the UploadPart operation: Part number must be an integer between 1 and 10000, inclusive Child exited with code 2â > > It seems that Bacula does no like âpart<timestamp>.3â and wants âpart.3â - but why - why on earth - would the file being transferred use this filename including the timestamp? Or is this a transitory file name used during transfer and being renamed after complete transfer? > > Has anyone an idea what is going wrong here? > > > > > _______________________________________________ > Bacula-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/bacula-users > > -- Robert Heller -- Cell: 413-658-7953 GV: 978-633-5364 Deepwoods Software -- Custom Software Services http://www.deepsoft.com/ -- Linux Administration Services [email protected] -- Webhosting Services _______________________________________________ Bacula-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-users