Re: copy job performance problem

Hi Chris,

Am 14.11.2025 um 11:42 schrieb Chris Wright:
<snip detailed description>
> all seems well for a time then throughput drops to essentially zero

The time things go well -- can you more or less reproduce that, or can 
you even identify time or number of bytes transferred beyond which 
things slow down?

>   - SD1 will have a single CPU pegged at 100%, with minimal IO traffic 
> (both ops and bandwidth) from the open volume file, we will get spikes 
> of good speed but average throughput after leaving a job running for a 
> week is <1 MiB/sec.
>   - SD2 is quiet, happily handling normal backup jobs from other clients 
> with normal performance
> 
> If we start a second, parallel, copy job we get similar initially good 
> throughput then peg a second CPU on SD1 to 100% but there isn't exactly 
> a big jump in performance.

You could try to identify potential problem points by experimenting with 
different job sizes, different directions, and different storage 
targets. I like using FIFO storage backed by /dev/null plus a pool where 
I disable cataloging of files. Just make sure you never send actual 
backups there, or migration jobs...

> There are no warnings/errors being logged and everything appears to be 
> "working", just glacially slow and apparently totally bottlenecked on 
> whatever that single CPU thread is doing with minimal reads from the 
> volumes.
> 
> Any suggestions on where to look for the root cause here?

Not quite a root cause, but I'd start with tracing the SD activities 
and/or system calls. strace with time stamps in my experience can help a 
lot in identifying underlying issues, but will probably create rather 
unwieldy output.

So just a few suggestions how to investigate, nothing I can see that 
would quickly fix the problem.

But let us know what you do and find!

Cheers,

Arno

> Thanks
> -- 
> 
> Chris Wright
> 
> Application Software Developer
> 
> <http://www.maglabs.net>
> 
> 
> T:  0203 515 1000 | www.maglabs.net <http://www.maglabs.net> | Follow us 
> <https://bit.ly/3x215vn>
> 
> MagLabs Limited is a Limited Liability Company registered at Companies 
> House, Cardiff. Registration No 06715580.
> DISCLAIMER: This email and any attachments sent with it may contain 
> confidential and legally privileged information. It is intended solely 
> for the individual or entity to whom the email was addressed. If you are 
> not the intended recipient please notify the sender via email 
> immediately, delete the email (and attachments) from your computer 
> system and destroy any copies you may have in your possession. You are 
> prohibited from using, printing, copying or disclosing any of the 
> information contained within the email and its attachment(s). MagLabs 
> Limited does not accept any responsibility or liability for any changes 
> made to this email after it was sent or for any viruses transmitted 
> through it. Opinions, comments, and conclusions made in this email may 
> be that of the author and may not reflect the view of MagLabs Limited.
> 
> Please consider the environment before printing this email
> 
> 
> 
> _______________________________________________
> Bacula-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/bacula-users

-- 
Arno Lehmann

IT-Service Lehmann
Sandstr. 6, 49080 Osnabrück



_______________________________________________
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users