Re: benchmarking effective_io_concurrency

Hi Rick, 

thanks for your inputs.

On 22/07/2019 14:06, Rick Otten wrote:
> 
> 
> 
> You didn't mention what type of disk storage you are using, or if that matters. 

I actually mentioned I m using SSD, in RAID 10. Also is mentioned I tested in a no-RAID setup. Is that what you mean?

 The number of cores in your database could also matter.
> 

True, when scaling I think it can actually bring up problems as you mention here below. (BTW, Tested on a VM with 6 cores and on HW with 32. I updated the blogpost, thanks)


> Does the max_parallel_workers setting have any influence on how effective_io_concurrency works?
> 

I m not sure about that one related to the tests I ran, because the query plan does not show parallelism. 

> Based on your data, one should set effective_io_concurrency at the highest possible setting with no ill effects with the possible exception that your disk will get busier.  Somehow I suspect that as you scale the number of concurrent disk i/o tasks, other things may start to suffer.  For example does CPU wait time start to increase as more and more threads are consumed waiting for i/o instead of doing other processing?  Do you run into lock contention on the i/o subsystem?  (Back in the day, lock contention for /dev/tcp was a major bottleneck for scaling busy webservers vertically.  I have no idea if modern linux kernels could run into the same issue waiting for locks for /dev/sd0.  Surely if anything was going to push that issue, it would be setting effective_io_concurrency really
  high and then demanding a lot of concurrent disk accesses.)
> 
> 
>  

My suggestion would be to try by your own and find out what works for you, maybe slowly increasing the value of effective_io_concurrency. 

Every workload is peculiar, so I suspect there is no silver bullet here. Also the documentation gives you directions in that way...



regards,

fabio pardi