Re: smartctl kills performance

I've tested with short selftest, just in case - the same problem arises.

sda - with test has been started, sdb,c,d - not, the all disks are in
the same raid10 array:

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.88    0.00    0.25   86.38    0.00   12.50

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s
avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sdb               2.00     0.00   21.00    6.00   292.00    20.50
23.15     0.34   12.59   13.14   10.67   6.81  18.40
sdd               0.00     0.00    7.00    6.00    44.00    20.50
9.92     0.12    9.54    8.57   10.67   5.85   7.60
sdc               1.00     0.00   14.00    6.00   352.00    20.50
37.25     0.16    8.00    6.86   10.67   4.60   9.20
sda               1.00     0.00    7.00    4.00    72.00   668.50
134.64    28.30 2695.27 2090.86 3753.00  90.91 100.00

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           4.64    0.00    0.88   82.96    0.00   11.53

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s
avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sdb               0.00     3.00   16.00   53.00    84.00   620.00
20.41     1.59   10.38   20.25    7.40   3.65  25.20
sdd               3.00     1.00    9.00   57.00   348.00   612.00
29.09     0.64    9.45   16.44    8.35   2.30  15.20
sdc               0.00     1.00   17.00   57.00   172.00   612.00
21.19     1.34   16.22   12.94   17.19   3.14  23.20
sda               4.00     3.00   12.00    7.00   392.00    68.00
48.42    40.50 1772.84 2275.67  910.86  52.63 100.00




On Sat, Feb 6, 2016 at 10:20 PM, CoolCold <[email protected]> wrote:
> Hello!
> Can't reply directly into the thread like with Google Groups, but
> anyway. I'm writing in reply to
> http://sourceforge.net/p/smartmontools/mailman/message/29431706/ about
> offline test kills the performace.
>
> On the new server on Hetzner, strange performance degradation was
> detected. During several days, degrade started at 4:15 AM, reboot
> helped.
>
> The server is equipped with disks:
> root@phi1:~# for i in  /dev/sd?;do smartctl -i $i;done|grep Model
> Device Model:     HGST HUS726060ALE610
> Device Model:     HGST HUS726060ALE610
> Device Model:     HGST HUS726060ALE610
> Device Model:     HGST HUS726060ALE610
>
> As revealed, when regular offline scan fires up (at 4:15 AM), disk
> performance is slow as turtle.
>
> Feb  6 04:15:01 phi1 smartd[2232]: Device: /dev/sdb [SAT], starting
> scheduled Long Self-Test.
> Feb  6 04:15:01 phi1 smartd[2232]: Device: /dev/sdc [SAT], starting
> scheduled Long Self-Test.
>
> I can see large awaits in iostat, collectd showed disk io time up to
> 75 seconds! ( http://gyazo.coolcold.org/upload/fe129abcc401bd61496273c32d4c.png
> )
>
> Stopping offline test with "-x" restores the performance.
>
> May be this going to be useful to someone else.
>
>
> P.S. I'm off the list, so if you want to reply, add me to CC/TO fields.
> --
> Best regards,
> [COOLCOLD-RIPN]



-- 
Best regards,
[COOLCOLD-RIPN]

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140