Re: Tyan 2885 ftp freeze

Michael Riss <[email protected]> Fri, 29 Apr 2005 12:20:12 +0200
Newsgroups gmane.linux.mandrake.cooker.amd64
Organization TU Munich
Message-ID <[email protected]>
On Thursday 28 April 2005 20:31, Michael Riß wrote:
> Gwenole Beauchesne wrote:
> > On Wed, 27 Apr 2005, Michael Riss wrote:
> >>I ran into problems myself with the 10.1-x86_64 Mandrake kernel on our
> >> 16GB Opteron machine
> >>(http://qa.mandrakesoft.com/show_bug.cgi?id=13069)
> >
> > I now have Update CDs for testing for Corporate Desktop 3.0 and 10.1.
> > Though, I don't remember having seen the problem on 10.1. Will try again
> > other configurations and tell you where to find that if you wan to test
> > too.
>
> Right now the 16GB machine runs 10.2|LE2005 x86_64 and there is no kernel
> panic at startup anymore. I still have to repeat the NFS stress test, but I
> have only little time at the moment, so it might take some time.

Fortunately it did not take too long, but unfortunately the bug is still
there. :(
Setup:
A Quad-Opteron machine with 16 GB memory and 2.6.11-6mdksmp writes out
a huge file via NFS over Gigabit-Ethernet to a fileserver.
The command is:
time bash -c "dd if=/dev/zero of=dump bs=1048576 count=102400; sync"
Everything works well up to nearly 15GB, at that point errors show up in the 
syslog (see attachment) and some processes get killed.
The process for file writing (rpciod) continues to draw CPU cycles but the
file writing progress comes to a halt or is at least very, very slow.
Strangely even on the fileserver the nfsds are still extremely active as if
they would still consume 70-80MB/sec. 
It is somehow like the NFS protokol still works - but without data.

Gwenole, did you mean the new corporate kernels address this problem
or do they solve the panic at startup problem?
Should I open a new bug for the NFS problem?

cu
Michi
messages (text/plain, 5.4 KB)
kernel: oom-killer: gfp_mask=0xd2
kernel: Node 3 DMA per-cpu: empty
kernel: Node 3 Normal per-cpu:
kernel: cpu 0 hot: low 32, high 96, batch 16
kernel: cpu 0 cold: low 0, high 32, batch 16
kernel: cpu 1 hot: low 32, high 96, batch 16
kernel: cpu 1 cold: low 0, high 32, batch 16
kernel: cpu 2 hot: low 32, high 96, batch 16
kernel: cpu 2 cold: low 0, high 32, batch 16
kernel: cpu 3 hot: low 32, high 96, batch 16
kernel: cpu 3 cold: low 0, high 32, batch 16
kernel: Node 3 HighMem per-cpu: empty
kernel: Node 2 DMA per-cpu: empty
kernel: Node 2 Normal per-cpu:
kernel: cpu 0 hot: low 32, high 96, batch 16
kernel: cpu 0 cold: low 0, high 32, batch 16
kernel: cpu 1 hot: low 32, high 96, batch 16
kernel: cpu 1 cold: low 0, high 32, batch 16
kernel: cpu 2 hot: low 32, high 96, batch 16
kernel: cpu 2 cold: low 0, high 32, batch 16
kernel: cpu 3 hot: low 32, high 96, batch 16
kernel: cpu 3 cold: low 0, high 32, batch 16
kernel: Node 2 HighMem per-cpu: empty
kernel: Node 1 DMA per-cpu: empty
kernel: Node 1 Normal per-cpu:
kernel: cpu 0 hot: low 32, high 96, batch 16
kernel: cpu 0 cold: low 0, high 32, batch 16
kernel: cpu 1 hot: low 32, high 96, batch 16
kernel: cpu 1 cold: low 0, high 32, batch 16
kernel: cpu 2 hot: low 32, high 96, batch 16
kernel: cpu 2 cold: low 0, high 32, batch 16
kernel: cpu 3 hot: low 32, high 96, batch 16
kernel: cpu 3 cold: low 0, high 32, batch 16
kernel: Node 1 HighMem per-cpu: empty
kernel: Node 0 DMA per-cpu:
kernel: cpu 0 hot: low 2, high 6, batch 1
kernel: cpu 0 cold: low 0, high 2, batch 1
kernel: cpu 1 hot: low 2, high 6, batch 1
kernel: cpu 1 cold: low 0, high 2, batch 1
kernel: cpu 2 hot: low 2, high 6, batch 1
kernel: cpu 2 cold: low 0, high 2, batch 1
kernel: cpu 3 hot: low 2, high 6, batch 1
kernel: cpu 3 cold: low 0, high 2, batch 1
kernel: Node 0 Normal per-cpu:
kernel: cpu 0 hot: low 32, high 96, batch 16
kernel: cpu 0 cold: low 0, high 32, batch 16
kernel: cpu 1 hot: low 32, high 96, batch 16
kernel: cpu 1 cold: low 0, high 32, batch 16
kernel: cpu 2 hot: low 32, high 96, batch 16
kernel: cpu 2 cold: low 0, high 32, batch 16
kernel: cpu 3 hot: low 32, high 96, batch 16
kernel: cpu 3 cold: low 0, high 32, batch 16
kernel: Node 0 HighMem per-cpu: empty
kernel: 
kernel: Free pages:       48132kB (0kB HighMem)
kernel: Active:938 inactive:3590430 dirty:0 writeback:3120578 unstable:469102 free:12033 slab:483672 mapped:1167 pagetables:578
kernel: Node 3 DMA free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
kernel: lowmem_reserve[]: 0 4095 4095
kernel: Node 3 Normal free:9152kB min:8204kB low:10252kB high:12304kB active:3224kB inactive:3617536kB present:4194300kB pages_scanned:2032451 all_unreclaimable? no
kernel: lowmem_reserve[]: 0 0 0
kernel: Node 3 HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
kernel: lowmem_reserve[]: 0 0 0
kernel: Node 2 DMA free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
kernel: lowmem_reserve[]: 0 4095 4095
kernel: Node 2 Normal free:9824kB min:8204kB low:10252kB high:12304kB active:244kB inactive:3324808kB present:4194300kB pages_scanned:17589 all_unreclaimable? no
kernel: lowmem_reserve[]: 0 0 0
kernel: Node 2 HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
kernel: lowmem_reserve[]: 0 0 0
kernel: Node 1 DMA free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
kernel: lowmem_reserve[]: 0 4095 4095
kernel: Node 1 Normal free:9296kB min:8204kB low:10252kB high:12304kB active:0kB inactive:3894060kB present:4194300kB pages_scanned:243903 all_unreclaimable? no
kernel: lowmem_reserve[]: 0 0 0
kernel: Node 1 HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
kernel: lowmem_reserve[]: 0 0 0
kernel: Node 0 DMA free:11460kB min:32kB low:40kB high:48kB active:0kB inactive:0kB present:16384kB pages_scanned:530 all_unreclaimable? yes
kernel: lowmem_reserve[]: 0 4015 4015
kernel: Node 0 Normal free:8400kB min:8044kB low:10052kB high:12064kB active:284kB inactive:3525188kB present:4112380kB pages_scanned:186879 all_unreclaimable? no
kernel: lowmem_reserve[]: 0 0 0
kernel: Node 0 HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
kernel: lowmem_reserve[]: 0 0 0
kernel: Node 3 DMA: empty
kernel: Node 3 Normal: 176*4kB 48*8kB 30*16kB 15*32kB 17*64kB 1*128kB 11*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 9152kB
kernel: Node 3 HighMem: empty
kernel: Node 2 DMA: empty
kernel: Node 2 Normal: 32*4kB 36*8kB 20*16kB 24*32kB 16*64kB 13*128kB 8*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 9824kB
kernel: Node 2 HighMem: empty
kernel: Node 1 DMA: empty
kernel: Node 1 Normal: 18*4kB 31*8kB 17*16kB 14*32kB 11*64kB 21*128kB 3*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 9296kB
kernel: Node 1 HighMem: empty
kernel: Node 0 DMA: 1*4kB 4*8kB 4*16kB 5*32kB 5*64kB 3*128kB 3*256kB 1*512kB 1*1024kB 2*2048kB 1*4096kB = 11460kB
kernel: Node 0 Normal: 40*4kB 82*8kB 6*16kB 2*32kB 20*64kB 2*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 1*4096kB = 8400kB
kernel: Node 0 HighMem: empty
kernel: Swap cache: add 9641, delete 9348, find 152/231, race 0+0
kernel: Free swap  = 33507392kB
kernel: Total swap = 33543640kB
kernel: Out of Memory: Killed process 22647 (bash).