"Out of Memory: Killed process" errors on server running Oracle
Gene <[email protected]> Thu, 12 Jul 2007 14:17:10 -0700
| Newsgroups | gmane.linux.redhat.release.taroon.general |
|---|---|
| Message-ID | <[email protected]> |
Hola, Every 5-6 days I'm getting "Out of Memory: Killed process" messages on a Redhat AS 3 server running Oracle 9.20. These errors will occur for a couple hours and will eventually lead to Oracle shutting down. The load on the server did increase recently, but there's still 1.8GB memory free on the system during these errors. The box is running the smp kernel. Should I do something like increase the stack size or use the hugemem kernel? $ uname -a Linux prddb02 2.4.21-15.0.3.ELsmp #1 SMP Tue Jun 29 18:04:47 EDT 2004 i686 i686 i386 GNU/Linux $ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) 4 max memory size (kbytes, -m) unlimited open files (-n) 65536 pipe size (512 bytes, -p) 8 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 16384 virtual memory (kbytes, -v) unlimited >From /var/log/messages: Jul 11 19:06:31 prddb02 kernel: Mem-info: Jul 11 19:06:31 prddb02 kernel: Zone:DMA freepages: 2549 min: 0 low: 0 high: 0 Jul 11 19:06:31 prddb02 kernel: Zone:Normal freepages: 1254 min: 1279 low: 4544 high: 6304 Jul 11 19:06:31 prddb02 kernel: Zone:HighMem freepages:501745 min: 255 low: 20992 high: 31488 Jul 11 19:06:31 prddb02 kernel: Free pages: 505548 (501745 HighMem) Jul 11 19:06:31 prddb02 kernel: aa:0 ac:0 id:0 il:0 ic:0 fr:2549 Jul 11 19:06:31 prddb02 kernel: aa:0 ac:40 id:82 il:73 ic:0 fr:1254 Jul 11 19:06:31 prddb02 kernel: aa:530276 ac:82904 id:17203 il:6249 ic:8419 fr:501745 Jul 11 19:06:31 prddb02 kernel: 1*4kB 2*8kB 2*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 2*4096kB = 10196kB) Jul 11 19:06:31 prddb02 kernel: 137*4kB 121*8kB 45*16kB 19*32kB 6*64kB 2*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 5020kB) Jul 11 19:06:31 prddb02 kernel: 35001*4kB 11806*8kB 4209*16kB 1479*32kB 1490*64kB 513*128kB 291*256kB 48*512kB 15*1024kB 5*2048kB 335*4096kB = 2006980kB) Jul 11 19:06:31 prddb02 kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0 Jul 11 19:06:31 prddb02 kernel: 192549 pages of slabcache Jul 11 19:06:31 prddb02 kernel: 1192 pages of kernel stacks Jul 11 19:06:31 prddb02 kernel: 0 lowmem pagetables, 127372 highmem pagetables Jul 11 19:06:31 prddb02 kernel: Free swap: 8192980kB Jul 11 19:06:31 prddb02 kernel: 1572864 pages of RAM Jul 11 19:06:31 prddb02 kernel: 1277920 pages of HIGHMEM Jul 11 19:06:31 prddb02 kernel: 89860 reserved pages Jul 11 19:06:31 prddb02 kernel: 24863915 pages shared Jul 11 19:06:31 prddb02 kernel: 0 pages swap cached Jul 11 19:06:31 prddb02 kernel: Out of Memory: Killed process 8302 (oracle). $ sar -r -f /var/log/sa/sa11 10:00:00 AM kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree kbswpused %swpused kbswpcad 05:00:00 PM 1901836 4030180 67.94 4228 1976412 8192980 0 0.00 0 05:10:00 PM 1755672 4176344 70.40 1456 2115832 8192980 0 0.00 0 05:20:00 PM 1901120 4030896 67.95 2952 1976168 8192980 0 0.00 0 05:30:00 PM 1896664 4035352 68.03 2392 1975920 8192980 0 0.00 0 05:39:59 PM 1892232 4039784 68.10 2200 1975900 8192980 0 0.00 0 05:50:00 PM 1889808 4042208 68.14 2408 1976076 8192980 0 0.00 0 06:00:00 PM 1897984 4034032 68.00 2336 1976244 8192980 0 0.00 0 06:10:00 PM 1831708 4100308 69.12 1192 2040788 8192980 0 0.00 0 06:20:01 PM 1832808 4099208 69.10 1372 2040912 8192980 0 0.00 0 06:30:03 PM 1844548 4087468 68.91 1284 2040988 8192980 0 0.00 0 06:40:01 PM 1895092 4036924 68.05 1304 2041120 8192980 0 0.00 0 06:50:00 PM 1909852 4022164 67.80 1300 2041248 8192980 0 0.00 0 07:00:01 PM 1906980 4025036 67.85 1104 2041300 8192980 0 0.00 0 07:10:01 PM 4363412 1568604 26.44 13816 726432 8192980 0 0.00 0 -Gene [email protected] -- Taroon-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/taroon-list