Re: ls: can't open '.': Cannot allocate memory
Greg Ungerer <[email protected]>
| Newsgroups | gmane.linux.uclinux.devel |
|---|---|
| Message-ID | <[email protected]> |
Hi Larry, On 31/10/12 03:39, Larry Baker wrote: >> On 10/30/2012 08:38 PM, Luis Alves wrote: >>>> What you are seeing though is a classic case of memory fragmentation. >>> >>> Thats true. If I spam a lot of 'ls' with other commands between, it >>> eventually works. (I guess at some point a memory chunk of 1024k >>> becomes available). >>> I've tried to rebuild the kernel using as memory allocator SLOB and >>> SLUB, but the result seems to be the same as using SLAB. >>> > > I've tried those solutions too -- no good. I decided that was because those are kernel memory allocation policies, and have no effect on user memory allocation. Is that correct? > >>>> I would dig into the busybox ls code. Wanting to allocate a block of >>>> 512k in a single chunk seems a bit crazy. >>> >>> I've tried with standalone 'ls' and the result is the same. >>> It also fails wanting to allocate exacly 528384 (always the same value >>> and only when ls'ing NFS mounts). >>> (the mem alloc value is actually pretty rounded 512k + 4k) >>> >>> Could it be related to NFS kernel code? >> >> It could be. From the stack back trace: >> >> [<008446e8>] 0x8446e8 (inside <warn_alloc_failed>) >> [<0084683a>] 0x84683a (inside <__alloc_pages_nodemask>) >> [<00851e34>] 0x851e34 (inside <do_mmap_pgoff>) >> [<008526c2>] 0x8526c2 <sys_old_mmap> >> [<008526c2>] 0x8526c2 <sys_old_mmap> >> [<0084e200>] 0x84e200 (inside <vm_mmap_pgoff>) >> [<0085270a>] 0x85270a (inside <sys_old_mmap>) >> >> That looks like an mmap system call. And mmap is the underlying >> allocator used by malloc() in uClibc (for non-MMU). So to me it >> looks like a malloc somewhere in busybox. That is just a first >> guess, it needs some further investigation. >> >> >>> Don't know if it's relevant, but I'm using latest kernel sources from >>> your git, gcc-4.7.2, running kernel from flash (XIP) and user binaries >>> compiled with -msep-data (running from flash also). >> >> I wouldn't suspect anything different. But if you can try older kernel >> versions you could confirm that. >> >> >>> By the way, like Larry pointed out, is there a way to compile user >>> binaries with shared flat libraries? >>> Long time ago I've tried to enable UCLIBC_FORMAT_SHARED_FLAT in uClibc >>> config, but then it was failing building (don't remember where/why it >>> fails). >> >> I haven't used shared libraries in these setups for years. So I am >> not sure what state they are in. Seems like it might have bit rotted >> from your experiences. >> > > Rats. > > It seems to me the root of the problem on memory constrained systems is the power-of-2 memory allocator; a boxcar memory allocator would work better, I think. I have read there used to be a non-power-of-2 memory allocator, named page_alloc2. Is there any chance it can be revived for 2.6 kernels? (See my post of 23 October 2012.) I am sure it could be. But it would take a little effort though. I think Phil Wilshire made an effort a few years back to bring the page_alloc2 code up to date with 2.6 kernels. Google and check the archives you may be able to find it. Regards Greg >>> On Tue, Oct 30, 2012 at 12:33 AM, Greg Ungerer <[email protected] <mailto:[email protected]>> wrote: >>>> Hi Luis, >>>> >>>> >>>> On 30/10/12 05:38, Luis Alves wrote: >>>>> >>>>> Thanks for the explanation Larry. >>>>> >>>>> Meanwhile I've read some stuff about non-MMU memory alocation (I had >>>>> the wrong idea of how memory was allocated). >>>>> I guess my system ram is not that much (only 8Mb)... >>>> >>>> >>>> What you are seeing though is a classic case of memory fragmentation. >>>> You have eneough free RAM in total, but not enough in s ingle >>>> contiguous block. On a paged MMU system this is not a problem, pages >>>> can be mapped so you see a contiguous block the size you want. You >>>> can't do that without no MMU and paging. >>>> >>>> >>>> >>>>> How would shared libs solve the problem? >>>>> The problem is allocating memory for 'ls' data and not 'ls' itself. Is >>>>> this correct? >>>> >>>> >>>> I would dig into the busybox ls code. Wanting to allocate a block of >>>> 512k in a single chunk seems a bit crazy. >>>> >>>> Regards >>>> Greg >>>> >>>> >>>> >>>>> On Mon, Oct 29, 2012 at 5:49 PM, Larry Baker <[email protected] <mailto:[email protected]>> wrote: >>>>>> >>>>>> Luis, >>>>>> >>>>>> On 29 Oct 2012, at 8:32 AM, Luis Alves wrote: >>>>>> >>>>>> DMA: 36*4kB 30*8kB 29*16kB 21*32kB 6*64kB 7*128kB 4*256kB 3*512kB >>>>>> 0*1024kB 0*2048kB 0*4096kB = 5360kB >>>>>> >>>>>> >>>>>> The largest block of memory your system has available is 524288 >>>>>> (512*1024). >>>>>> >>>>>> Allocation of length 528384 from process 94 (ls) failed >>>>>> >>>>>> >>>>>> The requested size is larger than that, so uClinux tries to allocate the >>>>>> next larger block size (1024K). There are none available. >>>>>> >>>>>> Any sugestion? >>>>>> >>>>>> >>>>>> I do not know what determines how much memory ls allocates. >>>>>> >>>>>> I have similar memory allocation failures on my system. It does not >>>>>> appear >>>>>> to use a shared uclibc. I do not know (yet) how to enable building a >>>>>> shared >>>>>> uclibc or how to change the builds to use it. If the instructions are >>>>>> straightforward, I would like to hear from someone. >>>>>> >>>>>> Regards, >>>>>> Luis >>>>>> _______________________________________________ >>>>>> uClinux-dev mailing list >>>>>> [email protected] <mailto:[email protected]> >>>>>> http://mailman.uclinux.org/mailman/listinfo/uclinux-dev >>>>>> This message was resent by [email protected] <mailto:[email protected]> >>>>>> To unsubscribe see: >>>>>> http://mailman.uclinux.org/mailman/options/uclinux-dev >>>>>> >>>>>> >>>>>> Larry Baker >>>>>> US Geological Survey >>>>>> 650-329-5608 >>>>>> [email protected] <mailto:[email protected]> >>>>> >>>>> _______________________________________________ >>>>> uClinux-dev mailing list >>>>> [email protected] <mailto:[email protected]> >>>>> http://mailman.uclinux.org/mailman/listinfo/uclinux-dev >>>>> This message was resent by [email protected] <mailto:[email protected]> >>>>> To unsubscribe see: >>>>> http://mailman.uclinux.org/mailman/options/uclinux-dev >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> ------------------------------------------------------------------------ >>>> Greg Ungerer -- Principal Engineer EMAIL: [email protected] <mailto:[email protected]> >>>> SnapGear Group, McAfee PHONE: +61 7 3435 2888 >>>> 8 Gardner Close FAX: +61 7 3217 5323 >>>> Milton, QLD, 4064, Australia WEB: http://www.SnapGear.com >>> >>> >>> >> >> >> -- >> ------------------------------------------------------------------------ >> Greg Ungerer -- Principal Engineer EMAIL: [email protected] <mailto:[email protected]> >> SnapGear Group, McAfee PHONE: +61 7 3435 2888 >> 8 Gardner Close, FAX: +61 7 3891 3630 >> Milton, QLD, 4064, Australia WEB: http://www.SnapGear.com > > Larry Baker > US Geological Survey > 650-329-5608 > [email protected] <mailto:[email protected]> > > > -- ------------------------------------------------------------------------ Greg Ungerer -- Principal Engineer EMAIL: [email protected] SnapGear Group, McAfee PHONE: +61 7 3435 2888 8 Gardner Close FAX: +61 7 3217 5323 Milton, QLD, 4064, Australia WEB: http://www.SnapGear.com _______________________________________________ uClinux-dev mailing list [email protected] http://mailman.uclinux.org/mailman/listinfo/uclinux-dev This message was resent by [email protected] To unsubscribe see: http://mailman.uclinux.org/mailman/options/uclinux-dev