Re: Logfiles above 2G

I'm in favor putting this in default value
But we need to be sure it does not break any existing installations.
(for fs that does not support it for example)

Vincent.

--
Telemaque - 06200 NICE - (FR)
Service Technique/Reseau - NOC
Developpement SMS/MMS/Kiosques
http://www.telemaque.fr/
[email protected]
Tel : +33 4 93 97 71 64 (fax 68)
----- Original Message ----- 
From: "Peter Christensen" <[email protected]>
To: "Andreas Fink" <[email protected]>
Cc: <[email protected]>
Sent: Wednesday, September 06, 2006 11:48 AM
Subject: Re: Logfiles above 2G


> Hi,
>
> The problem is not the kernel - 32-bit linux kernels have no problem 
> handling files larger than 2GB. But if the program is not compiled with 
> support for large files, it will receive an interrupt when it attempts to 
> write past the 2GB barrier. The solution is to compile with the CFLAGS 
> "-D_LARGE_FILES -D_FILE_OFFSET_BITS=64". Actually this just makes the 
> program call open64() and lseek64() instead of open() and lseek().
>
> Compile kannel in this way, and kannel can handle enormous logs without 
> crashing.
>
> I thought that large file support was actually enabled by default now (in 
> the latest CVS) - I believe I saw a note about that some time ago.
>
> Med venlig hilsen / Best regards
>
> Peter Christensen
>
> Developer
> ------------------
> Cool Systems ApS
>
> Tel: +45 2888 1600
> Mai: [email protected]
> www: www.coolsystems.dk
>
>
> Andreas Fink wrote:
>> Vincent says:
>>>> - logrotate is your friend... :)
>>>> - or use 64 bits arch
>>
>>
>> 1. Logrotate is already my friend (it was misconfigured though...)
>> 2. 64bit architecture is not a limit for having larger files. MacOS X on 
>> intel has no problem creating such large files.
>>
>> On 06.09.2006, at 02:34, Mi Reflejo wrote:
>>
>>> The 2gb limit is in 32-bit cpu: Most exactly in libc, kernel fs
>>> drivers and VFS layer.
>>
>> This is not correct anymore. The kernel is running 32bit (but is actually 
>> a AMD Opteron with EMT64 support) but can deal with large filesystems. 
>> The filesystem also supports file larger than 2GB (people using VMWAre 
>> would cry otherwise). So the only location left is in libc.
>>
>> See this:
>> Let me create a file larger than 2G
>>
>> $ dd if=/dev/zero of=test bs=1M count=2050
>> 2050+0 records in
>> 2050+0 records out
>> 2149580800 bytes (2.1 GB) copied, 48.2032 seconds, 44.6 MB/s
>> [afink@octopus ~]$ ls -l test
>> -rw-rw-r-- 1 afink afink 2149580800 Sep  6 05:42 test
>> [afink@octopus ~]$
>>
>> So we can create files larger than 2GB on that filesystem. This was not 
>> the case in 2001 when I run into that problem too where the filesystem 
>> was really the limit. So it can not be kernel, not filesystem. So it must 
>> be a libc limitation but for appending data to a existing file, it sounds 
>> rather strange to me that fprintf has this limit. Lets test this with a 
>> small piece of C:
>>
>> #include <stdio.h>
>> #include <errno.h>
>>
>> main()
>> {
>>         FILE *f;
>>         int i;
>>
>>         f = fopen("test.bin","a");
>>
>>         if(f==NULL)
>>         {
>>                 fprintf(stderr,"could not open file\nerrno= %d\n",errno);
>>                 return -1;
>>         }
>>         fseek(f,0,SEEK_END);
>>         for(i=0;i<1000;i++)
>>         {
>>                 fprintf(f,"The world is nice\n");
>>         }
>>         fclose(f);
>> }
>>
>> This code shows that when I do a fopen on the 2.1GB file, I get an error 
>> 27 back. ( EFBIG). So it must be libc issue. However libc is used on my 
>> other platforms too and doesnt show this behaviours.
>>
>> As "dd" has no probelm, Kannel can be made not having a problem too.
>> Furthermore, Kannel should not crash in that scenario which it currently 
>> does.
>> And Kannel doesnt recover from that crash after a restart because the 
>> logfile doesnt change its size (we dont truncate it). So using the box 
>> around the box to autorestart it, wont solve it.
>>
>>
>>
>>
>>
>>
>>
>>
>
>