Re: Segfault while transfering

Moreover I tried with a smaller subset of these files and still got a 
segfault at 1G transfered over 7,5

Le jeudi 23 mai 2024 à 14:56:54 UTC+2, Julien David a écrit :

> Hello
>
> OS is Debian 12 with 32G RAM
> There are around 250k files to transfer
> It goes a bit more further after upgrade to 2.53.5 and changed settings 
> but still crash around 31% (14G over 47)
> htop do not show any memory pic
> Le jeudi 23 mai 2024 à 14:31:44 UTC+2, Greg Troxel a écrit :
>
>> Julien David <[email protected]> writes: 
>>
>> > I setup a new replica on quite large directory (~50G) and got a 
>> segfault on 
>> > the first occurence. 
>> > 
>> > Here is the log: 
>> > 
>> > 2024-05-23T11:43:16.592072+00:00 web1-1 kernel: [27651508.074760] 
>> > unison[187262]: segfault at 7fff04c04fc8 ip 00000000005182c4 sp 
>> > 00007fff04c04fb0 error 6 in unison[400000+183000] likely on CPU 6 (core 
>> 0, 
>> > socket 0) 
>> > 2024-05-23T11:43:16.621810+00:00 web1-1 kernel: [27651508.100149] Code: 
>> 01 
>> > 00 00 00 48 81 ec 28 08 00 00 64 48 8b 04 25 28 00 00 00 48 89 84 24 18 
>> 08 
>> > 00 00 31 c0 48 81 fe 01 01 00 00 b8 00 01 00 00 <48> 89 4c 24 18 4c 0f 
>> 43 
>> > e8 48 d1 ff 48 d1 ea 48 89 fd 49 89 d7 45 
>>
>> We are very aware of "unison is ungraceful when running out of memory" 
>> but that has generally not manifested as segfaults. 
>>
>> In theory ocaml code should not do this, absent a bug in the ocaml 
>> compiler. (While I don't think it's very likely, you could also have a 
>> hardware issue.) 
>>
>> But, there is C code in unsion. 
>>
>> My belief, not super well founded, is that unison maintains per-file 
>> data structures, but that the total size of data is not so relevant. So 
>> the question is perhaps how many files you are syncing, vs total size. 
>> But perhaps something else is going on. 
>>
>> > root@itc-rbx-web1-1:/var/log/unison# unison -version 
>> > unison version 2.53.4 (ocaml 4.14.1) 
>>
>> Actually 2.53.5 is out, since May 6. I am not claiming it will be 
>> different, but it would be great if you could upgrade and debug with 
>> that. 
>>
>> > Transfer is over ssh with the following options: 
>> > 
>> > times = true 
>> > ignore = Path .ssh 
>> > ignore = Path .unison 
>> > log = true 
>> > logfile = /var/log/unison/source.log 
>> > maxthreads = 10 
>>
>> You didn't mention OS, how much RAM you have, and what your data and 
>> stack sizes are. I would suggest two paths: 
>>
>> see if you can find the edges of the bad behavior, by 
>> - setting times=false 
>> - setting maxthreads=1 
>> - watching with ps in a fairly tight loop to see the mem/rss of the 
>> process before it crashes 
>> - turn up data segment size (see ulimit(1)) and stack 
>> - sync less 
>>
>> run under gdb and see if you can get a stack trace. You might have to 
>> build it yourself and enable some kind of debugging symbols 
>>
>

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].