Re: 6.1 crashes
Daniel-Constantin Mierla via sr-users <[email protected]> Sun, 19 Apr 2026 18:28:29 +0200
| Newsgroups | gmane.comp.voip.ser |
|---|---|
| Message-ID | <[email protected]> |
Earlier today I backported a commit to dns cache that was supposed to
fix some types of queries. Was your build done today?
If yes, can you rebuild with latest 6.1 branch, as I just reverted that
commit?
Daniel
On 19.04.26 18:04, Juha Heinanen wrote:
> Daniel-Constantin Mierla writes:
>
>> You have to install the package with the debugging symbols for getting
>> proper details in the backtrace.
> I re-built Kamailio and installed also kamailio-dbgsym package. I'll
> wait if another crash happens. Below is another old core dump from
> today.
>
> -- Juha
>
> Core was generated by `/usr/sbin/kamailio -f /etc/sip-proxy/sip-proxy.cfg -P /run/sip-proxy/sip-proxy.pid -m 128 -M 16 -u root -g root -w /run/sip-proxy -Y /run/sip-proxy -n 8'.
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0 0x0000558b1c9ea97e in _dns_hash_remove_entry (e=0x7f7f75085ba0, line=662, fpath=0x558b1ccda66a "core/dns_cache.c")
> at core/dns_cache.c:501
>
> warning: 501 core/dns_cache.c: No such file or directory
> (gdb) bt full
> #0 0x0000558b1c9ea97e in _dns_hash_remove_entry (e=0x7f7f75085ba0, line=662, fpath=0x558b1ccda66a "core/dns_cache.c")
> at core/dns_cache.c:501
> __func__ = "_dns_hash_remove_entry"
> __llevel = <optimized out>
> __kld = <optimized out>
> __llevel = <optimized out>
> __kld = <optimized out>
> __llevel = <optimized out>
> __kld = <optimized out>
> #1 0x0000558b1c9f0e02 in dns_cache_clean (no=4294967295, expired_only=1) at core/dns_cache.c:662
> e = 0x7f7f75085ba0
> now = 658262396
> n = <optimized out>
> deleted = <optimized out>
> l = 0x7f7f75085bb0
> tmp = 0x0
> __func__ = "dns_cache_clean"
> #2 dns_timer (ticks=<optimized out>, tl=<optimized out>, data=<optimized out>) at core/dns_cache.c:223
> No locals.
> #3 dns_timer (ticks=<optimized out>, tl=<optimized out>, data=<optimized out>) at core/dns_cache.c:210
> No locals.
> #4 0x0000558b1cc6f61a in slow_timer_main () at core/timer.c:1200
> n = <optimized out>
> ret = <optimized out>
> tl = 0x7f7f74408120
> i = <optimized out>
> __func__ = "slow_timer_main"
> #5 0x0000558b1c939455 in main_loop () at ./obj-x86_64-linux-gnu/src/main.c:2007
> i = <optimized out>
> pid = <optimized out>
> si = <optimized out>
> sx = <optimized out>
> si_desc = "udp receiver child=7 sock=[2001:67c:224:666::38]:8002\000\000\000\022\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000\2007t\177\177\000\000\002\000\000\000\004\000\000\000\177\000\000\001\213U\000\000}\000\000\000\000\000\000\000\000\277Țs\373,&Sun Apr \t\000\000\000\000\000\000"
> nrprocs = <optimized out>
> woneinit = 1
> --Type <RET> for more, q to quit, c to continue without paging--
> agfound = <optimized out>
> __func__ = "main_loop"
> error = <optimized out>
> #6 0x0000558b1c92ae91 in main (argc=<optimized out>, argv=<optimized out>) at ./obj-x86_64-linux-gnu/src/main.c:3434
> cfg_stream = <optimized out>
> c = <optimized out>
> r = <optimized out>
> tmp = 0x7fffa1dede80 ""
> tmp_len = 0
> port = 5060
> proto = 0
> aproto = 0
> ahost = 0x0
> socket_name = <optimized out>
> aport = 0
> listen_field_count = <optimized out>
> listen_fields = {0x2 <error: Cannot access memory at address 0x2>, 0x7fffa1debb58 "",
> 0x1900000 <error: Cannot access memory at address 0x1900000>}
> options = 0x558b1ccede20 ":f:cm:M:dVIhEeb:B:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
> ret = -1
> debug_save = <optimized out>
> debug_flag = <optimized out>
> dont_fork_cnt = 0
> n_lst = <optimized out>
> p = <optimized out>
> tbuf = 0x7f7f7f1d21fe <handle_intel+142> "H\205\300u\272L\211\341H\211\352D\211\356D\211\377\350\354\374\377\377H\205\300u\244L\211\341H\211\352D\211\366D\211\377\350\326\374\377\377H\205\300u\216A\201", <incomplete sequence \357\277>
> tbuf_tmp = <optimized out>
> st = {st_dev = 24, st_ino = 35403, st_nlink = 2, st_mode = 16877, st_uid = 117, st_gid = 124, __pad0 = 0,
> st_rdev = 0, st_size = 60, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1776605683,
> tv_nsec = 661187506}, st_mtim = {tv_sec = 1776604769, tv_nsec = 912289336}, st_ctim = {
> tv_sec = 1776605719, tv_nsec = 209844336}, __glibc_reserved = {0, 0, 0}}
> l1 = <optimized out>
> lim = {rlim_cur = 262929407, rlim_max = 100}
> option_index = 0
> long_options = {{name = 0x558b1ccd72d8 "help", has_arg = 0, flag = 0x0, val = 104}, {
> --Type <RET> for more, q to quit, c to continue without paging--
> name = 0x558b1ccd9e5d "version", has_arg = 0, flag = 0x0, val = 118}, {name = 0x558b1ccdebf2 "alias",
> has_arg = 1, flag = 0x0, val = 1024}, {name = 0x558b1ccd72dd "domain", has_arg = 1, flag = 0x0,
> val = 1024}, {name = 0x558b1ccd72e4 "subst", has_arg = 1, flag = 0x0, val = 1025}, {
> name = 0x558b1ccd72ea "substdef", has_arg = 1, flag = 0x0, val = 1026}, {
> name = 0x558b1ccd72f3 "substdefs", has_arg = 1, flag = 0x0, val = 1027}, {
> name = 0x558b1ccd72fd "server-id", has_arg = 1, flag = 0x0, val = 1028}, {
> name = 0x558b1ccd7307 "loadmodule", has_arg = 1, flag = 0x0, val = 1029}, {
> name = 0x558b1ccd7312 "modparam", has_arg = 1, flag = 0x0, val = 1030}, {
> name = 0x558b1ccd731b "log-engine", has_arg = 1, flag = 0x0, val = 1031}, {name = 0x558b1ccd9f7a "debug",
> has_arg = 1, flag = 0x0, val = 1032}, {name = 0x558b1ccd7326 "cfg-print", has_arg = 0, flag = 0x0,
> val = 1033}, {name = 0x558b1ccd7330 "atexit", has_arg = 1, flag = 0x0, val = 1034}, {
> name = 0x558b1ccd7337 "all-errors", has_arg = 0, flag = 0x0, val = 1035}, {name = 0x558b1ccd7342 "iuid",
> has_arg = 1, flag = 0x0, val = 1036}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}}
> __func__ = "main"
>
>> However, as it looks now, the corefile is written at Kamailio shutdown,
>> either as an effect to another crash which has the core file overwritten
>> if you didn't enabled core files per pid, or during a stop/restart
>> command. To be sure the core files are not overwritten, enable core
>> files per pid or define core file name pattern to be individual per
>> running process. Kamailio wiki has an article about troubleshooting with
>> corefile where more details are provided on these options.
>>
>> Cheers,
>> Daniel
>>
>>
>> On 19.04.26 16:27, Juha Heinanen via sr-users wrote:
>>> After updating my sip proxy to latest kamailio 6.1 branch running on
>>> Debian Trixie , I have noticed several crashes.
>>>
>>> syslog has this:
>>>
>>> 2026-04-19T16:41:19.664532+03:00 box kernel: kamailio[944355]: segfault at 0 ip 0000558b1c9ea97e sp 00007fffa1deb610 er\
>>> ror 6 in kamailio[f297e,558b1c926000+3b0000] likely on CPU 2 (core 0, socket 4)
>>> 2026-04-19T16:41:19.664633+03:00 box kernel: Code: 00 00 00 48 8b 57 08 48 8b 07 f3 0f 7e 0d 8a 72 3f 00 48 8b 4f 10 64\
>>> 4c 8b 24 25 28 00 00 00 4c 89 a4 24 88 00 00 00 41 89 f4 <48> 89 02 66 0f 6c ca 48 89 50 08 0f 11 07 48 83 c3 10 74 13\
>>> 48 39
>>> 2026-04-19T16:41:19.936157+03:00 box /usr/sbin/kamailio[944455]: CRITICAL: <core> [core/pass_fd.c:284]: receive_fd(): E\
>>> OF on 34
>>> 2026-04-19T16:41:20.435008+03:00 box /usr/sbin/kamailio[944292]: ALERT: <core> [main.c:824]: handle_sigs(): child proce\
>>> ss 944355 exited by a signal 11
>>> 2026-04-19T16:41:20.435143+03:00 box /usr/sbin/kamailio[944292]: ALERT: <core> [main.c:828]: handle_sigs(): core was ge\
>>> nerated
>>> 2026-04-19T16:41:20.435191+03:00 box /usr/sbin/kamailio[944292]: INFO: <core> [main.c:851]: handle_sigs(): terminating \
>>> due to SIGCHLD
>>> 2026-04-19T16:41:20.435613+03:00 box /usr/sbin/kamailio[944405]: INFO: <core> [main.c:907]: sig_usr(): signal 15 receiv\
>>> ed
>>> ...
>>>
>>> and core dump shows this:
>>>
>>> #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0)
>>> at ./nptl/pthread_kill.c:44
>>>
>>> warning: 44 ./nptl/pthread_kill.c: No such file or directory
>>> (gdb) where
>>> #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0)
>>> at ./nptl/pthread_kill.c:44
>>> #1 0x00007f7f7ef509ff in __pthread_kill_internal (threadid=<optimized out>, signo=6)
>>> at ./nptl/pthread_kill.c:89
>>> #2 0x00007f7f7eefbcc2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
>>> #3 0x00007f7f7eee44ac in __GI_abort () at ./stdlib/abort.c:77
>>> #4 0x0000558b1c926d45 in ?? ()
>>> #5 <signal handler called>
>>> #6 0x00007f7f7efb4907 in __GI_sched_yield () at ../sysdeps/unix/syscall-template.S:120
>>> #7 0x0000558b1cc6c7ed in timer_del_safe ()
>>> #8 0x0000558b1c9f8899 in destroy_dns_cache ()
>>> #9 0x0000558b1c932537 in cleanup ()
>>> #10 0x0000558b1c9332ed in ?? ()
>>> #11 0x0000558b1c933fd3 in handle_sigs ()
>>> #12 0x0000558b1c9395fd in main_loop ()
>>> #13 0x0000558b1c92ae91 in main ()
>>>
>>> Any ideas?
>>>
>>> -- Juha
>>>
>>>
>>> __________________________________________________________
>>> Kamailio - Users Mailing List - Non Commercial Discussions -- [email protected]
>>> To unsubscribe send an email to [email protected]
>>> Important: keep the mailing list in the recipients, do not reply only to the sender!
>>
>> --
>> Daniel-Constantin Mierla (@ asipto.com)
>> twitter.com/miconda -- linkedin.com/in/miconda
>> Kamailio Consultancy, Training and Development Services -- asipto.com
>> Kamailio World Conference, May 7-8, 2026 - Berlin, Germany -- kamailioworld.com
--
Daniel-Constantin Mierla (@ asipto.com)
twitter.com/miconda -- linkedin.com/in/miconda
Kamailio Consultancy, Training and Development Services -- asipto.com
Kamailio World Conference, May 7-8, 2026 - Berlin, Germany -- kamailioworld.com
__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions -- [email protected]
To unsubscribe send an email to [email protected]
Important: keep the mailing list in the recipients, do not reply only to the sender!