Re: Conflict between TLS module and HTTP_CLIENT causing kamailio 5.8.8 to crash

Hello,

I would probably just try Kamailio 6.1.3 (from last week) instead of trying to backport the mentioned PR to the 5.8 branch. There were AFAIK also other changes done related to the TLS module in this release, so it might be not that trivial to backport in the end.

Your jump from 4.7 to a more modern release is large enough anyway, so it does not make a big difference to not going to the most recent version. 😉

Cheers,

Henning

From: Emmanuel BUU via sr-users <[email protected]>
Sent: Montag, 1. Juni 2026 13:58
To: [email protected]
Cc: Emmanuel BUU <[email protected]>
Subject: [SR-Users] Conflict between TLS module and HTTP_CLIENT causing kamailio 5.8.8 to crash


Hi everybody,

We are in the process of migrating from kamailio 4.7 to 5.8. We are hitting a reproducible SIGSEGV when the http_client module performs an
HTTPS request while the tls module is also loaded. After analysis it looks like a conflict between the tls module's OpenSSL memory allocator
override and libcurl sharing the same OpenSSL library. I would like to report it and get the developers' opinion

Environment

- kamailio 5.8.8 (x86_64/linux), AlmaLinux 9.6, sofware recompiled from source.
- OpenSSL 3.5.1
- libcurl 7.76.1, OpenSSL backend
- http_client.so and tls.so are linked against the SAME libcrypto.so.3
- Multi-process (default forking model), tls used for WSS listeners

Call scenario

Call to a registed UA. When the call is being process, the kamailio script (kamailio.cfg) uses http_client_query to POST to Amazon SNS (Simple Notification Service) to handle push notifications. When the call is being process, kamailio crashes when the http_client_query function is called. This is happening for every single call.

I did make sure that tls.so is loaded BEFORE http_client.so as per the doc.

Analysis

Here is the core stack trace:

== Backtrace (trimmed) ==

  #0  ossl_asn1_primitive_free (pval=..., it=0x0, embed=0) at crypto/asn1/tasn_fre.c:174
  #1  ossl_asn1_primitive_free (...) at crypto/asn1/tasn_fre.c:203
  #2  ossl_asn1_template_free (...) at crypto/asn1/tasn_fre.c:146
  #3  ossl_asn1_item_embed_free (...) at crypto/asn1/tasn_fre.c:114
  #4  ASN1_item_free (...) at crypto/asn1/tasn_fre.c:20
  #5  x509_pubkey_ex_free (...) at crypto/x509/x_pubkey.c:91
  ...
  #27 PEM_X509_INFO_read_bio_ex (...) at crypto/pem/pem_info.c:168
  #28 X509_load_cert_crl_file_ex (...) at crypto/x509/by_file.c:250
  #31 X509_STORE_load_file_ex (..., file="/etc/pki/tls/certs/ca-bundle.crt", ...) at crypto/x509/x509_d2.c:57
  #32 SSL_CTX_load_verify_file (..., CAfile="/etc/pki/tls/certs/ca-bundle.crt") at ssl/ssl_lib.c:5613
  #33 ossl_connect_step1 (...) at ../../lib/vtls/openssl.c:3043
  #34 ossl_connect_common (...) at ../../lib/vtls/openssl.c:4045
  #35 Curl_ssl_connect_nonblocking (...) at ../../lib/vtls/vtls.c:370
  #37 Curl_http_connect (...) at ../../lib/http.c:1519
  #40 multi_runsingle (...) at ../../lib/multi.c:1847
  #41 curl_multi_perform (...) at ../../lib/multi.c:2403
  #44 curl_easy_perform (data=...) at ../../lib/easy.c:715
  #45 curL_request_url (..., _url="https://.../notifications/event"<https://.../notifications/event>, ...) at functions.c:298
  #46 http_client_request_c (...) at functions.c:720
  #48 http_client_query (...) at functions.c:744
  #50 w_http_query_script (...) at http_client.c:1007
  #51 w_http_query_post (...) at http_client.c:1024
  #52 do_action ()
  ...

Claude point of view on the issue

Because http_client uses libcurl, which uses the very same OpenSSL, libcurl's
TLS allocations also go into Kamailio shared memory. OpenSSL 3.x keeps process
-global state (OSSL_LIB_CTX, providers, the OSSL_DECODER method cache, algorithm
fetch caches). When libcurl triggers the ASN.1 decode of the CA bundle, OpenSSL
writes into those global caches - which now live in SHM and are shared by every
worker process - while the pointers referencing them are in each process's
private (COW) memory. OpenSSL's internal mutexes are process-local, so multiple
workers end up reading/writing the same SHM OpenSSL structures concurrently
without effective cross-process locking, corrupting the SHM heap. The crash then
surfaces in the ASN.1 decoder (corrupted item, it=NULL).

Related issues / pull requests :

  *   https://github.com/kamailio/kamailio/pull/4640 - removal of SHM and pthreads shims in multi-threaded mode
  *   https://github.com/kamailio/kamailio/issues/2912

My questions:

  *   Should we migrate to 6.1 or backport the PR 4640?
  *   Could it be caused by the fact that libssh is autoinitializing itself when the .so is loaded ?
  *   Has someone run into this?

Thanks in advance,

Emmanuel

__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions -- [email protected]
To unsubscribe send an email to [email protected]
Important: keep the mailing list in the recipients, do not reply only to the sender!