Re: Squid 6 with ssl-bump doesn't cache binary content over 100 kb

On 2025-02-25 09:47, Thomas PALFRAY wrote:

> we tried version 6.13 as recommended, but the behavior is the same.

Thank you for testing v6.13. That test eliminates many suspects.


> What additional information would you need to understand the the problem 

For the next step in triage, I can offer a free private review of your 
cache.log file collected while reproducing the problem using as few 
transactions as possible and enabling full debugging (e.g., setting 
debug_options to ALL,9). More hints are available at
https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction

If you would like to proceed with the above analysis, please email me a 
link to the corresponding compressed cache.log.


Thank you,

Alex.


> *De :*squid-users <[email protected]> *De la 
> part de* Thomas PALFRAY
> *Envoyé :* lundi 3 février 2025 17:08
> *À :* [email protected]
> *Objet :* [squid-users] Squid 6 with ssl-bump doesn't cache binary 
> content over 100 kb
> 
> Hello everyone,
> 
> My team and I are working on setting up *a squid with ssl-bump* to cache 
> binary content (jpeg, png, pdf and json) on a remote site over HTTPS.
> 
> The size of the binary content can vary from a few dozen KB to several 
> hundred MB.
> 
> We had a working HTTP configuration under squid 3.14, but for security 
> reasons, all our links must now go through HTTPS.
> 
> We've tried configuring Squid 5 and Squid 6, but the behavior described 
> below is the same for both versions of the tool.
> 
> In fact, we're seeing 200/TCP_MISS_ABORTED codes for content larger than 
> around 100kb.
> 
> When content sizes are smaller, the expected behaviour occurs and data 
> is returned from the cache.
> 
> On the client side, for content larger than 100kb :
> 
>   * squid 5 returns x-cache = “MISS” and x-cache-lookup = “HIT”.
>   * squid 6 returns cache-status = “url;detail=match”, which is
>     equivalent if my understanding is correct.
> 
> We first thought it might be linked to the following bug: 
> _https://bugs.squid-cache.org/show_bug.cgi?id=5214 
> <https://bugs.squid-cache.org/show_bug.cgi?id=5214>_, which encouraged 
> us to try version 6, but without success.
> 
>   * Is it a configuration problem that we missed?
>   * Can you help us ?
> 
> HTTP client-side headers returned by squid 6 :
> 
> //
> 
> /[2025-01-14T08:46:49.875] [TRACE] default - [0/1] getContentStream 
> (user) header: {/
> 
> /  "date": "Tue, 14 Jan 2025 05:47:30 GMT",/
> 
> /  "server": "Apache-Chemistry-OpenCMIS/1.2.0_1859862-XXXXX-1",/
> 
> /  "strict-transport-security": "max-age=15768000",/
> 
> /"x-xss-protection": "1; mode=block",/
> 
> ///"x-frame-options": "SAMEORIGIN",/
> 
> /  "x-content-type-options": "nosniff",/
> 
> /  "access-control-allow-origin": "*",/
> 
> /  "access-control-expose-headers": "Content-Disposition",/
> 
> /  "cache-control": "public, s-maxage=3600, must-revalidate",/
> 
> /  "etag": 
> "\"08bd240128b475722db82d36c7ae7f164c37cab4ad2480abae052875fe7bc3bfdfef9996197d40110a13208d39a3db3a789879bc31803c82f25211eeba505455\"",/
> 
> /  "content-disposition": "inline; filename=FILENAME.jpg",/
> 
> /  "content-type": "image/jpeg",/
> 
> /  "content-length": "2069587",/
> 
> /  "content-security-policy": "default-src 'none'; connect-src 'self'; 
> font-src 'self' fonts.gstatic.com; img-src blob: 'self'; script-src 
> 'self' 'unsafe-eval' 'unsafe-inline'; style-src 'self' 'unsafe-inline' 
> fonts.googleapis.com",/
> 
> */  "cache-status": "sv-infra-pxy4;detail=match",/*
> 
> */"via": "1.1 sv-infra-pxy4 (squid/6.10)",/*
> 
> ///"connection": "close"/
> 
> /}/
> 
> /[2025-01-14T08:46:49.875] [INFO] default - [0/1][2] getCS oId:'369926' 
> sId:*'80669'*name:' filename= FILENAME.jpg' type:'image/jpeg' 
> size:*'2021.08ko'*res:[via:true hit:false length:true]/
> 
> //
> 
> //
> 
> squid 6 acces.log extract:
> 
> //
> 
> //
> 
> //
> 
> full server configuration file :
> 
> //
> 
> /acl localnet src 0.0.0.1-0.255.255.255        # RFC 1122 "this" network 
> (LAN)/
> 
> /acl localnet src 10.0.0.0/8                   # RFC 1918 local private 
> network (LAN)/
> 
> /acl localnet src 100.64.0.0/10                           # RFC 6598 
> shared address space (CGN)/
> 
> /acl localnet src 169.254.0.0/16        # RFC 3927 link-local (directly 
> plugged) machines/
> 
> /acl localnet src 172.16.0.0/12                           # RFC 1918 
> local private network (LAN)/
> 
> /acl localnet src 192.168.0.0/16                        # RFC 1918 local 
> private network (LAN)/
> 
> /acl localnet src fc00::/7         # RFC 4193 local private network range/
> 
> /acl localnet src fe80::/10      # RFC 4291 link-local (directly 
> plugged) machines/
> 
> //
> 
> /acl SSL_ports port 443/
> 
> /acl Safe_ports port 80                              # http/
> 
> /acl Safe_ports port 21                              # ftp/
> 
> /acl Safe_ports port 443                           # https/
> 
> /acl Safe_ports port 70                              # gopher/
> 
> /acl Safe_ports port 210                           # wais/
> 
> /acl Safe_ports port 1025-65535       # unregistered ports/
> 
> /acl Safe_ports port 280                           # http-mgmt/
> 
> /acl Safe_ports port 488                           # gss-http/
> 
> /acl Safe_ports port 591                           # filemaker/
> 
> /acl Safe_ports port 777                           # multiling http/
> 
> //
> 
> /#ACL pour SmartGED/
> 
> /acl smartged-mime req_mime_type -i ^image/jpeg$/
> 
> /acl smartged-mime req_mime_type -i ^image/png$/
> 
> /acl smartged-mime req_mime_type -i ^application/pdf$/
> 
> /acl smartged-mime req_mime_type -i ^application/json$/
> 
> //
> 
> /acl intermediate_fetching transaction_initiator certificate-fetching/
> 
> //
> 
> /http_access deny !Safe_ports/
> 
> /http_access deny CONNECT !SSL_ports/
> 
> //
> 
> /# Only allow cachemgr access from localhost/
> 
> /http_access allow localhost manager/
> 
> /http_access deny manager/
> 
> /http_access allow localnet/
> 
> /http_access deny all/
> 
> //
> 
> /logformat toto %{%d/%b/%Y:%H:%M:%S}tl.%tu %>a %rm %03Hs/%Ss %6tr %ru %mt/
> 
> //
> 
> /access_log /var/log/squid/access.log toto/
> 
> //
> 
> /http_port 3128  ssl-bump generate-host-certificates=on 
> dynamic_cert_mem_cache_size=4MB tls-cert=/etc/squid/ssl_cert/xxxx_.pem 
> tls-key=/etc/squid/ssl_cert/xxxx_.key 
> tls-dh=/etc/squid/ssl_cert/dhparam.pem/
> 
> /tls_outgoing_options 
> cipher=ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384/
> 
> /sslcrtd_program /usr/lib/squid/security_file_certgen -s 
> /var/spool/squid/ssl_db -M 20MB/
> 
> /sslcrtd_children 5/
> 
> /ssl_bump server-first all/
> 
> /ssl_bump splice all/
> 
> /sslproxy_cert_error allow all/
> 
> //
> 
> /# ACL deny pour les fichiers javascript/
> 
> /acl denyjs urlpath_regex \.js/
> 
> /# ok deny all requests above/
> 
> /cache deny denyjs/
> 
> //
> 
> /# Leave coredumps in the first cache dir/
> 
> /coredump_dir /cache1/squid/
> 
> //
> 
> /# Affichage des requetes GET completes dans cache.log/
> 
> /strip_query_terms off/
> 
> //
> 
> /cache_mem 20480 MB/
> 
> /# début d'éviction du cache/
> 
> /cache_swap_low 50/
> 
> /# aggressivité maximal de l'algorithme d'éviction du cache/
> 
> /cache_swap_high 80/
> 
> /# The LRU policies keeps recently referenced objects./
> 
> /cache_replacement_policy lru/
> 
> //
> 
> /#/
> 
> /# Add any of your own refresh_pattern entries above these./
> 
> /#/
> 
> /refresh_pattern ^ftp:                1440    20%     10080/
> 
> /refresh_pattern -i (/cgi-bin/|\?) 0       0%        0/
> 
> /refresh_pattern .                          0             20%     4320/
> 
> //
> 
> /ftp_user [email protected] <mailto:[email protected]>_/
> 
> /#acl QUERY urlpath_regex cgi-bin \?/
> 
> /#no_cache deny QUERY/
> 
> /client_request_buffer_max_size 10240 KB/
> 
> /maximum_object_size 1024 MB/
> 
> /# to keep object in memory cache/
> 
> /maximum_object_size_in_memory 10240 KB/
> 
> /#to cause Squid to prefetch the whole file/
> 
> /range_offset_limit 16 MB/
> 
> /#quick_abort_min -1/
> 
> /#Définition de la taille maximum d'un en-tête HTTP lors d'une requête/
> 
> /reply_header_max_size 8192 KB/
> 
> /#taille maximum d'un en-tête de réponse HTTP/
> 
> /reply_header_max_size 8192 KB/
> 
> //
> 
> /# Uncomment and adjust the following to add a disk cache directory./
> 
> /cache_dir aufs /cache1/squid 1024000 16 256 max-size=16777216/
> 
> //
> 
> //
> 
> /error_directory /usr/share/squid/errors/fr/
> 
> /half_closed_clients off/
> 
> /max_filedescriptors 8192/
> 
> /forward_max_tries 50/
> 
> //
> 
> /#Affichage du store-id/
> 
> /cache_store_log daemon:/var/log/squid/store_daemon.log/
> 
> /cache_store_log stdio:/var/log/squid/store_stdio.log/
> 
> /#debug_options "ALL,3 33,7 47,7 61,7 85,7"/
> 
> /# Log all critical and important messages./
> 
> /#debug_options ALL,1/
> 
> /debug_options ALL,3/
> 
> //
> 
> /# Enable SQUID's SNMP/
> 
> /snmp_port 3401/
> 
> /acl snmppublic snmp_community PASdePUBLIC/
> 
> /snmp_access allow snmppublic all/
> 
> Best regards,
> 
> *Thomas PALFRAY*
> 
> 
> _______________________________________________
> squid-users mailing list
> [email protected]
> https://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[email protected]
https://lists.squid-cache.org/listinfo/squid-users