Re: assertion failed: Queue.cc:388: "EX"
Alex Rousskov <[email protected]>
| Newsgroups | gmane.comp.web.squid.general |
|---|---|
| Message-ID | <[email protected]> |
On 2025-03-13 07:49, Andrey K wrote: > I implemented the A-example functionality and made PR: > https://github.com/squid-cache/squid/pull/2023 Thank you. > ButthemodifiedSquid projectis notbeingbuilt ongithub,althoughthere are > noproblemsduringthe localbuilds. > Alex, Amos and Francesco, could you see what the problem is? Just to close this mailing list thread: PR #2023 problems are being handled on GitHub. Alex. > вт, 11 мар. 2025 г. в 17:40, Andrey K: > > Hello Alex, > > Thank you for sharing your thoughts. > The A-example looks easier to implement and more reliable. > > Kind regards, > Ankor. > > вт, 11 мар. 2025 г. в 17:12, Alex Rousskov > <[email protected] > <mailto:[email protected]>>: > > On 2025-03-10 23:56, Andrey K wrote: > > > > Alex: FWIW, related future Squid improvements may include: > > > * Detecting such shared memory segments clashes; refusing > to start. > > > * Disabling shared memory use when caching is completely > disabled. > > > But ... segments may remain from the previous > > Squid crash. Thus, in my opinion, it is impractical to refuse > to launch > > the program. > > "Squid should overwrite segments left by a crashed Squid" does > not imply > that "detecting segment clashes among independent Squid > instances is > impractical". > > The following rough examples are _not_ implementation > blueprints, but > they illustrate the lack of the above implication: > > A. If Squid were to incorporate PID filename[^1] into segment > names, > then segments from different Squid instances will not clash. > > B. If Squid were to write its PID filename[^1] into a shared memory > segment, then Squid would be able to detect that another running > instance is using the same shared memory segment (while still > overwriting segments left by an earlier crash). > > > [^1]: Full Squid PID filename are already guaranteed to be > unique across > Squid instances that may overwrite each other segments because > Squid > refuses to start if another Squid instance is using its PID > file. The > only red flag I can think of here is esoteric chrooted > instances, but I > would not be surprised if those instances cannot share segments. > > > > I think it's enough to put a warning in the code when > detecting old > > segments. > > I disagree that a level-0 warning during regular/uneventful > startups is > a good idea, especially if there will be one warning for every > segment. > We should try to address the actual problem (i.e. refuse to start a > segment-clashing concurrent instance) rather than allowing > clashes but > adding a yet another warning about a usually benign event. > > Alex. > > > > Something like that: > > void > > Ipc::Mem::Segment::create(const off_t aSize) > > { > > assert(aSize > 0); > > assert(theFD < 0); > > > > int xerrno = 0; > > > > // Why a brand new segment? A Squid crash may leave a > reusable > > segment, but > > // our placement-new code requires an all-0s segment. We > could > > truncate and > > // resize the old segment, but OS X does not allow using > O_TRUNC with > > // shm_open() and does not support ftruncate() for old > segments. > > if (!createFresh(xerrno) && xerrno == EEXIST) { > > *debugs(54, DBG_CRITICAL, "WARNING: there is an old shared > memory > > segment: '" << theName << "'. Perhaps it was left after the > previous > > crash of the Squid. We will remove it. Or it may be a sign > that another > > instance of the Squid is running. In this case, you must > launch the > > program with the -n option and specify the unique name of the > service.");* > > unlink(); > > createFresh(xerrno); > > } > > Kind regards, > > Ankor. > > > > > > сб, 8 мар. 2025 г. в 08:14, Andrey K <[email protected] > <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>>: > > > > Hello Alex, > > > > Thanks for the analysis. > > Squidonlyallowsalphanumericcharactersinthe servicename, > > soitrefusesto usethe originalservice namein the -n option (-n > > squid.user.service). > > I added-n squiduser option to the ExecStart string of the > second > > instance. > > Now it looks good: > > > > # the first instance > > lsof -p 3746105 | grep shm > > squid 3746105 root mem REG 0,23 525572 > > 1213614834 /dev/shm/squid-cf__queues.shm > > squid 3746105 root mem REG 0,23 136 > > 1213614835 /dev/shm/squid-cf__readers.shm > > squid 3746105 root mem REG 0,23 8 > > 1213614833 /dev/shm/squid-cf__metadata.shm > > squid 3746105 root 7u REG 0,23 8 > > 1213614833 /dev/shm/squid-cf__metadata.shm > > squid 3746105 root 8u REG 0,23 525572 > > 1213614834 /dev/shm/squid-cf__queues.shm > > squid 3746105 root 9u REG 0,23 136 > > 1213614835 /dev/shm/squid-cf__readers.shm > > > > # the second instance > > lsof -p 3685356 | grep shm > > squid.use 3685356 root mem REG 0,23 > 2093368 > > 1212704041 /dev/shm/squiduser-tls_session_cache.shm > > squid.use 3685356 root mem REG 0,23 > 525572 > > 1212704039 /dev/shm/squiduser-cf__queues.shm > > squid.use 3685356 root mem REG 0,23 > 136 > > 1212704040 /dev/shm/squiduser-cf__readers.shm > > squid.use 3685356 root mem REG 0,23 > 8 > > 1212704038 /dev/shm/squiduser-cf__metadata.shm > > squid.use 3685356 root 6u REG 0,23 > 8 > > 1212704038 /dev/shm/squiduser-cf__metadata.shm > > squid.use 3685356 root 7u REG 0,23 > 525572 > > 1212704039 /dev/shm/squiduser-cf__queues.shm > > squid.use 3685356 root 8u REG 0,23 > 136 > > 1212704040 /dev/shm/squiduser-cf__readers.shm > > squid.use 3685356 root 9u REG 0,23 > 2093368 > > 1212704041 /dev/shm/squiduser-tls_session_cache.shm > > > > Kind regards, > > Ankor. > > > > > > > > > > > > пт, 7 мар. 2025 г. в 17:48, Alex Rousskov > > <[email protected] > <mailto:[email protected]> > > <mailto:[email protected] > <mailto:[email protected]>>>: > > > > On 2025-03-07 06:50, Andrey K wrote: > > > > > Squid Cache: Version 6.13 > > > Service Name: squid > > > > > Squid Cache: Version 6.10 > > > Service Name: squid > > > > > # the first instance > > > 1318 DEL ... 30205 > /dev/shm/squid-cf__queues.shm > > > 1318 DEL ... 30206 > /dev/shm/squid-cf__readers.shm > > > 1318 DEL ... 30204 > /dev/shm/squid-cf__metadata.shm > > > 1318 8u ... 8 30204 > /dev/shm/squid-cf__metadata.shm > > (deleted) > > > 1318 9u ... 525572 30205 > /dev/shm/squid-cf__queues.shm > > (deleted) > > > 1318 10u ... 136 30206 > /dev/shm/squid-cf__readers.shm > > (deleted) > > > > > > # the second instance > > > 1514 mem ... 2093368 31497 > > /dev/shm/squid-tls_session_cache.shm > > > 1514 mem ... 525572 31495 > /dev/shm/squid-cf__queues.shm > > > 1514 mem ... 136 31496 > /dev/shm/squid-cf__readers.shm > > > 1514 mem ... 8 31494 > /dev/shm/squid-cf__metadata.shm > > > 1514 6u ... 8 31494 > /dev/shm/squid-cf__metadata.shm > > > 1514 7u ... 525572 31495 > /dev/shm/squid-cf__queues.shm > > > 1514 8u ... 136 31496 > /dev/shm/squid-cf__readers.shm > > > 1514 9u ... 2093368 31497 > > /dev/shm/squid-tls_session_cache.shm > > > > As suspected, these two Squid instances use the same > shared memory > > segments (e.g., /dev/shm/squid-cf*). Such shared use > violates > > critical > > code assumptions and results in undefined behavior. > > > > > > > Maybe I'm not experiencing any difficulties > because I have > > caching turned off on > > > both instances? > > > > Well, you _are_ experiencing at least one difficulty > -- the > > assertion > > that started this email thread. If you have fully > disabled > > caching, the > > difficulties you experience should not include cache > corruption. > > However, it looks like at least one of the two > instances does > > cache TLS > > sessions (in /dev/shm/squid-tls_session_cache.shm). > If both > > instances do > > that, then all bets are off! > > > > > > FWIW, related future Squid improvements may include: > > > > * Detecting such shared memory segments clashes; > refusing to start. > > * Disabling shared memory use when caching is > completely disabled. > > > > Quality pull requests welcome. > > > > > > Cheers, > > > > Alex. > > > > > > > чт, 6 мар. 2025 г. в 17:11, Alex Rousskov: > > > > > > On 2025-03-06 08:59, Amos Jeffries wrote: > > > > On 6/03/25 19:17, Andrey K wrote: > > > >> Hello, > > > >> > > > >> I have a similar configuration: two SMP squids > > running on the > > > same OEL > > > >> host. > > > >> > > > >> They were built with different > configurations: with > > different > > > >> installation path prefixes and > different names of > > binary files: > > > squid > > > >> and squid.user and they listen to > different ports. > > > >> They are launched from two different > > services:squid.service and > > > >> squid.user.service with the service > Type=forking: > > > >> > > > >> ExecStart=/usr/sbin/squid -sYC > > > >> ExecStart=/sbin/squid.user -f > > /etc/squid.user/squid.conf > > > >> > > > >> I have not experienced any troubles with this > > configuration yet. > > > >> > > > >> /> Please be aware that "squid -n ..." is a > > REQUIREMENT for running/ > > > >> /multiple Squid instances on the same machine > > regardless of what > > > features > > > >> are used./ > > > >> > > > >> Could you please tell me if I should use > the -n > > option in the > > > >> ExecStart strings? > > > >> The arguments of the options should be the > service names? > > > >> > > > >> ExecStart=/usr/sbin/squid -sYC -n > squid.service > > > >> ExecStart=/sbin/squid.user -f > > /etc/squid.user/squid.conf -n > > > >> squid.user.service > > > >> > > > > Yes you should. The different ./configure > options has > > helped you > > > avoid > > > > major issues, but some may still appear. > > > > > > I agree. Moreover, I do not understand how > your two SMP > > Squids could > > > work correctly without distinct service names > because (on > > OEL) I would > > > expect them to share the same shared memory > segments > > (which they must > > > not do to remain distinct instances). > > > > > > What is your Squid version? Can you tell how > your Squids > > name their > > > shared memory segment "files"? For example, on > some Linux > > OSes, those > > > segments could be in /var/run/shm/ with names like > > > squid-tr_map_anchors.shm and squid-tr_spaces.shm. > > > > > > > > > Thank you, > > > > > > Alex. > > > > > > _______________________________________________ > > > squid-users mailing list > > > [email protected] > <mailto:[email protected]> > > <mailto:[email protected] > <mailto:[email protected]>> > > > <mailto:[email protected] > <mailto:[email protected]> > > <mailto:[email protected] > <mailto:[email protected]>>> > > > https://lists.squid-cache.org/listinfo/squid-users > <https://lists.squid-cache.org/listinfo/squid-users> > > <https://lists.squid-cache.org/listinfo/squid-users > <https://lists.squid-cache.org/listinfo/squid-users>> > > > > <https://lists.squid-cache.org/listinfo/squid-users > <https://lists.squid-cache.org/listinfo/squid-users> > > <https://lists.squid-cache.org/listinfo/squid-users > <https://lists.squid-cache.org/listinfo/squid-users>>> > > > > > > > _______________________________________________ > squid-users mailing list > [email protected] > <mailto:[email protected]> > https://lists.squid-cache.org/listinfo/squid-users > <https://lists.squid-cache.org/listinfo/squid-users> > _______________________________________________ squid-users mailing list [email protected] https://lists.squid-cache.org/listinfo/squid-users