[Lvfs-announce] Back up and running

Richard Hughes <[email protected]> Thu, 26 Nov 2020 16:20:51 +0000
Newsgroups dev.linux.lists.lvfs-announce
Message-ID <CAD2FfiFeJv4TaDmbR9B7M+sUBDVToxn61_AyKtcgi4_D6RzsQg@mail.gmail.com>
Hi all,

I know some of you have been frustrated by recent LVFS instability in
recent days, and for that I apologise. Two issues were identified and
both were fixed today. One was dealing with a socket timeout between
the load balancer and the webapp and another was dealing with how
asynchronous threads were not being used correctly. As you can
appreciate neither were trivial to track down and debug. We didn't see
this on the old instance as we used nginx as a proxy which masked the
problem, and didn't identify this during pre-go-live testing as it
only happens when the server is serving a huge number of slow clients
rather than a synthetic test benchmark.

If anyone gets any further 502 or 503 errors from the webapp please
let me know, but fingers crossed it all seems to be working as I hoped
it would.

Many thanks to the sysadmin team at the Linux Foundation, especially
DW who put in the long hours putting all the pieces together so that I
could do a quick code fix.

Richard.