Update on CPANTesters.org stability

CPT Community:

Here is a quick update on some changes we've made to stabilize 
*cpantesters.org*. Preaction was able to get a handful of us SSH access 
to the CPT servers. The "outages" and latency we've all noticed lately 
have been related to Apache resource exhaustion on the CPT4 web server 
that handles: www, api, reports, etc. We're still not 100% what is 
causing Apache to go unresponsive but we wanted to let everyone know 
we've made some improvements in the last week:

  * Increased Apache listeners to facilitate higher levels of traffic
  * Preaction reconfigured the Fastly CDN to remove a failed node
  * 8GB swapfile was added to give the box some extra resources
  * API.pm cleanup to fix a Mojolicious configuration artifact
  * Robust monitoring tools installed to watch health in real time

Next we will need to investigate the health of the DB server as well. If 
you have MySQL skills we could use your help diagnosing high load we're 
seeing. We have a rather large MySQL DB (~1.5TB) that contains about 20 
years of CPT history. The DB server gets ~37 queries per second and has 
a sustained throughput of about 600Mb/s of read from the disks. From 
what I can tell the DB server is healthy and functional, but it's 
working /*hard*/.

If you have expertise in this field please reach out, or speak out at 
PTC so we can not only stabilize CPT but improve it and take it to the 
next level.

-- Scottchiefbaker