Re: Deferred: Upgrades to datatracker infrastructure.

Today’s attempt to deploy a series of upgrades to the infrastructure 
underlying the datatracker was not successful, despite extensive testing 
in staging.


We have reverted to the previous state and all services are available as 
before. We are monitoring for any further signs of trouble. We are 
currently analyzing the deployment logs to identify the cause of the 
failure.


A time for the next attempt will be set and announced once the analysis 
and actions required to address the cause(s) are complete.


The disruption (from 1824 to 1852 UTC) occurred during the initial step 
of moving from memcached to Redis, affecting the stability of the 
cluster. Other applications, primarily mailarchive, were impacted during 
the disruption. We did not start upgrading Kubernetes.


Robert Sparks

Senior Director of IT for the IETF LLC.