Fwd: [Wikitech-l] eqiad->codfw datacenter switchover, weeks of Apr 17th/May 1st

> From: Faidon Liambotis <[email protected]>
> Date: April 7, 2017 at 8:58:09 AM CDT
> To: Wikimedia developers <[email protected]>
> Subject: [Wikitech-l] eqiad->codfw datacenter switchover, weeks of Apr 17th/May 1st
> Reply-To: Wikimedia developers <[email protected]>
> 
> Hi all,
> 
> You may have heard already that, like last year, we are planning to
> switch our active datacenter from eqiad to codfw in the week of April
> 17th and back to eqiad two weeks later, on the week of May 1st. We do
> this periodically in order to exercise our ability to run from the
> backup site in case of a disaster, as well as our ability to switch
> seamlessly to it with little user impact.
> 
> Switching will be a gradual, multi-step process, the most visible step
> of which will be the switch of MediaWiki application servers and
> associated data stores. This will happen on April 19th (eqiad->codfw)
> and May 3rd (codfw->eqiad), both at 14:00 UTC. During those windows, the
> sites will be placed into read-only mode, for a period that we estimate
> to last approximately 20 to 30 minutes.
> 
> Furthermore, the deployment train will freeze for the weeks of April
> 17th and May 1st[1], but operate normally on the week of April 24th, in
> order to exercise our ability to deploy code while operating from the
> backup datacenter.
> 
> 1: https://wikitech.wikimedia.org/wiki/Deployments
> 
> Compared to last year we have improved our processes considerably[2], in
> particular by making more services operate in an active/active manner,
> as well as by working on an automation and orchestration framework[3] to
> perform parallel executions across the fleet. The core of the MediaWiki
> switchover will be performed semi-automatically using a new software[4]
> that will execute all the necessary commands in sequence with little
> human involvement, and thus lowering the risk of introducing errors and
> delays.
> 
> 2: https://wikitech.wikimedia.org/wiki/Switch_Datacenter
> 3: https://github.com/wikimedia/cumin
> 4: https://github.com/wikimedia/operations-switchdc
> 
> Improving and automating our processes means that we're not going to be
> following the exact same steps as last year. Because of that, and
> because of other changes introduced in our environment over the course
> of the year, there is a possibility of errors creeping into the process.
> We'll certainly try to fix any issues that arise during those weeks and
> we'd like to ask everyone to be on high-alert and vigilant.
> 
> To report any issues, please use one of the following channels:
> 
> 1. File a Phabricator issue with project #codfw-rollout
> 2. Report issues on IRC: Freenode channel #wikimedia-tech (if urgent, or
> during the migration)
> 3. Send an e-mail to the Operations list: [email protected] (any time)
> 
> Thanks,
> Faidon
> --
> Faidon Liambotis
> Principal Operations Engineer
> Acting Director of Technical Operations
> Wikimedia Foundation
> 
> _______________________________________________
> Wikitech-l mailing list
> [email protected]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikipedia-l mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikipedia-l