Re: Are incident reports still being published to Wikitech?

--===============5016589017804175944==
Content-Type: multipart/alternative; boundary="0000000000006259e806519d298f"

--0000000000006259e806519d298f
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hello,

On Sun, 26 Apr 2026 at 21:11, a smart kitten via Wikitech-l <
[email protected]> wrote:

> Hey all,
>
> Just a question -- do folks know if incident reports are still being
> completed & published on Wikitech (such that they're listed at [0])?
>
> I just thought to ask given that - for me - the Phabricator search result=
s
> at [1] list 9 tasks created since the start of this year that are tagged
> with `#Wikimedia-Incident` (& some folks with an NDA may be able to see
> more tasks than that), but the Wikitech page at [0] currently only lists
> there as having been one incident that's occurred since the start of this
> year [2].
>

The short answer is no, they are not being published regularly any more -
there have been a few shifts in incident response that we need to document
and align our processes on.

There are a few factors at work here:
* One of the main aspects is that the majority of infrastructure-related
incidents that we=E2=80=99ve been dealing with over the last few months hav=
e been
of a sensitive nature. As per WP:DNFTT
<https://en.wikipedia.org/wiki/Wikipedia:Deny_recognition> and our policies
around sensitive information, we don=E2=80=99t publish information on our r=
esponse
to these outages even if some of the action items are fine to be public.
* Another factor is the need to scale our incident tracking process. We now
coordinate all incidents using Corto
<https://wikitech.wikimedia.org/wiki/Corto>, which creates Phabricator
tickets automatically - defaulting to closed.
* The last factor is our incident review process. To account for the
diversity of incidents and involvement of many teams, we have moved to a
less rigid process where the filling out of the wikitech document and the
scorecard etc don=E2=80=99t feature. This is the main reason for the lack o=
f
updates to the incident status page.

The process of filling out the wikitech incident reports has been a fairly
arduous process of copying information out of Phabricator tasks and Google
docs which duplicates effort.
For the time being, given the more concerted use of Phabricator tasks in
the era of Corto, we will try to update and publish Phabricator tickets
wherever possible. When an incident is being closed out by the incident
coordinator, they will transfer relevant information from the Google doc
(if needed) and open up the task if it is suitable to do so in order to
provide context on our outages and any interesting technical lessons
learned. We=E2=80=99ll also work towards formalising a process to replace t=
he
existing process officially.

Cheers,
Hugh

--0000000000006259e806519d298f
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hello,=C2=A0</div><div><br></div><div dir=3D"ltr">On =
Sun, 26 Apr 2026 at 21:11, a smart kitten via Wikitech-l &lt;<a href=3D"mai=
lto:[email protected]" target=3D"_blank">[email protected]=
media.org</a>&gt; wrote:</div><div dir=3D"ltr"><div class=3D"gmail_quote"><=
blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l=
eft:1px solid rgb(204,204,204);padding-left:1ex">Hey all,<br>
<br>
Just a question -- do folks know if incident reports are still being comple=
ted &amp; published on Wikitech (such that they&#39;re listed at [0])?<br>
<br>
I just thought to ask given that - for me - the Phabricator search results =
at [1] list 9 tasks created since the start of this year that are tagged wi=
th `#Wikimedia-Incident` (&amp; some folks with an NDA may be able to see m=
ore tasks than that), but the Wikitech page at [0] currently only lists the=
re as having been one incident that&#39;s occurred since the start of this =
year [2].<br></blockquote><div><br></div><div>The short answer is no, they =
are not being published regularly any more - there have been a few shifts i=
n incident response that we need to document and align our processes on.=C2=
=A0</div><div><br></div><div>There are a few factors at work here:</div><di=
v>* One of the main aspects is that the majority of infrastructure-related =
incidents that we=E2=80=99ve been dealing with over the last few months hav=
e been of a sensitive nature. As per <a href=3D"https://en.wikipedia.org/wi=
ki/Wikipedia:Deny_recognition">WP:DNFTT</a> and our policies around sensiti=
ve information, we don=E2=80=99t publish information on our response to the=
se outages even if some of the action items are fine to be public.</div><di=
v>* Another factor is the need to scale our incident tracking process. We n=
ow coordinate all incidents using <a href=3D"https://wikitech.wikimedia.org=
/wiki/Corto">Corto</a>, which creates Phabricator tickets automatically - d=
efaulting to closed.</div><div>* The last factor is our incident review pro=
cess. To account for the diversity of incidents and involvement of many tea=
ms, we have moved to a less rigid process where the filling out of the wiki=
tech document and the scorecard etc don=E2=80=99t feature. This is the main=
 reason for the lack of updates to the incident status page.</div><div><br>=
</div><div>The process of filling out the wikitech incident reports has bee=
n a fairly arduous process of copying information out of Phabricator tasks =
and Google docs which duplicates effort.</div><div>For the time being, give=
n the more concerted use of Phabricator tasks in the era of Corto, we will =
try to update and publish Phabricator tickets wherever possible. When an in=
cident is being closed out by the incident coordinator, they will transfer =
relevant information from the Google doc (if needed) and open up the task i=
f it is suitable to do so in order to provide context on our outages and an=
y interesting=C2=A0technical lessons learned. We=E2=80=99ll also work towar=
ds formalising a process to replace the existing process officially.</div><=
div><br></div><div>Cheers,</div><div>Hugh</div></div></div>
</div>

--0000000000006259e806519d298f--

--===============5016589017804175944==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Wikitech-l mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
--===============5016589017804175944==--