[Lvfs-announce] Please upload a signed report before pushing to LVFS stable

Hi all,

As the LVFS edges towards 80 million firmware downloads we need to
continue to think about scaling up to the next order of magnitude. We
want to push more updates to more devices, faster, and at the same
time increase the quality of updates delivered. At the moment the
success rate for delivering updates is about 99.97%, which sounds
great until you calculate what 0.03% of 80 million is.

As well as fixing the =E2=80=9Coops I pushed the wrong firmware for the wro=
ng
device=E2=80=9D problem we also want to fix the scaling problem. At the mom=
ent
various QA teams test each firmware update on the actual target
device, which is something that doesn=E2=80=99t scale as we=E2=80=99ve blow=
n past 1000
supported devices a long time ago. I=E2=80=99m no longer testing all the
different plugins for each fwupd release as this was taking tens of
hours to unplug and plug different devices, chargers, docks etc. I
know Google also does semi-automated testing like this for ChromeOS
for WWCB devices but even that doesn=E2=80=99t scale up to thousands of
devices.

The only thing we can do is push the onus of testing onto the firmware
uploader, which is hopefully _already_ testing the firmware before
pushing it to millions of people. I now need the vendors uploading
firmware to actually make testing a formal part of the =E2=80=9Cput files o=
n
the LVFS=E2=80=9D process, and to attest that the process was done. The way
we=E2=80=99ve been doing this for the last 2 years is with a firmware =E2=
=80=9Creport=E2=80=9D
and this is something that tens of thousands of users, and a few dozen
vendors are already doing.

I=E2=80=99m soon going to merge some new deliberately scary warning dialogs=
 to
the LVFS https://gitlab.com/fwupd/lvfs-website/-/merge_requests/1382
for vendors that want to push firmware to the stable without uploading
a report. As a reminder, an anonymous report can be uploaded by
=E2=80=9Cfwupdmgr update=E2=80=9D and then =E2=80=9Cfwupdmgr report-history=
=E2=80=9D =E2=80=93 we ask all
Linux users to upload the same kind of report even if they do
=E2=80=9Cfwupdmgr get-updates=E2=80=9D after the update has completed. Even=
 better
would be for the uploader to sign the report, so we know that it was
the uploader that performed the testing rather than some random
person.

It takes two minutes to set up =E2=80=9Csigned reports=E2=80=9D by followin=
g the steps
in https://lvfs.readthedocs.io/en/latest/testing.html#signed-reports =E2=80=
=93
but the simple version is:
    =E2=80=A2 Log into the LVFS as your user
    =E2=80=A2 Click the =E2=80=9Cperson icon=E2=80=9D on the top right, or =
click
https://fwupd.org/lvfs/profile
    =E2=80=A2 Click the =E2=80=9CUpload Certificate=E2=80=9D button in the =
=E2=80=9CClient Certificates=E2=80=9D section
    =E2=80=A2 Navigate to /var/lib/fwupd/pki and upload client.pem on each
machine you're using for testing (you can upload multiple certificates
for different computers)
    =E2=80=A2 When uploading reports, instead of =E2=80=9Cfwupdmgr report-h=
istory=E2=80=9D use
=E2=80=9Cfwupdmgr report-history --sign=E2=80=9D and then authenticate when=
 prompted
    =E2=80=A2 If you=E2=80=99ve already uploaded reports without the --sign=
 flag, you
can use =E2=80=9Cfwupdmgr report-history --sign --force=E2=80=9D to re-uplo=
ad the same
report to the LVFS

There are a lot more details in
https://lvfs.readthedocs.io/en/latest/testing.html#signed-reports
including screenshots. The report will also include what OS and
platform you=E2=80=99re testing on, so if you=E2=80=99re telling Red Hat yo=
u=E2=80=99re
testing on RHEL 9, and telling Google you=E2=80=99re testing on ChromeOS yo=
u
need to make sure that=E2=80=99s 100% true :)

I=E2=80=99m going to start enforcing this for newly added protocols and the=
n
in several months time also make this a hard requirement for existing
protocols like UEFI capsules and NVMe.

I=E2=80=99m happy to answer questions and queries off-list. Please share th=
is
email with your respective teams, and please convey both how important
and how simple this workflow change will be.

Thanks.

Richard.