Got "fact submission failed: first byte timeout", but still report got posted; why?

Today I experienced something unusual (to me, at least):  When submitting CPANtesters reports, I saw error output I have always associated with a failure in submitting the report to the metabase -- and then the web interface indicated that the reports had been submitted!

1. I was testing Module-CPANTS-Analyse-1.03 against a perl built from the perl-5.44.0-RC1 tarball, using Task::CPAN::Reporter.  This was conducted on the FreeBSD-14 server I use often in testing blead and CPAN distros.  I spotted this output:
#####
CPAN::Reporter: Test result is 'pass', All tests successful.
CPAN::Reporter: preparing a CPAN Testers report for Module-CPANTS-Analyse-1.03
CPAN::Reporter: sending test report with 'pass' via Metabase
CPAN::Reporter: Test::Reporter: error from 'Test::Reporter::Transport::Metabase:'
fact submission failed: first byte timeout at /home/jkeenan/testing/perl-5.44.0-RC1/lib/site_perl/5.44.0/Metabase/Client/Simple.pm line 129.
	Metabase::Client::Simple::submit_fact(Metabase::Client::Simple=HASH(0x3a3c82011ba0), CPAN::Testers::Report=HASH(0x3a3c8200ab70)) called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/site_perl/5.44.0/Test/Reporter/Transport/Metabase.pm line 122
	Test::Reporter::Transport::Metabase::send(Test::Reporter::Transport::Metabase=HASH(0x3a3c821d4fa8), Test::Reporter=HASH(0x3a3c82129e28)) called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/site_perl/5.44.0/Test/Reporter.pm line 272
	eval {...} called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/site_perl/5.44.0/Test/Reporter.pm line 272
	Test::Reporter::send(Test::Reporter=HASH(0x3a3c82129e28)) called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/site_perl/5.44.0/CPAN/Reporter.pm line 495
	CPAN::Reporter::_dispatch_report(HASH(0x3a3c8212a978)) called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/site_perl/5.44.0/CPAN/Reporter.pm line 107
	CPAN::Reporter::grade_test(CPAN::Distribution=HASH(0x3a3c7c065f90), "/usr/bin/make test", ARRAY(0x3a3c822d8bd0), 0) called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/site_perl/5.44.0/CPAN/Reporter.pm line 202
	CPAN::Reporter::test(CPAN::Distribution=HASH(0x3a3c7c065f90), "/usr/bin/make test") called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/5.44.0/CPAN/Distribution.pm line 3854
	CPAN::Distribution::test(CPAN::Distribution=HASH(0x3a3c7c065f90)) called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/5.44.0/CPAN/Distribution.pm line 4167
	CPAN::Distribution::install(CPAN::Distribution=HASH(0x3a3c7c065f90)) called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/5.44.0/CPAN/Module.pm line 484
	eval {...} called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/5.44.0/CPAN/Module.pm line 483
	CPAN::Module::rematein(CPAN::Module=HASH(0x3a3c808a9780), "install") called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/5.44.0/CPAN/Module.pm line 592
	CPAN::Module::install(CPAN::Module=HASH(0x3a3c808a9780)) called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/5.44.0/CPAN/Shell.pm line 1901
	CPAN::Shell::rematein("CPAN::Shell", "install", "Module::CPANTS::Analyse") called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/5.44.0/CPAN/Shell.pm line 2068
	CPAN::Shell::__ANON__("CPAN::Shell", "Module::CPANTS::Analyse") called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/5.44.0/App/Cpan.pm line 635
	App::Cpan::__ANON__("Module::CPANTS::Analyse") called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/5.44.0/App/Cpan.pm line 652
	App::Cpan::_default(ARRAY(0x3a3c6a626870), HASH(0x3a3c6c3ee090)) called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/5.44.0/App/Cpan.pm line 536
	App::Cpan::run("App::Cpan", "Module::CPANTS::Analyse") called at ./bin/cpan line 18
#####
As soon as I saw that "fact submission failed: first byte timeout" output, I groaned.  My expectation was that this report would not get into the metabase and would therefore not be reported via the web interface -- which for me is typically https://fast2-matrix.cpantesters.org/.  I further expected that this test run would not be logged in ~/.cpanreporter/reports-sent.db; it was not.

2. I then switched to a different perl build on the same machine: my perl-5.43.12 build from last week.  This run proceeded as desired:  no error output; report visible at https://www.cpantesters.org/cpan/report/42dae052-6fc5-11f1-9d34-a5b4381e9350; an entry recorded in ~/.cpanreporter/reports-sent.db.

3. I then switched back to my perl-5.44.0-RC1 build.  I called:

$ ./bin/cpan -t -f Module::CPANTS::Analyse

Once again I got error output beginning with "fact submission failed: first byte timeout"; once again no entry was logged in ~/.cpanreporter/reports-sent.db.

4. I repeated the previous step against perl-5.44.0-RC1.  This time there was no error output.  The report was soon visible at https://www.cpantesters.org/cpan/report/730feb96-6fc5-11f1-8255-c6d66019e392.  This run was logged in ~/.cpanreporter/reports-sent.db.
#####
$ tail -n 2 ~/.cpanreporter/reports-sent.db
test PASS Module-CPANTS-Analyse-1.03 (perl-5.43.12) amd64-freebsd-thread-multi 14.4-release-p3
test PASS Module-CPANTS-Analyse-1.03 (perl-5.44.0) amd64-freebsd-thread-multi 14.4-release-p3
#####
At that point I made my complaint public on irc.perl.org#cpantesters-discuss
#####
08:09:58 < kid51> Just now it took me 3 tries to get a report on new release of Module::CPANTS::Analyse accepted by metabase
#####
Shortly thereafter, however, I went to https://fast2-matrix.cpantesters.org/?dist=Module-CPANTS-Analyse;os=freebsd;reports=1 and saw that *all four* of my CPANtesters runs had entered the metabase!  (The chronologically first and third entries are the ones I thought were transmission failures.)

5. Later in the day, I had occasion to do more CPANtesting, some on this FreeBSD-14 machine, some on my Debian 13 Linux Linode.  On the FreeBSD machine I spied this error output while testing Cache-File-Simple-0.1 and other distros against perl-5.44.0-RC1:
CPAN::Reporter: preparing a CPAN Testers report for Cache-File-Simple-0.1
CPAN::Reporter: sending test report with 'pass' via Metabase
CPAN::Reporter: Test::Reporter: error from 'Test::Reporter::Transport::Metabase:'
fact submission failed: first byte timeout at /home/jkeenan/testing/perl-5.44.0-RC1/lib/site_perl/5.44.0/Metabase/Client/Simple.pm line 129.
	Metabase::Client::Simple::submit_fact(Metabase::Client::Simple=HASH(0x32d19e5b16a8), CPAN::Testers::Report=HASH(0x32d19e611840)) called at /home/jkeenan/testing/perl-5.44.0-RC1/lib/site_perl/5.44.0/Test/Reporter/Transport/Metabase.pm line 122
...
[snip]
....
	App::Cpan::run("App::Cpan", "Time::Nanos", "Template::Sluz", "Encode::Bijou64", "Random::Simple", "ULID::Tiny", "Dump::Krumo", "Date::Parse::Modern", ...) called at ./bin/cpan line 18
#####
There was no entry logged for this at ~/.cpantesters/reports-sent.db -- but https://fast2-matrix.cpantesters.org/?dist=Cache-File-Simple;perl=5.44.0;reports=1 shows that a report *did* make it to the web:  https://www.cpantesters.org/cpan/report/7eece76a-6fcb-11f1-a0a2-869af8e95ecd.  (I did not try to re-test this distro.)

6. Meanwhile, over on Linux, I spotted this error output while testing ULID-Tiny-1.0.0 and other distros against perl-5.44.0-RC1:
#####
fact submission failed: first byte timeout at /home/jkeenan/testing/perl-5.44.0-RC1/lib/site_perl/5.44.0/Metabase/Client/Simple.pm line 129.
...
[snip]
....
	App::Cpan::run("App::Cpan", "Time::Nanos", "Template::Sluz", "Encode::Bijou64", "Random::Simple", "ULID::Tiny", "Dump::Krumo", "Date::Parse::Modern", ...) called at ./bin/cpan line 18
#####
There was no entry logged for this at ~/.cpantesters/reports-sent.db -- but https://fast2-matrix.cpantesters.org/?dist=ULID-Tiny shows that a report did make it into the metabase:  https://www.cpantesters.org/cpan/report/7f19d84c-6fcb-11f1-8a94-c9fdbafadc03.  (I did not try to re-test this distro.)

As stated above, I have always assumed that if I get error output beginning with "fact submission failed: first byte timeout", the report for that particular distro did not get transmitted and is utterly lost.  Has the behavior of the metabase changed?  Or was my assumption wrong from the start?

-- 
  James Keenan
  [email protected]