Re: BBC testing from upriver downwards
Todd Rinaldo <[email protected]> Tue, 21 Mar 2017 11:17:58 -0500
| Newsgroups | gmane.comp.lang.perl.quality-assurance |
|---|---|
| Message-ID | <[email protected]> |
--Apple-Mail=_502B9B88-7F64-4686-B868-32174622F866 Content-Type: multipart/alternative; boundary="Apple-Mail=_8BEABE1E-E25E-4FB4-9C70-A9F9D97E4ABA" --Apple-Mail=_8BEABE1E-E25E-4FB4-9C70-A9F9D97E4ABA Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Mar 17, 2017, at 4:32 PM, James E Keenan <[email protected]> wrote: >=20 > perl-5.25.11 will be released on Monday March 20. Since this will be = the first monthly dev release to reflect the banishment of '.' from the = default @INC, it is the first monthly release in which we can assess the = effect of that change on CPAN. >=20 > Up until now we've mainly relied on Andreas and Slaven's CPAN tester = reports in this regard. But we have a problem. We're having problems = with cpantesters.org such that, while we rapidly get PASS/FAIL = indicators for a given distro on a given platform on a given version of = perl at fast-matrix.cpantesters.org, we are having big delays at getting = the full report of a test failure at matrix.cpantesters.org. Currently, = 'matrix' is behind 'fast-matrix' by ... more than a few days. >=20 > I know that Doug Bell++ and others are examining this problem, but I = would like to try some modest workarounds. What I would like to do is: >=20 > 1. Compose a list of CPAN distros starting with those farthest up = river, i.e., distros that only depend on the perl 5 core. Within that = set of distros I'd like to order them from most reverse dependencies to = fewest. Then go down river from there. >=20 > 2. Get an up-to-date minicpan. >=20 > 3. Use that minicpan as the source of tests of the CPAN distros. >=20 > 4. Run something like a full CPAN test for perl 5.25.11 -- but be able = to cut that off at either a certain number of distros -- e.g., the 5000 = farthest upriver -- or at a certain number of level below the core = itself. > (a) I would like the location where the tests are run, the version of = 'cpan', 'cpaniminus', etc. to be completely distinct from whatever my = "usual" procedure is on a given platform -- that so I can just blow away = everything I've done once I'm through. >=20 > 5. Capture reports of test failures so that we can identify their = cause -- e.g., is this failure due to: >=20 I don't know if this would be helpful, but I've been tracking all of the = modules I've been reporting here. Feel free to scribble there if you = like so we don't duplicate effort and reports. https://github.com/toddr/perl/issues/14 = <https://github.com/toddr/perl/issues/14> Todd --Apple-Mail=_8BEABE1E-E25E-4FB4-9C70-A9F9D97E4ABA Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><br class=3D""><div><blockquote type=3D"cite" class=3D""><div = class=3D"">On Mar 17, 2017, at 4:32 PM, James E Keenan <<a = href=3D"mailto:[email protected]" class=3D"">[email protected]</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div = class=3D"">perl-5.25.11 will be released on Monday March 20. Since = this will be the first monthly dev release to reflect the banishment of = '.' from the default @INC, it is the first monthly release in which we = can assess the effect of that change on CPAN.<br class=3D""><br = class=3D"">Up until now we've mainly relied on Andreas and Slaven's CPAN = tester reports in this regard. But we have a problem. We're = having problems with <a href=3D"http://cpantesters.org" = class=3D"">cpantesters.org</a> such that, while we rapidly get PASS/FAIL = indicators for a given distro on a given platform on a given version of = perl at <a href=3D"http://fast-matrix.cpantesters.org" = class=3D"">fast-matrix.cpantesters.org</a>, we are having big delays at = getting the full report of a test failure at <a = href=3D"http://matrix.cpantesters.org" = class=3D"">matrix.cpantesters.org</a>. Currently, 'matrix' is = behind 'fast-matrix' by ... more than a few days.<br class=3D""><br = class=3D"">I know that Doug Bell++ and others are examining this = problem, but I would like to try some modest workarounds. What I = would like to do is:<br class=3D""><br class=3D"">1. Compose a list of = CPAN distros starting with those farthest up river, i.e., distros that = only depend on the perl 5 core. Within that set of distros I'd = like to order them from most reverse dependencies to fewest. Then = go down river from there.<br class=3D""><br class=3D"">2. Get an = up-to-date minicpan.<br class=3D""><br class=3D"">3. Use that minicpan = as the source of tests of the CPAN distros.<br class=3D""><br = class=3D"">4. Run something like a full CPAN test for perl 5.25.11 -- = but be able to cut that off at either a certain number of distros -- = e.g., the 5000 farthest upriver -- or at a certain number of level below = the core itself.<br class=3D"">(a) I would like the location where the = tests are run, the version of 'cpan', 'cpaniminus', etc. to be = completely distinct from whatever my "usual" procedure is on a given = platform -- that so I can just blow away everything I've done once I'm = through.<br class=3D""><br class=3D"">5. Capture reports of test = failures so that we can identify their cause -- e.g., is this failure = due to:<br class=3D""><br class=3D""></div></div></blockquote><br = class=3D""></div><div>I don't know if this would be helpful, but I've = been tracking all of the modules I've been reporting here. Feel free to = scribble there if you like so we don't duplicate effort and = reports.</div><div><br class=3D""></div><div><a = href=3D"https://github.com/toddr/perl/issues/14" = class=3D"">https://github.com/toddr/perl/issues/14</a></div><div = class=3D""><br class=3D""></div><div class=3D"">Todd</div><div = class=3D""><br class=3D""></div></body></html>= --Apple-Mail=_8BEABE1E-E25E-4FB4-9C70-A9F9D97E4ABA-- --Apple-Mail=_502B9B88-7F64-4686-B868-32174622F866 Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJ7jCCBK8w ggOXoAMCAQICEQDgI8sVEoNTia1hbnpUZ2shMA0GCSqGSIb3DQEBCwUAMG8xCzAJBgNVBAYTAlNF MRQwEgYDVQQKEwtBZGRUcnVzdCBBQjEmMCQGA1UECxMdQWRkVHJ1c3QgRXh0ZXJuYWwgVFRQIE5l dHdvcmsxIjAgBgNVBAMTGUFkZFRydXN0IEV4dGVybmFsIENBIFJvb3QwHhcNMTQxMjIyMDAwMDAw WhcNMjAwNTMwMTA0ODM4WjCBmzELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hl c3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RPIENBIExpbWl0ZWQxQTA/BgNV BAMTOENPTU9ETyBTSEEtMjU2IENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWls IENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAibEN2npTGU5wUh28VqYGJre4SeCW 51Gr8fBaE0kVo7SMG2C8elFCp3mMpCLfF2FOkdV2IwoU00oCf7YdCYBupQQ92bq7Fv6hh6kuQ1JD FnyvMlDIpk9a6QjYz5MlnHuI6DBk5qT4VoD9KiQUMxeZrETlaYujRgZLwjPU6UCfBrCxrJNAubUI kzqcKlOjENs9IGE8VQOO2U52JQIhKfqjfHF2T+7hX4Hp+1SA28N7NVK3hN4iPSwwLTF/Wb1SN7Az aS1D6/rWpfGXd2dRjNnuJ+u8pQc4doykqTj/34z1A6xJvsr3c5k6DzKrnJU6Ez0ORjpXdGFQvsZA P8vk4p+iIQIDAQABo4IBFzCCARMwHwYDVR0jBBgwFoAUrb2YejS0Jvf6xCZU7wO94CTLVBowHQYD VR0OBBYEFJJha4LhoqCqT+xn8cKj97SAAMHsMA4GA1UdDwEB/wQEAwIBhjASBgNVHRMBAf8ECDAG AQH/AgEAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEFBQcDBDARBgNVHSAECjAIMAYGBFUdIAAw RAYDVR0fBD0wOzA5oDegNYYzaHR0cDovL2NybC51c2VydHJ1c3QuY29tL0FkZFRydXN0RXh0ZXJu YWxDQVJvb3QuY3JsMDUGCCsGAQUFBwEBBCkwJzAlBggrBgEFBQcwAYYZaHR0cDovL29jc3AudXNl cnRydXN0LmNvbTANBgkqhkiG9w0BAQsFAAOCAQEAGypurFXBOquIxdjtzVXzqmthK8AJECOZD8Vm am+x9bS1d14PAmEA330F/hKzpICAAPz7HVtqcgIKQbwFusFY1SbC6tVNhPv+gpjPWBvjImOcUvi7 BTarfVil3qs7Y+Xa1XPv7OD7e+Kj//BCI5zKto1NPuRLGAOyqC3U2LtCS5BphRDbpjc06HvgARCl nMo6x59PiDRuimXQGoq7qdzKyjbR9PzCZCk1r9axp3ER0gNDsY8+muyeMlP0dpLKhjQHuSzK5hxK 2JkNwYbikJL7WkJqIyEQ6WXH9dW7fuqMhSACYurROgcsWcWZM/I4ieW26RZ6H3kU9koQGib6fIr7 mzCCBTcwggQfoAMCAQICEFqfByjLUeuvrntkEQDjeFUwDQYJKoZIhvcNAQELBQAwgZsxCzAJBgNV BAYTAkdCMRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAY BgNVBAoTEUNPTU9ETyBDQSBMaW1pdGVkMUEwPwYDVQQDEzhDT01PRE8gU0hBLTI1NiBDbGllbnQg QXV0aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQTAeFw0xNzAyMDIwMDAwMDBaFw0xODAy MDIyMzU5NTlaMCExHzAdBgkqhkiG9w0BCQEWEHRvZGRyQGNwYW5lbC5uZXQwggEiMA0GCSqGSIb3 DQEBAQUAA4IBDwAwggEKAoIBAQCYYvj69schz61JbDrv17YWSc12e/pa34DNMehovIhPTylKDU36 CHsZQHX4R+AeMVAnZhVdkeO7e5BXArUwLTtQr7o4Da2uYEPnPth9AZpFBZQlJZ7inSDob7fIFKhs OmiqtK2wT3/3RBSIJvtwlg1BmCnWa05JHnuybVIegHf2uvnlYR+pHQllN+KFG0Pasks+Eo2lO+5c l5W/fbUSY+lcHZf0WoE4Abv/MEkPfiacuySpJadXfEQxEWwglQjv2qWNMuf6pTWElXNsFCoRbNZp u5qVHdMwxV2azI9rGpRj6EfBgXDaQvDbJFIVCv0/JRSWj6jdlxyUW29XINYidbO/AgMBAAGjggHu MIIB6jAfBgNVHSMEGDAWgBSSYWuC4aKgqk/sZ/HCo/e0gADB7DAdBgNVHQ4EFgQUNgv8Ie5OhV7G e9y9VQzmDfBsfNQwDgYDVR0PAQH/BAQDAgWgMAwGA1UdEwEB/wQCMAAwIAYDVR0lBBkwFwYIKwYB BQUHAwQGCysGAQQBsjEBAwUCMBEGCWCGSAGG+EIBAQQEAwIFIDBGBgNVHSAEPzA9MDsGDCsGAQQB sjEBAgEBATArMCkGCCsGAQUFBwIBFh1odHRwczovL3NlY3VyZS5jb21vZG8ubmV0L0NQUzBdBgNV HR8EVjBUMFKgUKBOhkxodHRwOi8vY3JsLmNvbW9kb2NhLmNvbS9DT01PRE9TSEEyNTZDbGllbnRB dXRoZW50aWNhdGlvbmFuZFNlY3VyZUVtYWlsQ0EuY3JsMIGQBggrBgEFBQcBAQSBgzCBgDBYBggr BgEFBQcwAoZMaHR0cDovL2NydC5jb21vZG9jYS5jb20vQ09NT0RPU0hBMjU2Q2xpZW50QXV0aGVu dGljYXRpb25hbmRTZWN1cmVFbWFpbENBLmNydDAkBggrBgEFBQcwAYYYaHR0cDovL29jc3AuY29t b2RvY2EuY29tMBsGA1UdEQQUMBKBEHRvZGRyQGNwYW5lbC5uZXQwDQYJKoZIhvcNAQELBQADggEB ADZ/Wcb9bxQjk3gf1VCP3S+2zPF0LAL+m8ZZbRa4BAVsUiJSu0qqPWjbqOMxHqC7HyRcMiz5pHCf CIOoxDBAN0+cSnNoD6C2JnUs+29elG8WxCcSoiRTrAQk6EhU6fstyyRw7km8uTUjXponD61oMTmJ npwmlwbUwSOMkZfiinaK9RyEYcl7OyFN5xi+HuzeL07LGr9/tEt1oJNR2V9qqLS8oTasEaLdnh+Y t3/p6RVng6iLstW8Ma2AaLVY31Mcr+3/E4ISchFccH5L6KMPSyC33Gs9MDa68Ht54dfucH4X6fQq lGesvUKKFhKe3XeR4PLTXyYL7LgxqwJEU/rpIWMxggPDMIIDvwIBATCBsDCBmzELMAkGA1UEBhMC R0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgGA1UE ChMRQ09NT0RPIENBIExpbWl0ZWQxQTA/BgNVBAMTOENPTU9ETyBTSEEtMjU2IENsaWVudCBBdXRo ZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBAhBanwcoy1Hrr657ZBEA43hVMAkGBSsOAwIa BQCgggHnMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE3MDMyMTE2 MTc1OVowIwYJKoZIhvcNAQkEMRYEFJAvXsoGzFBCGpyG8kDQd8zQJKlSMIHBBgkrBgEEAYI3EAQx gbMwgbAwgZsxCzAJBgNVBAYTAkdCMRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNV BAcTB1NhbGZvcmQxGjAYBgNVBAoTEUNPTU9ETyBDQSBMaW1pdGVkMUEwPwYDVQQDEzhDT01PRE8g U0hBLTI1NiBDbGllbnQgQXV0aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQQIQWp8HKMtR 66+ue2QRAON4VTCBwwYLKoZIhvcNAQkQAgsxgbOggbAwgZsxCzAJBgNVBAYTAkdCMRswGQYDVQQI ExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAYBgNVBAoTEUNPTU9ETyBD QSBMaW1pdGVkMUEwPwYDVQQDEzhDT01PRE8gU0hBLTI1NiBDbGllbnQgQXV0aGVudGljYXRpb24g YW5kIFNlY3VyZSBFbWFpbCBDQQIQWp8HKMtR66+ue2QRAON4VTANBgkqhkiG9w0BAQEFAASCAQAw 62SfmtchjD8HkvSnKchVCTJ5vnynJtBrR2yrgbMQS7/jmolYmcMyZPfoWTi0iR82rAsKlobzZacv uVx9oMk1asxz05a+bbkCEckb+HliYRd9rRKLQxYx22t0fvVimFIJcGvbnzKK1Mm0fRzugfzvJOP1 38pxbbsJ0QAnjnUhuUrI1aDHktdtIMQ4JO8zMyo6hrssDjxNWwQUVokahtK7IB8+5ZxokWYr2wFQ M4x3NrE2wbWR4mTDJMkgMziyz9SyQ/4oZMUk7rsqxtX6gCiBFrLfV86XbNkjey26WBoksnreUxlq 5LUzsp0oDL7eCtHdlJ3lQyWP80rxI5lRdgWqAAAAAAAA --Apple-Mail=_502B9B88-7F64-4686-B868-32174622F866--