[Biopython] (no subject)

--===============6092076544846909417==
Content-Type: multipart/alternative; boundary="00000000000086fbd60652f5de83"

--00000000000086fbd60652f5de83
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

*Subject:* Reproducibility and evaluation instability in clinical ML
pipelines using Biopython-based workflows
------------------------------

Hi Biopython team,

I=E2=80=99m reaching out as a user of Biopython in the context of biomedica=
l data
processing pipelines for clinical machine learning research.

While developing a reproducible ICU prediction pipeline using
MIMIC-IV-derived datasets (https://github.com/netanelcyber/PenuX), we
encountered an interesting observation that may be relevant to the broader
bioinformatics community.

Although Biopython is primarily used for sequence and molecular data
workflows, we found it useful as part of a broader preprocessing and
integration pipeline alongside clinical datasets and downstream ML models.
------------------------------
Observation (methodological, not tool-specific)

Across multiple modeling experiments, we observed that:

   - standard performance metrics (e.g., AUROC) remain relatively stable
   across implementations
   - however, model reliability varies significantly under:
      - temporal validation vs random splits
      - different preprocessing strategies
      - subgroup stratification and missing-data regimes

These effects appear to be *evaluation-design dependent rather than
model-dependent*, and raise broader questions about reproducibility in
biomedical ML pipelines.
------------------------------
Why this may be relevant to Biopython users

Even though Biopython is not directly responsible for clinical ML
evaluation, many real-world pipelines combine:

   - biological data processing (where Biopython is used)
   - clinical datasets (e.g., MIMIC-IV)
   - downstream predictive modeling

This creates a gap where upstream reproducibility (sequence/biological
processing) is strong, but downstream evaluation protocols may still
introduce instability.
------------------------------
Question to the community

I would be interested to hear whether others in the Biopython community
have encountered:

   - reproducibility issues when Biopython pipelines are integrated into
   larger ML systems
   - challenges in maintaining consistency across downstream evaluation
   setups
   - best practices for ensuring pipeline-level reproducibility beyond
   sequence processing itself

------------------------------
Context

Project reference (for reproducibility context only):
https://github.com/netanelcyber/PenuX
------------------------------

Thank you for your work on Biopython =E2=80=94 it remains a foundational to=
ol in
computational biology and bioinformatics workflows.

Best regards,

--00000000000086fbd60652f5de83
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><p><br></p>
<p><strong>Subject:</strong> Reproducibility and evaluation instability in =
clinical ML pipelines using Biopython-based workflows</p>
<hr>
<p>Hi Biopython team,</p>
<p>I=E2=80=99m reaching out as a user of Biopython in the context of biomed=
ical data processing pipelines for clinical machine learning research.</p>
<p>While developing a reproducible ICU prediction pipeline using MIMIC-IV-d=
erived datasets (<a href=3D"https://github.com/netanelcyber/PenuX">https://=
github.com/netanelcyber/PenuX</a>), we encountered an interesting observati=
on that may be relevant to the broader bioinformatics community.</p>
<p>Although Biopython is primarily used for sequence and molecular data wor=
kflows, we found it useful as part of a broader preprocessing and integrati=
on pipeline alongside clinical datasets and downstream ML models.</p>
<hr>
<h3>Observation (methodological, not tool-specific)</h3>
<p>Across multiple modeling experiments, we observed that:</p>
<ul><li>standard performance metrics (e.g., AUROC) remain relatively stable=
 across implementations</li><li>however, model reliability varies significa=
ntly under:
<ul><li>temporal validation vs random splits</li><li>different preprocessin=
g strategies</li><li>subgroup stratification and missing-data regimes</li><=
/ul>
</li></ul>
<p>These effects appear to be <strong>evaluation-design dependent rather th=
an model-dependent</strong>, and raise broader questions about reproducibil=
ity in biomedical ML pipelines.</p>
<hr>
<h3>Why this may be relevant to Biopython users</h3>
<p>Even though Biopython is not directly responsible for clinical ML evalua=
tion, many real-world pipelines combine:</p>
<ul><li>biological data processing (where Biopython is used)</li><li>clinic=
al datasets (e.g., MIMIC-IV)</li><li>downstream predictive modeling</li></u=
l>
<p>This creates a gap where upstream reproducibility (sequence/biological p=
rocessing) is strong, but downstream evaluation protocols may still introdu=
ce instability.</p>
<hr>
<h3>Question to the community</h3>
<p>I would be interested to hear whether others in the Biopython community =
have encountered:</p>
<ul><li>reproducibility issues when Biopython pipelines are integrated into=
 larger ML systems</li><li>challenges in maintaining consistency across dow=
nstream evaluation setups</li><li>best practices for ensuring pipeline-leve=
l reproducibility beyond sequence processing itself</li></ul>
<hr>
<h3>Context</h3>
<p>Project reference (for reproducibility context only):
<a href=3D"https://github.com/netanelcyber/PenuX">https://github.com/netane=
lcyber/PenuX</a></p>
<hr>
<p>Thank you for your work on Biopython =E2=80=94 it remains a foundational=
 tool in computational biology and bioinformatics workflows.</p>
<p>Best regards,</p></div>

--00000000000086fbd60652f5de83--

--===============6092076544846909417==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Biopython mailing list  -  [email protected]
https://mailman.open-bio.org/mailman/listinfo/biopython

--===============6092076544846909417==--