Re: [docs] [PATCH v3 5/5] tools: Add check-confusables pre-commit hook

--=-lsRMPmgQLcxbRvsnHmxg
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, 2026-07-28 at 19:40 +0300, Niko Mauno wrote:
>=20
> On 7/28/26 1:41 PM, Paul Barker wrote:
> > On Tue, 2026-07-28 at 12:07 +0300, Niko Mauno via lists.yoctoproject.or=
g
> > wrote:
> > > From: Niko Mauno <[email protected]>
> > >=20
> > > Add a check-confusables script, in the same fashion as
> > > check-glossaries, that scans the documentation .rst sources for
> > > non-ASCII "confusable" characters (curly quotes, en/em dashes,
> > > non-breaking and zero-width spaces, etc.) and reports each occurrence
> > > with its location and suggested ASCII replacement, exiting non-zero i=
f
> > > any are found. This guards against the class of breakage fixed in the
> > > preceding commit, e.g. curly quotes causing recipe ParseErrors.
> > >=20
> > > Legitimate non-ASCII such as box-drawing characters used in directory
> > > trees, accented letters in contributor names and CJK characters are
> > > intentionally left untouched. No-break spaces are likewise tolerated
> > > on lines containing box-drawing characters, since the tree command
> > > emits them as indentation in directory listings.
> > >=20
> > > Wire it up both as a local pre-commit hook, which checks the changed
> > > files, and in the Makefile "checks" target, which scans the whole
> > > tree, alongside check-glossaries.
> > >=20
> > > Suggested-by: Quentin Schulz <[email protected]>
> > > Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
> > > Signed-off-by: Niko Mauno <[email protected]>
> >=20
> > Hi Niko,
> >=20
> > Are there any existing tools we can use instead of implementing this
> > ourselves? Perhaps https://pypi.org/project/confusables/ ?
> >=20
> > Some comments below if we do want to merge our own implementation of
> > this...
> >=20
>=20
> Hi Paul,
>=20
> Thank you for these observations. It seems that confusables (and its=20
> alternatives) would not directly resolve the exact problems that=20
> check-confusables script addresses, therefore retained the=20
> dependency-free approach in v4 which I just submitted. It addresses the=
=20
> missing copyright/license header, and changes the literal glyphs to \u=
=20
> escape sequences.
>=20
> Also the topmost commit message now contains a segment explaining why=20
> the "own short table" is used instead of depending on a 3rd party package=
.

Hi Niko,

Thanks for following up. If existing Python packages don't meet our
needs then I agree it makes sense to carry our own curated list.

Best regards,

--=20
Paul Barker


--=-lsRMPmgQLcxbRvsnHmxg
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iIcEABYKAC8WIQSzjPXf5Y1BDWhU2iCrY1Tsnbr0bgUCamjdMBEccGF1bEBwYmFy
a2VyLmRldgAKCRCrY1Tsnbr0bjLoAQCTMsDtS9anOaFFNuc0+7RPoZaZWl+sWMop
9n812uYqiwD/Zea5IYbJljd+l3X1pzKUkpB+UkeyZAGO85n5rhBz7g4=
=D8Wl
-----END PGP SIGNATURE-----

--=-lsRMPmgQLcxbRvsnHmxg--