proj/portage:master commit in: src/, lib/portage/tests/dep/, lib/portage/dep/
"Matt Turner" <[email protected]>
| Newsgroups | gmane.linux.gentoo.cvs |
|---|---|
| Message-ID | <1785963929.677b8cb790cd297115ee8ff3fd91246b1e59958e.mattst88@gentoo> |
commit: 677b8cb790cd297115ee8ff3fd91246b1e59958e
Author: Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Sun Aug 2 20:21:02 2026 +0000
Commit: Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed Aug 5 21:05:29 2026 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=677b8cb7
dep: add C fast path for Atom construction
Atom.__init__ runs a large combined regex plus slot/use sub-parsing on
every direct construction (~550k times in a profiled @world resolve).
Add a C fast path: a new _parser.scan_atom(s) entry parses a single
atom, and Atom.__init__ uses it for the common case (no wildcards, no
injected _use, no is_valid_flag, no virtual-expansion original, EAPI
whose grammar the scanner implements), falling back to the regex path on
any ValueError.
scan_atom is made byte-exact with the regex atom grammar by adding the
validations the low-level scanner lacked:
- a version requires an operator ("cat/pkg-1" is invalid),
- a package name must not end in "-<version>", where the version may
include a "-rN" revision ("<cat/bar-2-0", "=cat/bar-1-r1-1-r1"),
- a trailing "*" (the "=*" glob) is only valid with the "=" operator,
- category and slot/sub-slot names must start with [A-Za-z0-9_] (not
"+"),
- repo specs and build-ids are not handled, so the
whole-string-consumed check rejects them and the caller falls back.
scan_slot() needed tightening for the same reason. It accepted three
forms the regex rejects:
cat/pkg:0/* cat/pkg:0/= cat/pkg:0=/53
":=" and ":*" are whole slot deps, not sub-slots, and the "=" operator
only ever comes last. Scan the sub-slot as a plain name and move the
trailing "=" after it, so "0=/53" stops at "0=" and the caller rejects
the atom on the leftover "/53". parse_slot_raw() loses the branches that
existed only to take those forms apart.
USE deps are built from the raw tokens via _use_dep exactly as the regex
path does, so conflicting-flag validation is preserved.
The scalar fields are copied by _c_fill_atom_fields(), shared with the
use_reduce fast path's _c_atom_from_c(), so the two cannot drift.
A differential fuzz over ~1.1M atoms (real md5-cache atoms plus
adversarial strings, across EAPIs and wildcard/repo/build-id flag
variants) found no differences in any Atom field, str(), without_use, or
raised exception. test_parser under ASan+UBSan (127 tests) clean.
Profiled cProfile emerge -uDNp @world: the fast path serves 288281 of
779663 Atom.__init__ calls (37%); Atom.__init__ 12.30s -> 10.68s cum.
@world merge list unchanged.
Adds TestScanAtom, asserting the C fast path is byte-exact with the
regex path for valid atoms and rejects the same invalid ones, and that
repo specs and build-ids fall back to the regex path.
Adds TestScanAtomNameGrammar, a category/package/version/revision/
suffix corpus adapted from pkgcore's tests/ebuild/test_cpv.py, plus slot
grammar and truncated-atom cases. Every string is asserted for parity
against the regex path rather than for a hardcoded verdict, so it cannot
encode pkgcore's rules where they differ from portage's. The shared
helpers move to _AtomParityMixin.
Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>
lib/portage/dep/__init__.py | 125 ++++++++++++---
lib/portage/tests/dep/test_c_parser.py | 270 +++++++++++++++++++++++++++++++++
src/dep_parser.c | 91 ++++++-----
src/dep_parser_core.c | 59 ++++---
src/test_parser.c | 9 ++
5 files changed, 479 insertions(+), 75 deletions(-)
diff --git a/lib/portage/dep/__init__.py b/lib/portage/dep/__init__.py
index ef345e807..1556c1a62 100644
--- a/lib/portage/dep/__init__.py
+++ b/lib/portage/dep/__init__.py
@@ -68,34 +68,46 @@ if TYPE_CHECKING:
from _emerge.Package import Package
-def _c_atom_from_c(catom, eapi, eapi_attrs, uselist, matchall):
- """Construct a portage.dep.Atom from a _parser.Atom, bypassing __init__ regex."""
- a = Atom.__new__(Atom)
- a._string = str(catom)
- a._cp = catom.cp
- a._repo = None
- a._slot = catom.slot
- a._sub_slot = catom.sub_slot
- a._slot_operator = catom.slot_operator
- ver = catom.version
+def _c_fill_atom_fields(atom, catom, eapi):
+ """Copy the scalar fields of a _parser.Atom onto a portage.dep.Atom.
+
+ Everything except _string, _use and _unevaluated_atom, which the two
+ callers set differently. The C scanner reports the "=*" glob as operator
+ "=" with a trailing "*" on the version, so undo that here.
+ """
op = catom.operator
+ ver = catom.version
+ cpv = catom.cpv
if op == "=" and ver is not None and ver.endswith("*"):
op = "=*"
ver = ver[:-1]
- a._operator = op
- a._version = ver
- a._cpv = catom.cpv[:-1] if op == "=*" else catom.cpv
- a._eapi = eapi
- a._extended_syntax = False
- a._build_id = None
- a._unevaluated_atom = a
- a._orig_atom = None
+ cpv = cpv[:-1]
+ atom._cp = catom.cp
+ atom._cpv = cpv
+ atom._version = ver
+ atom._operator = op
+ atom._slot = catom.slot
+ atom._sub_slot = catom.sub_slot
+ atom._slot_operator = catom.slot_operator
+ atom._repo = None
+ atom._eapi = eapi
+ atom._extended_syntax = False
+ atom._build_id = None
+ atom._orig_atom = None
blocker_str = catom.blocker
- a._blocker_obj = (
+ atom._blocker_obj = (
Atom._blocker(forbid_overlap=blocker_str == "!!")
if blocker_str is not None
else None
)
+
+
+def _c_atom_from_c(catom, eapi, eapi_attrs, uselist, matchall):
+ """Construct a portage.dep.Atom from a _parser.Atom, bypassing __init__ regex."""
+ a = Atom.__new__(Atom)
+ a._string = str(catom)
+ _c_fill_atom_fields(a, catom, eapi)
+ a._unevaluated_atom = a
use_tokens = catom.use
if use_tokens is not None:
en, dis, miss_en, miss_dis, cond, req = _c_dep_parser.classify_use_deps(
@@ -1814,6 +1826,59 @@ class Atom:
def __init__(self, forbid_overlap=False):
self.overlap = self._overlap(forbid=forbid_overlap)
+ def _c_fast_init(self, catom, eapi, eapi_attrs, unevaluated_atom):
+ """Populate this Atom from a C scan_atom result.
+
+ Raises InvalidAtom/TypeError for the same cases as the regex path.
+ """
+ use_tokens = catom.use
+ if use_tokens is not None:
+ # _use_dep validates conflicting flags, same as the regex path.
+ self._use = _use_dep(list(use_tokens), eapi_attrs)
+ else:
+ self._use = None
+
+ _c_fill_atom_fields(self, catom, eapi)
+ self._unevaluated_atom = unevaluated_atom if unevaluated_atom else self
+
+ if eapi is None:
+ return
+
+ if not isinstance(eapi, str):
+ raise TypeError(
+ f"expected eapi argument of {str}, got {type(eapi)}: {eapi}"
+ )
+
+ if self._slot and not eapi_attrs.slot_deps:
+ raise InvalidAtom(
+ f"Slot deps are not allowed in EAPI {eapi}: '{self}'",
+ category="EAPI.incompatible",
+ )
+
+ if self._use:
+ if not eapi_attrs.use_deps:
+ raise InvalidAtom(
+ f"Use deps are not allowed in EAPI {eapi}: '{self}'",
+ category="EAPI.incompatible",
+ )
+ if not eapi_attrs.use_dep_defaults and (
+ self._use.missing_enabled or self._use.missing_disabled
+ ):
+ raise InvalidAtom(
+ f"Use dep defaults are not allowed in EAPI {eapi}: '{self}'",
+ category="EAPI.incompatible",
+ )
+
+ if (
+ self._blocker_obj
+ and self._blocker_obj.overlap.forbid
+ and not eapi_attrs.strong_blocks
+ ):
+ raise InvalidAtom(
+ f"Strong blocks are not allowed in EAPI {eapi}: '{self}'",
+ category="EAPI.incompatible",
+ )
+
def __init__(
self,
s,
@@ -1853,6 +1918,28 @@ class Atom:
if allow_build_id is None:
allow_build_id = True
+ # Fast path: parse the atom in C for the common case. scan_atom is
+ # byte-exact with the regex path or raises ValueError (repo specs,
+ # build-ids, wildcards, anything invalid), in which case we fall
+ # through to the pure-Python regex path below. Wildcards, an injected
+ # _use, flag validation, and virtual-expansion originals are handled
+ # only by the regex path.
+ if (
+ _c_dep_parser is not None
+ and not allow_wildcard
+ and _use is None
+ and is_valid_flag is None
+ and orig_atom is None
+ and (eapi is None or eapi_attrs.slot_operator)
+ ):
+ try:
+ catom = _c_dep_parser.scan_atom(s)
+ except ValueError:
+ catom = None
+ if catom is not None:
+ self._c_fast_init(catom, eapi, eapi_attrs, unevaluated_atom)
+ return
+
if s[:1] == "!":
blocker = self._blocker(forbid_overlap=s[1:2] == "!")
if blocker.overlap.forbid:
diff --git a/lib/portage/tests/dep/test_c_parser.py b/lib/portage/tests/dep/test_c_parser.py
index 3e91bd809..62b6aaec6 100644
--- a/lib/portage/tests/dep/test_c_parser.py
+++ b/lib/portage/tests/dep/test_c_parser.py
@@ -906,6 +906,276 @@ class TestLongAtoms(TestCase):
)
+class _AtomParityMixin:
+ """Helpers for asserting that _parser.scan_atom + Atom._c_fast_init agrees
+ with the pure-Python regex path, both on what it accepts and on the fields
+ it produces."""
+
+ def setUp(self):
+ if _orig_c_dep_parser is None:
+ self.skipTest("_parser extension not available")
+
+ def _both(self, s, **kw):
+ with _use_c_parser(False):
+ try:
+ py = Atom(s, **kw)
+ py_exc = None
+ except Exception as e:
+ py, py_exc = None, type(e)
+ try:
+ c = Atom(s, **kw)
+ c_exc = None
+ except Exception as e:
+ c, c_exc = None, type(e)
+ return (py, py_exc), (c, c_exc)
+
+ def _assert_same(self, s, **kw):
+ (py, pe), (c, ce) = self._both(s, **kw)
+ self.assertEqual(pe, ce, f"{s!r}: exception {pe} vs {ce}")
+ if pe is None:
+ for attr in (
+ "_cp",
+ "_cpv",
+ "_version",
+ "_operator",
+ "_slot",
+ "_sub_slot",
+ "_slot_operator",
+ "_repo",
+ "_build_id",
+ "_extended_syntax",
+ ):
+ self.assertEqual(getattr(py, attr), getattr(c, attr), f"{s!r}: {attr}")
+ self.assertEqual(str(py._use or ""), str(c._use or ""), f"{s!r}: use")
+ self.assertEqual(str(py), str(c))
+
+
+class TestScanAtom(_AtomParityMixin, TestCase):
+ """Test that _parser.scan_atom + Atom._c_fast_init matches the pure-Python
+ regex path for both valid atoms and invalid ones."""
+
+ def test_valid_atoms(self):
+ for s in (
+ "sys-apps/portage",
+ "=sys-apps/portage-2.1",
+ ">=dev-libs/foo-1.2.3-r1:0/1=[a,-b,c?,!d?,e=]",
+ "!!media-libs/x:2",
+ "~cat/pkg-1.0",
+ "cat/pkg:0/1=",
+ "dev-libs/gtk+",
+ ):
+ with self.subTest(s=s):
+ self._assert_same(s, eapi="8")
+ self._assert_same(s, eapi=None)
+
+ def test_glob_only_with_equals(self):
+ self._assert_same("=cat/pkg-1.2*", eapi="8")
+ self._assert_same(">=cat/pkg-1.2*", eapi="8") # invalid -> both reject
+ self._assert_same("<cat/pkg-1*", eapi="8")
+
+ def test_version_requires_operator(self):
+ self._assert_same("cat/pkg-1", eapi="8") # invalid
+ self._assert_same("cat/pkg-1.2.3", eapi="8") # invalid
+
+ def test_name_must_not_end_in_version(self):
+ for s in ("<cat/bar-2-0", "=foo/bar-1-r1-1-r1", "=cat/libc-2-9999"):
+ with self.subTest(s=s):
+ self._assert_same(s, eapi="8") # invalid, both reject
+
+ def test_leading_plus_rejected(self):
+ for s in ("+cat/pkg", "cat/pkg:+slot", "cat/pkg:0/+sub"):
+ with self.subTest(s=s):
+ self._assert_same(s, eapi="8")
+
+ def test_conflicting_use_rejected(self):
+ self._assert_same("cat/pkg[a(+),-a]", eapi="8")
+ self._assert_same("cat/pkg[a,a]", eapi="8")
+
+ def test_repo_and_build_id_fall_back(self):
+ # scan_atom rejects these; the regex path handles them when allowed.
+ self._assert_same("cat/pkg::gentoo", eapi=None, allow_repo=True)
+ self._assert_same("=cat/pkg-1-3", eapi=None, allow_build_id=True)
+
+ def test_eapi_incompatibility(self):
+ self._assert_same("cat/pkg:0", eapi="0") # slot deps invalid in EAPI 0
+ self._assert_same("cat/pkg[a]", eapi="1") # use deps invalid in EAPI 1
+ self._assert_same("cat/pkg[a(+)]", eapi="4") # defaults invalid in EAPI 4
+
+
+class TestScanAtomNameGrammar(_AtomParityMixin, TestCase):
+ """Category, package-name, version and revision edge cases, adapted from
+ pkgcore's tests/ebuild/test_cpv.py.
+
+ The corpus only supplies awkward shapes; it does not assert which of them
+ are valid. Portage's own regex path is the reference, and every string is
+ checked for parity against it, so a C scanner that is stricter or laxer
+ than portage anywhere in this grammar fails."""
+
+ # Names that are legal per PMS 3.1.1/3.1.2, including the ones that look
+ # like they should not be: a bare "_" category, a category with a dot in
+ # it, and package names ending in hyphens or in a hyphen-digit sequence
+ # that is not a version.
+ GOOD_CATS = (
+ "dev-util",
+ "dev+",
+ "DEV-UTIL+",
+ "aaa0",
+ "aaa-0",
+ "multi--hyphen",
+ "_dev",
+ "_",
+ "cross-hppa2.0-unknown-linux-gnu",
+ )
+ BAD_CATS = (
+ "",
+ ".reject",
+ " reject",
+ "-",
+ "+",
+ "dev-util ",
+ "multi/blah/depth",
+ "multi//depth",
+ )
+ GOOD_PKGS = (
+ "diffball",
+ "a9",
+ "a9+",
+ "a-100dpi",
+ "diff-mode-",
+ "multi--hyphen",
+ "timidity--",
+ "frob---",
+ "diffball-9-",
+ "7z",
+ "xf86-video-r128",
+ "emacs-cvs",
+ )
+ # "diffball-9" and "bar-11-r3" are rejected because an unversioned atom's
+ # name may not end in something that parses as a version.
+ BAD_PKGS = (
+ "diffball ",
+ "diffball-9",
+ "a-3D",
+ "-df",
+ "+dfa",
+ "timidity--9f",
+ "ormaybe---13_beta",
+ "bar-11-r3",
+ )
+
+ GOOD_VERS = ("1", "2.3.4", "2.3.4a", "02.3", "2.03", "3d")
+ BAD_VERS = ("2.3a.4", "2.a.3", "2.3_", "2.3 ", "2.3.", "cvs.2", "3D")
+ GOOD_REVS = ("", "-r0", "-r1", "-r300", "-r1000000000000000000")
+ BAD_REVS = ("-r", "-ra", "-R1")
+
+ SIMPLE_SUFS = ("_alpha", "_beta", "_pre", "_p", "_rc")
+ GOOD_SUFS = SIMPLE_SUFS + tuple(f"{x}{n}" for n, x in enumerate(SIMPLE_SUFS))
+ BAD_SUFS = ("_a", "_9", "_") + tuple(f"{x} " for x in SIMPLE_SUFS)
+
+ def test_category_grammar(self):
+ for cat in self.GOOD_CATS + self.BAD_CATS:
+ with self.subTest(cat=cat):
+ self._assert_same(f"{cat}/diffball", eapi="8")
+
+ def test_package_name_grammar(self):
+ for pkg in self.GOOD_PKGS + self.BAD_PKGS:
+ with self.subTest(pkg=pkg):
+ self._assert_same(f"dev-util/{pkg}", eapi="8")
+
+ def test_category_package_matrix(self):
+ for cat in self.GOOD_CATS:
+ for pkg in self.GOOD_PKGS:
+ with self.subTest(cat=cat, pkg=pkg):
+ self._assert_same(f"{cat}/{pkg}", eapi="8")
+
+ def test_version_grammar(self):
+ for ver in self.GOOD_VERS + self.BAD_VERS:
+ with self.subTest(ver=ver):
+ self._assert_same(f"=dev-util/diffball-{ver}", eapi="8")
+ self._assert_same(f"~dev-util/diffball-{ver}", eapi="8")
+
+ def test_revision_grammar(self):
+ for rev in self.GOOD_REVS + self.BAD_REVS:
+ with self.subTest(rev=rev):
+ self._assert_same(f"=dev-util/diffball-2.3.4{rev}", eapi="8")
+
+ def test_version_suffix_grammar(self):
+ for suf in self.GOOD_SUFS + self.BAD_SUFS:
+ with self.subTest(suf=suf):
+ self._assert_same(f"=dev-util/diffball-1{suf}", eapi="8")
+ self._assert_same(f"=dev-util/diffball-1{suf}-r1", eapi="8")
+
+ def test_version_suffix_and_revision_matrix(self):
+ for ver in self.GOOD_VERS:
+ for rev in self.GOOD_REVS + self.BAD_REVS:
+ with self.subTest(ver=ver, rev=rev):
+ self._assert_same(f"=dev-util/diffball-{ver}{rev}", eapi="8")
+
+ def test_version_glob_grammar(self):
+ for ver in self.GOOD_VERS + self.BAD_VERS:
+ with self.subTest(ver=ver):
+ self._assert_same(f"=dev-util/diffball-{ver}*", eapi="8")
+
+ def test_package_name_containing_version_like_words(self):
+ # A hyphen-digit run only terminates the name if what follows it is a
+ # complete version, so these all stay part of the package name.
+ for s in (
+ "dev-util/diffball-blah-monkeys",
+ "bah/f-100dpi",
+ "dev-ut-asdf/emacs-cvs",
+ "bbb-9/foon",
+ "dev-util/foo-123-bar",
+ "app-text/foo-2abc",
+ "app-text/foo-2_bar",
+ ):
+ with self.subTest(s=s):
+ self._assert_same(s, eapi="8")
+
+ def test_slot_grammar(self):
+ # ":=" and ":*" are whole slot deps, not sub-slots, and the "="
+ # operator only ever comes last.
+ for s in (
+ "cat/pkg:0",
+ "cat/pkg:0/53",
+ "cat/pkg:0=",
+ "cat/pkg:0/53=",
+ "cat/pkg:=",
+ "cat/pkg:*",
+ "cat/pkg:my-slot_2.1/other+sub=",
+ "cat/pkg:0/*", # invalid
+ "cat/pkg:0/=", # invalid
+ "cat/pkg:0=/53", # invalid
+ "cat/pkg:0=/53=", # invalid
+ "cat/pkg:0/", # invalid
+ "cat/pkg:", # invalid
+ "cat/pkg:/53", # invalid
+ "cat/pkg:-slot", # invalid
+ "cat/pkg:0//53", # invalid
+ "cat/pkg:0/53/54", # invalid
+ "cat/pkg:*=", # invalid
+ "cat/pkg:=*", # invalid
+ ):
+ with self.subTest(s=s):
+ self._assert_same(s, eapi="8")
+
+ def test_truncated_atoms(self):
+ for s in ("cat/", "cat/pkg[", "cat/pkg[a", "cat/pkg[]", "cat", "/pkg", "/"):
+ with self.subTest(s=s):
+ self._assert_same(s, eapi="8")
+
+ def test_pathological_name(self):
+ # https://github.com/pkgcore/pkgcore/issues/463 and the oversized name
+ # from pkgcore's test_cpv.py, which also exercises the heap fallback in
+ # join_atom_string().
+ cat = "dev-java"
+ pkg = (
+ "log5j-777777777777777777777777777777777-777777777777777777"
+ "-7777777777777777777-7777777-7dev-q!7778"
+ )
+ self._assert_same(f"{cat}/{pkg}", eapi="8")
+ self._assert_same("cross-hppa2.0-unknown-linux-gnu/gcc", eapi="8")
+
+
class TestCParserMalformedGroups(TestCase):
"""Whitespace and delimiter errors in group syntax. Every one of these has
a distinct error path in scan_item()/scan_group_contents(), and each must
diff --git a/src/dep_parser.c b/src/dep_parser.c
index 95edd2ec8..191c4d56d 100644
--- a/src/dep_parser.c
+++ b/src/dep_parser.c
@@ -96,46 +96,25 @@ static void parse_slot_raw(const char *raw, int rlen,
return;
}
- const char *slash = memchr(raw, '/', rlen);
- char slot_op = 0;
-
- if (!slash) {
- int slen = rlen;
- if (slen > 0 && raw[slen - 1] == '=') {
- slot_op = '=';
- slen--;
- }
- *out_slot = PyUnicode_FromStringAndSize(raw, slen);
- *out_sub = Py_NewRef(Py_None);
+ /* Past the bare ":=" / ":*" forms above, the only operator is a trailing
+ * '=', and it always comes last -- what precedes it is "slot" or
+ * "slot/sub_slot". */
+ int len = rlen;
+ int slot_op = len > 0 && raw[len - 1] == '=';
+ if (slot_op)
+ len--;
+
+ const char *slash = memchr(raw, '/', len);
+ if (slash) {
+ *out_slot = PyUnicode_FromStringAndSize(raw, (int)(slash - raw));
+ *out_sub = PyUnicode_FromStringAndSize(slash + 1,
+ len - (int)(slash + 1 - raw));
} else {
- int main_len = (int)(slash - raw);
- if (main_len > 0 && raw[main_len - 1] == '=') {
- slot_op = '=';
- main_len--;
- }
- *out_slot = PyUnicode_FromStringAndSize(raw, main_len);
-
- const char *sub = slash + 1;
- int sub_len = rlen - (int)(sub - raw);
- if (sub_len == 1 && (sub[0] == '*' || sub[0] == '=')) {
- *out_sub = Py_NewRef(Py_None);
- slot_op = sub[0];
- } else {
- if (sub_len > 0 && sub[sub_len - 1] == '=') {
- slot_op = '=';
- sub_len--;
- }
- *out_sub = PyUnicode_FromStringAndSize(sub, sub_len);
- }
+ *out_slot = PyUnicode_FromStringAndSize(raw, len);
+ *out_sub = Py_NewRef(Py_None);
}
- if (slot_op == '=') {
- *out_op = Py_NewRef(interned.slot_op_eq);
- } else if (slot_op == '*') {
- *out_op = Py_NewRef(interned.slot_op_star);
- } else {
- *out_op = Py_NewRef(Py_None);
- }
+ *out_op = slot_op ? Py_NewRef(interned.slot_op_eq) : Py_NewRef(Py_None);
}
/* Split the raw text between '[' and ']' into one string per flag:
@@ -658,6 +637,32 @@ py_parse(UNUSED PyObject *self, PyObject *args, PyObject *kwargs)
return Py_NewRef(result);
}
+/*
+ * scan_atom(s) -> Atom
+ *
+ * Parse a single atom string; the whole string must be exactly one atom.
+ * Raises ValueError on anything the scanner does not fully consume -- in
+ * particular repository specs ("::repo") and build-ids, which it does not
+ * handle -- so the caller can fall back to the pure-Python regex path.
+ */
+static PyObject *
+py_scan_atom(UNUSED PyObject *self, PyObject *arg)
+{
+ Py_ssize_t n;
+ const char *s = PyUnicode_AsUTF8AndSize(arg, &n);
+ if (!s)
+ return NULL;
+
+ DepScanner p = { s, s + n, NULL };
+ AtomInfo info;
+ memset(&info, 0, sizeof(info));
+ if (!scan_atom(&p, &info) || p.cur != p.end) {
+ PyErr_SetString(PyExc_ValueError, p.err ? p.err : "invalid atom");
+ return NULL;
+ }
+ return build_atom_obj(&info, s, (int)n);
+}
+
static PyMethodDef methods[] = {
{
.ml_name = "parse",
@@ -671,6 +676,16 @@ static PyMethodDef methods[] = {
"Use conditionals are evaluated: an active one contributes a\n"
"sublist, an inactive one contributes nothing.",
},
+ {
+ .ml_name = "scan_atom",
+ .ml_meth = py_scan_atom,
+ .ml_flags = METH_O,
+ .ml_doc =
+ "scan_atom(s) -> Atom\n"
+ "Parse a string that must be exactly one atom. Raises ValueError\n"
+ "for anything the scanner does not fully consume, including repo\n"
+ "specs and build-ids, so the caller can fall back to the regex.",
+ },
{
.ml_name = "classify_use_deps",
.ml_meth = py_classify_use_deps,
@@ -679,7 +694,7 @@ static PyMethodDef methods[] = {
"classify_use_deps(tokens) -> tuple\n"
"Classify pre-split use-dep tokens into (enabled_fs, disabled_fs,\n"
"missing_enabled_fs, missing_disabled_fs, conditional_dict_or_None,\n"
- "required_fs). Raises ValueError if a token cannot be classified.",
+ "required_fs). Raises ValueError if a token cannot be classified."
},
{ NULL, NULL, 0, NULL },
};
diff --git a/src/dep_parser_core.c b/src/dep_parser_core.c
index 836b711e9..38aec32ac 100644
--- a/src/dep_parser_core.c
+++ b/src/dep_parser_core.c
@@ -135,7 +135,10 @@ int scan_version(DepScanner *p)
* optional '=' operator, or a bare ':=' / ':*'.
*
* "0" "myslot" "0/53" "0=" "0/53=" "=" "*" -> consumed
- * "/slot" "-slot" -> rejected
+ * "/slot" "-slot" "0/=" "0/*" "0=/53" -> rejected
+ *
+ * ':=' and ':*' are only the whole slot dep; they are not a sub-slot, and the
+ * '=' operator only ever comes last.
*
* Slot names share the category character set, except that the first
* character may not be '+'. */
@@ -151,34 +154,31 @@ int scan_slot(DepScanner *p)
return 1;
}
- if (!is_nw_char(*s))
+ /* PMS: a slot name's first character must be [A-Za-z0-9_] ('+' is a slot
+ * char only after the first position). */
+ if (!is_nw_char(*s) || *s == '+')
return 0;
s++;
while (s < p->end && is_slot_char(*s)) {
s++;
}
- if (s < p->end && *s == '=') {
- s++;
- }
if (s < p->end && *s == '/') {
s++;
- if (s < p->end && (*s == '*' || *s == '=')) {
- s++;
- } else if (s < p->end && is_nw_char(*s)) {
- s++;
- while (s < p->end && is_slot_char(*s)) {
- s++;
- }
- if (s < p->end && *s == '=') {
- s++;
- }
- } else {
+ if (s >= p->end || !is_nw_char(*s) || *s == '+')
return 0;
+
+ s++;
+ while (s < p->end && is_slot_char(*s)) {
+ s++;
}
}
+ if (s < p->end && *s == '=') {
+ s++;
+ }
+
p->cur = s;
return 1;
}
@@ -291,7 +291,9 @@ int scan_atom(DepScanner *p, AtomInfo *info)
/* category */
const char *cat = s;
- if (s >= p->end || !is_nw_char(*s))
+ /* PMS: the first character must be [A-Za-z0-9_]; '+' (a name-word char
+ * elsewhere) is not allowed to lead a category. */
+ if (s >= p->end || !is_nw_char(*s) || *s == '+')
goto fail;
s++;
@@ -410,9 +412,30 @@ after_pkgver:
p->cur++;
}
- if (op && !ver)
+ /* An operator requires a version and a version requires an operator:
+ * ">=cat/pkg" and a bare "cat/pkg-1" are both invalid atoms. */
+ if ((op != NULL) != (ver != NULL))
goto fail;
+ /* PMS: a package name must not end in a hyphen followed by a version.
+ * The trailing version may itself span a "-rN" revision, so check every
+ * '-' position: if the whole remainder after any '-' is a full version,
+ * the atom is invalid (e.g. "<cat/bar-2-0", "=cat/bar-1-r1-1-r1"). */
+ for (const char *t = pkg; t < pkg_end; t++) {
+ if (*t != '-')
+ continue;
+ DepScanner vt = { t + 1, pkg_end, NULL };
+ if (scan_version(&vt) && vt.cur == pkg_end)
+ goto fail;
+ }
+
+ /* A trailing '*' (the "=*" glob form) is only valid with the '='
+ * operator, e.g. "=cat/pkg-1.2*"; reject it with any other operator. */
+ if (ver && ver_len > 0 && ver[ver_len - 1] == '*' &&
+ !(op_len == 1 && op[0] == '=')) {
+ goto fail;
+ }
+
if (info) {
SPAN_SET(info, block);
SPAN_SET(info, op);
diff --git a/src/test_parser.c b/src/test_parser.c
index b6305aa47..486caadad 100644
--- a/src/test_parser.c
+++ b/src/test_parser.c
@@ -242,6 +242,15 @@ static void test_scan_slot(void)
{ "", 0, NULL },
{ "/slot", 0, NULL },
{ "-slot", 0, NULL },
+ { "+slot", 0, NULL },
+ /* ":=" and ":*" are the whole slot dep, never a sub-slot. */
+ { "0/*", 0, NULL },
+ { "0/=", 0, NULL },
+ { "0/", 0, NULL },
+ { "0/+sub", 0, NULL },
+ /* The '=' operator comes last, so "0=" is all that is consumed here
+ * and the caller rejects the atom on the leftover "/53". */
+ { "0=/53", 1, "0=" },
};
for (int i = 0; i < ARRAY_SIZE(cases); i++) {