Re: [PATCH] make: add valid-for and valid-max target TTL support
Yair Lenga <[email protected]> Thu, 19 Feb 2026 05:54:10 -0500
| Newsgroups | gmane.comp.gnu.make.bugs |
|---|---|
| Message-ID | <CAK3_KpMjSUHKJv4EJ4bW10=9d=1Okx3xV=tfBfWB1j6kihK+Kg@mail.gmail.com> |
--00000000000082767c064b2b2046 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Britton, Thank you for you feedback - I agree 100% better to use existing syntax than to expand syntax. I did some more experiments with the implementation I've submitted, and I found that there are valid use cases for using valid-max (and valid-for) in both situations (with and without private). As for error handing (on invalid values, e,g valid-max=3D"last Monday", or other) - the implementation is aligned with current make behavior - errors and warnings are logged with the standard macros already available (DBS, DBF, ...) - with proper level (warning, info). This make the existing debugging, troubleshooting tools (-d, ...) works - no new options to learn. Yair. On Wed, Feb 18, 2026 at 10:38=E2=80=AFPM Britton Kerin <britton.kerin@gmail= .com> wrote: > On Tue, Feb 17, 2026 at 5:16=E2=80=AFPM Yair Lenga <[email protected]>= wrote: > > > > Hi Britton, > > > > Thanks for taking the time to read my proposal. Good point about > documentation, should have included more use case. See below for some > additional example. If the proposal will be accepted - I look into creati= ng > a documentation pages for "info make" for various use cases: > > > > Is it usable only for leaves ? > > - Short answer: It can be used also on non-leaves to limit rebuild, or > limit reuse of targets. In it's simplest form - it can be used on leaves > that (including leaf that depend on "PHONY" targets). > > > > Below are the main use cases where new features will help, all involve > speeding up process when there is "expensive" operation, which can be > skipped MOST of the time, and it's OK to be slightly behind. I'm > implementing all of those cases with combination of macros/shell commands= . > > > > 1. Cache (original example): > > Assuming the build need a remote resource that take lot of time to > download/process, and does not change frequently, Adding "valid-max" make > it possible to declare - It's OK with this target if it was created in th= e > last day - no need to reload. > > > > cache.json: private valid-max=3D1d > > cache.json: > > curl -o $@ https://example/api > > > > 2. Reduce excessive updates. > > In some cases, the build process depend on files can be updated multipl= e > time. This could be a CI process that produces new artifacts on every > commit. when running heavy tests, it might not be practical to execute th= e > full test every time the remote artifact is recreated. Adding "valid-max= =3D" > on non-leaf node, limit the execution of the tests to no more than once > every 24 hours (1d). > > > > tests.txt: private valid-for=3D1d > > tests.txt: /path/to/remote/artifact.tar > > tar xvf $< > > run_tests > > touch $@ > > > > 3. Expiration of targets, when dependencies are incomplete. > > In some cases, it's not possible to describe the full set of > dependencies (e.g., when it's dynamic), or when dependencies are "noisy". > For example - I had a case where a job had a dependency on remote artifac= ts > and local artifacts, and we wanted to force a policy that require the job > to run (and re-fetch) the remote artifacts, even if none of the local > artifacts got updated. Adding valid-max indicate that the job should be > re-make if it's older than 1 day, even of leave dependency was updated. > > > > job.txt: private valid-max=3D1d > > job.txt: /path/to/other/artifact.tgz > > tar xvf $< > > curl .... | tar xvf - > > run_job.sh > > touch job.txt > > > > I hope that those examples show how the new functionality make it easie= r > to address common use cases when using make for CI/data integration > processes - where the simple DAG rule may result in non-practical results= . > I'm sure that there are other potential usages for building practical > processes for testing, deployment, database integration - and other syste= ms > that do not provide exact modification timestamp, or when it's not > practical to use simple DAG rules. > > Ok. I don't know much about CI or the cases you describe so my > comment is limited to the use of the variable syntax to add some new > functionality to make. I think it's an appropriate strategy for Make > (instead of expanding the syntax) but care is needed to make sure it's > completely clear what's going on, and that wrong uses give clear > errors. So I wonder what happens if the private modifier is omitted? > I tend to think both of these should be errors. (Maybe they already > are I haven't looked at the patch itself). > > Britton > --00000000000082767c064b2b2046 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Britton,<div><br></div><div>Thank you for you feedback - I= agree 100<font face=3D"verdana, sans-serif">% better to use existing synta= x than to expand syntax.</font></div><div><font face=3D"verdana, sans-serif= "><br></font></div><div><font face=3D"verdana, sans-serif">I did some more = experiments with the implementation I've submitted, and I found that th= ere are valid use cases for using valid-max (and valid-for) in both situati= ons (with and without private).=C2=A0</font></div><div><font face=3D"verdan= a, sans-serif"><br></font></div><div><font face=3D"verdana, sans-serif">As = for error handing (on invalid values, e,g valid-max=3D"last Monday&quo= t;, or other) - the implementation is aligned with current make behavior - = errors and warnings are logged with the standard macros already available (= DBS, DBF, ...) - with proper level (warning, info). This make the existing = debugging, troubleshooting tools (-d, ...) works - no new options to learn.= </font></div><div><br></div><div><font face=3D"verdana, sans-serif">Yair.</= font></div><div><font face=3D"verdana, sans-serif"><br></font></div><div><f= ont face=3D"verdana, sans-serif"><br></font></div></div><br><div class=3D"g= mail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On = Wed, Feb 18, 2026 at 10:38=E2=80=AFPM Britton Kerin <<a href=3D"mailto:b= [email protected]">[email protected]</a>> wrote:<br></div><bl= ockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-lef= t:1px solid rgb(204,204,204);padding-left:1ex">On Tue, Feb 17, 2026 at 5:16= =E2=80=AFPM Yair Lenga <<a href=3D"mailto:[email protected]" target= =3D"_blank">[email protected]</a>> wrote:<br> ><br> > Hi Britton,<br> ><br> > Thanks for taking the time to read my proposal. Good point about docum= entation, should have included more use case. See below for some additional= example. If the proposal will be accepted - I look into creating a documen= tation pages for "info make" for various use cases:<br> ><br> > Is it usable only for leaves ?<br> > - Short answer: It can be used also on non-leaves to limit rebuild, or= limit reuse of targets. In it's simplest form - it can be used on leav= es that (including leaf that depend on "PHONY" targets).<br> ><br> > Below are the main use cases where new features will help, all involve= speeding up process when there is "expensive" operation, which c= an be skipped MOST of the time, and it's OK to be slightly behind. I= 9;m implementing all of those cases with combination of macros/shell comman= ds.<br> ><br> > 1. Cache (original example):<br> > Assuming the build need a remote resource that take lot of time to dow= nload/process, and does not change frequently, Adding "valid-max"= make it possible to declare - It's OK with this target if it was creat= ed in the last day - no need to reload.<br> ><br> > cache.json: private valid-max=3D1d<br> > cache.json:<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0curl -o $@ <a href=3D"https://= example/api" rel=3D"noreferrer" target=3D"_blank">https://example/api</a><b= r> ><br> > 2. Reduce excessive updates.<br> > In some cases, the build process depend on files can be updated multip= le time. This could be a CI process that produces new artifacts on every co= mmit. when running heavy tests, it might not be practical to execute the fu= ll test every time the remote artifact is recreated. Adding "valid-max= =3D" on non-leaf node, limit the execution of the tests to no more tha= n once every 24 hours (1d).<br> ><br> > tests.txt: private valid-for=3D1d<br> > tests.txt: /path/to/remote/artifact.tar<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0tar xvf $<<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0run_tests<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0touch $@<br> ><br> > 3. Expiration of targets, when dependencies are incomplete.<br> > In some cases, it's not possible to describe the full set of depen= dencies (e.g., when it's dynamic), or when dependencies are "noisy= ". For example - I had a case where a job had a dependency on remote a= rtifacts and local artifacts, and we wanted to force a policy that require = the job to run (and re-fetch) the remote artifacts, even if none of the loc= al artifacts got updated. Adding valid-max indicate that the job should be = re-make if it's older than 1 day, even of leave dependency was updated.= <br> ><br> > job.txt: private valid-max=3D1d<br> > job.txt: /path/to/other/artifact.tgz<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0tar xvf $<<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0curl .... | tar xvf -<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0run_job.sh<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0touch job.txt<br> ><br> > I hope that those examples show how the new functionality make it easi= er to address common use cases when using make for CI/data integration proc= esses - where the simple DAG rule may result in non-practical results. I= 9;m sure that there are other potential usages for building practical proce= sses for testing, deployment, database integration - and other systems that= do not provide exact modification timestamp, or when it's not practica= l to use simple DAG rules.<br> <br> Ok.=C2=A0 I don't know much about CI or the cases you describe so my<br= > comment is limited to the use of the variable syntax to add some new<br> functionality to make.=C2=A0 I think it's an appropriate strategy for M= ake<br> (instead of expanding the syntax) but care is needed to make sure it's<= br> completely clear what's going on, and that wrong uses give clear<br> errors.=C2=A0 So I wonder what happens if the private modifier is omitted?<= br> I tend to think both of these should be errors.=C2=A0 (Maybe they already<b= r> are I haven't looked at the patch itself).<br> <br> Britton<br> </blockquote></div> --00000000000082767c064b2b2046--