Re: lucene3/lucene4 jenkins boxes - sha1 git errors

--000000000000dff8b4065470173c
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Thanks for digging into this, Dawid

On Wed, Jun 17, 2026, 4:59=E2=80=AFAM Dawid Weiss <[email protected]> w=
rote:

>
> Hello everyone,
>
> I cross-post to Solr and Lucene dev lists because we share ASF jenkins
> boxes that
> execute jenkins workflows.
>
> There has been a series of odd exceptions from Jenkins, looking pretty
> much like this:
>
> ...
> > git reset --hard # timeout=3D10
> ERROR: Error fetching remote repo 'origin'
> hudson.plugins.git.GitException: Failed to fetch from
> https://github.com/apache/lucene.git
> ...
> Caused by: hudson.plugins.git.GitException: Command "git reset --hard"
> returned status code 128:
> stdout:
> stderr: error: unable to read sha1 file of .asf.yaml
> (48083930a50d886827c1f87e23875e5da98c58c6)
> error: unable to read sha1 file of .dir-locals.el
> (c51e1232603b85b8bc74fbed7d1de08186920379)
> error: unable to read sha1 file of .git-blame-ignore-revs
> (945e687d0f3cdfcc99cb69b8b35ab631a70371a7)
> error: unable to read sha1 file of .gitattributes
> (a3135003e80fa8f49fc0f2250f40b85cde12ebc5)
> ...
>
> I've ssh'd to lucene3/lucene4 and it seems like something leaves the .git
> folder of a workspace checkout in a broken state, leading to subsequent g=
it
> reset failures. I'm not sure if it's a jenkins task timeout kicking in or
> something else.
>
> There seems to be enough drive space on both boxes, although it's close t=
o
> full. Looking at asf INFRA jira, the same type of exception was always
> caused by disk space running low so for now I've:
>
> - removed all existing workspaces under
> /home/jenkins/jenkins-agent/workspace/Lucene
> - removed all existing workspaces under
> /home/jenkins/jenkins-agent/workspace/Solr
>
> I didn't know how to pause jenkins for this, apologies if you get an erro=
r
> from a job that was running at the moment I removed those folders.
>
> Oddly enough, I can't see the same error appearing on Solr jenkins runs -
> could be that all the jobs there have a "wipe workspace before you start"
> checkbox enabled... I'm really not sure, sorry.
>
> Right now, df -h shows:
>
> lucene3 /dev/sda2 503G 59G 419G 13% /
> lucene4 /dev/sda2 503G 99G 379G 21% /
>
> Let's see if this fixes the problem.
>
> Dawid
>

--000000000000dff8b4065470173c
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto">Thanks for digging into this, Dawid</div><br><div class=
=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr=
">On Wed, Jun 17, 2026, 4:59=E2=80=AFAM Dawid Weiss &lt;<a href=3D"mailto:d=
[email protected]">[email protected]</a>&gt; wrote:<br></div><blockq=
uote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc =
solid;padding-left:1ex"><div dir=3D"ltr"><br><div>Hello everyone,</div><div=
><br></div><div>I cross-post to Solr and Lucene dev lists because we share =
ASF jenkins boxes that</div><div>execute jenkins workflows.</div><div><br><=
/div><div>There has been a series of odd exceptions from Jenkins, looking p=
retty much like this:</div><div><br></div><div>...</div><div>&gt; git reset=
 --hard # timeout=3D10<br>ERROR: Error fetching remote repo &#39;origin&#39=
;<br>hudson.plugins.git.GitException: Failed to fetch from <a href=3D"https=
://github.com/apache/lucene.git" target=3D"_blank" rel=3D"noreferrer">https=
://github.com/apache/lucene.git</a><br></div><div>...</div><div><span style=
=3D"background-color:transparent">Caused by: hudson.plugins.git.GitExceptio=
n: Command &quot;git reset --hard&quot; returned status code 128:</span><br=
></div><div>stdout: <br>stderr: error: unable to read sha1 file of .asf.yam=
l (48083930a50d886827c1f87e23875e5da98c58c6)<br>error: unable to read sha1 =
file of .dir-locals.el (c51e1232603b85b8bc74fbed7d1de08186920379)<br>error:=
 unable to read sha1 file of .git-blame-ignore-revs (945e687d0f3cdfcc99cb69=
b8b35ab631a70371a7)<br>error: unable to read sha1 file of .gitattributes (a=
3135003e80fa8f49fc0f2250f40b85cde12ebc5)<br>...</div><div><br></div><div>I&=
#39;ve ssh&#39;d to lucene3/lucene4 and it seems like something leaves the =
.git folder of a workspace checkout in a broken state, leading to subsequen=
t git reset failures. I&#39;m not sure if it&#39;s a jenkins task timeout k=
icking in or something else.<br><br>There seems to be enough drive space on=
 both boxes, although it&#39;s close to full. Looking at asf INFRA jira, th=
e same type of exception was always caused by disk space running low so for=
 now=C2=A0<span style=3D"background-color:transparent">I&#39;ve:</span></di=
v><div><br>- removed all existing workspaces under /home/jenkins/jenkins-ag=
ent/workspace/Lucene<br></div><div>- removed all existing workspaces under =
/home/jenkins/jenkins-agent/workspace/Solr</div><div><br></div><div>I didn&=
#39;t know how to pause jenkins for this, apologies if you get an error fro=
m a job that was running at the moment I removed those folders.</div><div><=
br></div><div>Oddly enough, I can&#39;t see the same error appearing on Sol=
r jenkins runs - could be that all the jobs there have a &quot;wipe workspa=
ce before you start&quot; checkbox enabled... I&#39;m really not sure, sorr=
y.</div><div><br></div><div>Right now, df -h shows:</div><div><br></div><di=
v><div style=3D"font-family:monospace;white-space:pre-wrap">lucene3
/dev/sda2       503G   59G  419G  13% /
<br></div><div style=3D"font-family:monospace;white-space:pre-wrap">lucene4
/dev/sda2       503G   99G  379G  21% /</div></div><div><br></div><div>Let&=
#39;s see if this fixes the problem.</div><div><br></div><div>Dawid</div></=
div>
</blockquote></div>

--000000000000dff8b4065470173c--