Re: Many orphaned inodes after resize2fs

--===============3167286509766611248==
Content-Type: multipart/alternative; boundary=047d7b5da5bbe4ef5d04f7596501

--047d7b5da5bbe4ef5d04f7596501
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi,

it seems you got it right! I don't know if you read email I sent you before
posting to the mailing list, but I accidentally diagnosed the cause... :)
I've noticed that inodes fsck warned me about, at least ones that I
checked, all have all four timestamps latest in 2010...

The filesystem has maximum 1281998848 inodes, which is timestamp in august
2010. I don't know how it got that big, I think I did not specified big
value initially. But I've resized it couple of times. BTW what is default
of group size / inode count ratio? Mine ratio is not at the maximum you
mentioned, but it is not that far.

So almost sure it is false positive by the code / bug in e2fsck/pass1.c
around line 1070 in current version. I want to be sure that all these
errors were caused by this, so can you please send me promptly patched
version? I can easily patch it myself by some fixed condition, but I don't
want miss something important... BTW maybe you can compare i_dtime with
filesystem creation timestamp, so you dont have to put fixed number there.

BTW I dont know specifics of ext3, I just looked at sources of kernel
driver and e2fsprogs now. But what indicates that inode is / was created
and valid ? (I did not need it to find problematic test you mentioned, did
not see it in part of code I look at and it is not apparent to me from
definition of struct ext3_inode).

Thanks.

Patrik


2014-04-18 22:20 GMT+02:00 <[email protected]>:

> On Fri, Apr 18, 2014 at 06:56:57PM +0200, Patrik Horn=EDk wrote:
> >
> > yesterday I experienced following problem with my ext3 filesystem:
> >
> > - I had ext3 filesystem of the size of a few TB with journal. I correct=
ly
> > unmounted it and it was marked clean.
> >
> > - I then ran fsck.etx3 -f on it and it did not find any problem.
> >
> > - After increasing size of its LVM volume by 1.5 TB I resized the
> > filesystem by resize2fs lvm_volume and it finished without problem.
> >
> > - But fsck.ext3 -f immediately after that showed "Inodes that were part
> of
> > a corrupted orphan linked list found." and many thousands of "Inode XXX
> was
> > part of the orphaned inode list." I did not accepted fix. According to
> > debugfs all the inodes I check from these reported orphaned inodes (I
> > checked only some from beginning of list of errors) have size 0.
>
> Can you send the output of dumpe2fs -h?  I'm curious how many inodes
> you had after the resize, and what file system features might have
> been enabled on your file system.
>
> If the only file system corruption errors that you saw were from about
> the corrupted orphan inode list, then things are probably OK.
>
> What this error message means is that there are d_time values which
> look like they belong to inode numbers (as opposed to number of
> seconds since January 1, 1970).  So if you ran the system where the
> clock was set incorrectly, so that the time was January 1, 1970, and
> you delete a lot of files, you can run into this error --- it's
> basically a sanity check that we put in a long time ago to catch
> potential file system bugs caused by a corrupted orphan inode list.
>
> I'm thinking that we should turn off this check if the e2fsck.conf
> "broken_system_lock" is enabled, since if the system has a busted
> system clock, this can end up triggering a bunch of scary warnings.
>
> In any case, when you grew the size of the file system, this also
> increased the number of inodes, which means it would increase the
> sensitivity of hitting this bug.  It's also possible that if you
> created your file system with the number of inodes per block group
> close to the maximum (assuming an average file size 4k, which would be
> highly wasteful of space, so it' s not the default), that you ended up
> with the maximum number of inodes exceeding 1.2 or 1.3 billion inodes,
> at which point it would trigger a false positive.  (And indeed, I
> should probably put in a fix to e2fsprogs so that if a file system
> does have more than 1.2 billion inodes, to disable this check.)
>
> Cheers,
>
>                                                 - Ted

--047d7b5da5bbe4ef5d04f7596501
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hi,</div><div><br></div><div>it seems you got it righ=
t! I don&#39;t know if you read email I sent you before posting to the mail=
ing list, but I accidentally diagnosed the cause... :) I&#39;ve noticed tha=
t inodes fsck warned me about, at least ones that I checked, all have all f=
our timestamps latest in 2010...</div>
<div><br></div><div>The filesystem has maximum 1281998848 inodes, which is =
timestamp in august 2010. I don&#39;t know how it got that big, I think I d=
id not specified big value initially. But I&#39;ve resized it couple of tim=
es. BTW what is default of group size / inode count ratio? Mine ratio is no=
t at the maximum you mentioned, but it is not that far.</div>
<div><br></div><div>So almost sure it is false positive by the code / bug i=
n e2fsck/pass1.c around line 1070 in current version. I want to be sure tha=
t all these errors were caused by this, so can you please send me promptly =
patched version? I can easily patch it myself by some fixed condition, but =
I don&#39;t want miss something important... BTW maybe you can compare i_dt=
ime with filesystem creation timestamp, so you dont have to put fixed numbe=
r there.</div>
<div><br></div><div>BTW I dont know specifics of ext3, I just looked at sou=
rces of kernel driver and e2fsprogs now. But what indicates that inode is /=
 was created and valid ? (I did not need it to find problematic test you me=
ntioned, did not see it in part of code I look at and it is not apparent to=
 me from definition of=A0<span class=3D"" style=3D"font-size:13px;color:rgb=
(131,0,0)">struct</span><span style=3D"color:rgb(0,0,0);font-size:13px"> ex=
t3_inode</span>).</div>
<div><br></div><div>Thanks.</div><div><br></div><div class=3D"gmail_extra">=
<div>Patrik</div>
<br><br><div class=3D"gmail_quote">2014-04-18 22:20 GMT+02:00  <span dir=3D=
"ltr">&lt;<a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]<=
/a>&gt;</span>:<br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0p=
x 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border=
-left-style:solid;padding-left:1ex">

<div>On Fri, Apr 18, 2014 at 06:56:57PM +0200, Patrik Horn=EDk wrote:<br>
&gt;<br>
&gt; yesterday I experienced following problem with my ext3 filesystem:<br>
&gt;<br>
&gt; - I had ext3 filesystem of the size of a few TB with journal. I correc=
tly<br>
&gt; unmounted it and it was marked clean.<br>
&gt;<br>
&gt; - I then ran fsck.etx3 -f on it and it did not find any problem.<br>
&gt;<br>
&gt; - After increasing size of its LVM volume by 1.5 TB I resized the<br>
&gt; filesystem by resize2fs lvm_volume and it finished without problem.<br=
>
&gt;<br>
&gt; - But fsck.ext3 -f immediately after that showed &quot;Inodes that wer=
e part of<br>
&gt; a corrupted orphan linked list found.&quot; and many thousands of &quo=
t;Inode XXX was<br>
&gt; part of the orphaned inode list.&quot; I did not accepted fix. Accordi=
ng to<br>
&gt; debugfs all the inodes I check from these reported orphaned inodes (I<=
br>
&gt; checked only some from beginning of list of errors) have size 0.<br>
<br>
</div>Can you send the output of dumpe2fs -h? =A0I&#39;m curious how many i=
nodes<br>
you had after the resize, and what file system features might have<br>
been enabled on your file system.<br>
<br>
If the only file system corruption errors that you saw were from about<br>
the corrupted orphan inode list, then things are probably OK.<br>
<br>
What this error message means is that there are d_time values which<br>
look like they belong to inode numbers (as opposed to number of<br>
seconds since January 1, 1970). =A0So if you ran the system where the<br>
clock was set incorrectly, so that the time was January 1, 1970, and<br>
you delete a lot of files, you can run into this error --- it&#39;s<br>
basically a sanity check that we put in a long time ago to catch<br>
potential file system bugs caused by a corrupted orphan inode list.<br>
<br>
I&#39;m thinking that we should turn off this check if the e2fsck.conf<br>
&quot;broken_system_lock&quot; is enabled, since if the system has a busted=
<br>
system clock, this can end up triggering a bunch of scary warnings.<br>
<br>
In any case, when you grew the size of the file system, this also<br>
increased the number of inodes, which means it would increase the<br>
sensitivity of hitting this bug. =A0It&#39;s also possible that if you<br>
created your file system with the number of inodes per block group<br>
close to the maximum (assuming an average file size 4k, which would be<br>
highly wasteful of space, so it&#39; s not the default), that you ended up<=
br>
with the maximum number of inodes exceeding 1.2 or 1.3 billion inodes,<br>
at which point it would trigger a false positive. =A0(And indeed, I<br>
should probably put in a fix to e2fsprogs so that if a file system<br>
does have more than 1.2 billion inodes, to disable this check.)<br>
<br>
Cheers,<br>
<br>
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 - Ted</blockquote></div></div></div>

--047d7b5da5bbe4ef5d04f7596501--


--===============3167286509766611248==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Ext3-users mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/ext3-users
--===============3167286509766611248==--