Re: timestamp(?) problems with incremental dumps

--===============0583006839==
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="zGQnqpIoxlsbsOfg"
Content-Disposition: inline


--zGQnqpIoxlsbsOfg
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

* Message by -Joerg Schilling- from Thu 2007-10-18:
> Lasse Kliemann <[email protected]> wrote:
>=20
> > > > The problem may be that the filesystem does not sync meta data fast=
 enough.
> > > =20
> > > To determine this, I must take a look at the file in question on the =
snapshot=20
> > > which was used for the level 2 dump. I will do so in the next tests.
> > =20
> > It's technically slightly more complicated than I thought, because I wi=
ll=20
> > have to keep the older snapshots around for this during the test. That'=
s why=20
> > I haven't looked into this yet.
> >
> > However, I discovered further details:
> >
> > > If this is the case, should a `sync' between touching the timestamp a=
nd=20
> > > creating the snapshot help? (Well, it depends on the sync implementat=
ion and=20
> > > the filesystem, I guess.)
> >
> > A single `sync' is not enough.
> >
> > It seems that `sleep x' with a high enough value of x between the times=
tamp=20
> > and the snapshot remedies the problem. The interesting point is the cho=
ice=20
> > of x. I was able to show that the problem occurs for x=3D0.299, but alt=
hough I=20
> > tried very hard, I could *not* provoke the problem with x=3D0.3. So, 30=
0=20
> > milliseconds seem to be the magic threshold.
>=20
> OK, this verifies that the problem is caused by the implementation of fil=
esystem=20
> in the kernel. I would even call it a bug if you may create a snapshot th=
at
> refers to unstable (unreliable) data.
>=20
> On Solaris, creating a snapshot results in write-loking the FS, syncing,=
=20
> creating the snapshot and removing the write-lock. This gives you reliable
> data in the snapshot.
>=20
> I would encourage you to send a bug report to the relevent people.

I haven't done so yet, mostly because this 300ms threshold soon after was=
=20
disproved. At least on a *busy* system, much larger values for x were neede=
d,=20
ranging in the order of a second or so. I had to stop these experiments due=
=20
to time constraints soon after that discovery.

But I tried something else, maybe it's interesting:

- Filesystem A is busy, take snapshot S from it.
- Synchronize filesystem B with S using 'rsync -axH --inplace'.
- Remount B read-only.
- Take an incremental dump from B.
- Restore the incremental dump(s) taken this way to a filesystem C.
- Compare the contents of S with C using diffopts=3D\!ctime,\!lmtime,\!atim=
e.
- Release S.
- Start over after some time.

So, in B we keep a kind of "mirror" of A, which has the advantage that we c=
an=20
mount it read-only and that it is not a snapshot. B is not changed between=
=20
iterations; all we do with it is one run of rsync before each dump. (With=
=20
rsync 3.0.0 a patch [1] is suggested for this.)

This procedure revealed no differences between the snapshot and the restore=
d=20
filesystem so far.

[1] http://www.mail-archive.com/[email protected]/msg20982.html ff.

Lasse

--zGQnqpIoxlsbsOfg
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)

iQIcBAEBAgAGBQJH00CeAAoJEFYll6N4nhdvxWYQAIcTVdGngrOPPykhaETsdcGW
Q6EJZJmoJaihppno2qayPEDyg/Lc7nVmCnl/EK8L4uYxk1+zWbtYf7v7rYz3WOuP
N6JZ8xasnovGtt7Aqv/ld6/fQjnychDy7rC1gaSPSfQWOO9Wm/r+47iVBqmT5w5l
WNgcLMlguX+O/M18UjFPouKO/6nQ+YIp5Q9BXZEZggSIubLWs7NPYa3V17+2gW6/
VdptXYE8w440A9wZOtW3fn+LCXrtoIQzUf/6vUMFmrv5iIW+1P2J9h4uc4ZytnQ6
41Rrq56VJzMtAJAarrAccFJSUXfD4CFXRYa3mLzS3cjD793zNalmdxLoVBkw4XA8
XBKSUnREicPAyIx+JgIt1GDwflKO9ydCGXM8cPv3KmT2vX1Nkd6Ijh+2lmo/1hjn
7eMHfDzrp2P6o5vUthq/oaxndib8b8lHcY8jOpCWQYrDskegH/eREQpq4XMVOmwW
pLWYiGTKKBAPZnQDFeRDOPTU5CN7mQehB7flYy8RP/0EVG5WQl6/0c7a4JX/FraD
PZPTI9vPh0ySnF9skr1aoJnS9cvfDmBfN5ES7fk2jEwgNwsBS5oL9enkpGYiQQc0
KWQC7sz19GMoj1xDdTX43DlE3hajblDY6+bqBl39vphuV/3A8QmYR9Dc1Na5fscJ
tIiEwHHpEROiDgMNpYgp
=C0xy
-----END PGP SIGNATURE-----

--zGQnqpIoxlsbsOfg--

--===============0583006839==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Star-users mailing list
[email protected]
https://lists.berlios.de/mailman/listinfo/star-users

--===============0583006839==--