Re: Re: Finding missing blobs

--000000000000a46942057c86f7f4
Content-Type: text/plain; charset="UTF-8"

On Wed, Dec 5, 2018 at 6:35 AM Alessandro Pisa <[email protected]>
wrote:

> On Wed, 5 Dec 2018 at 14:07, Alessandro Pisa <[email protected]>
> wrote:
> >
> > Hello everybody,
> >     I have a DB with missing blobs.
> > What is the proper way to scan all the blobs and report the missing ones?
> >
>
> For the moment I am checking https://pypi.org/project/zc.zodbdgc/
>
> Running it on a DB where I removed on purpose on blob tells me:
>
> $ cat zeo.conf
> <zodb main>
>  <blobstorage>
>    blob-dir /path/to/blobs
>    <filestorage>
>       path /path/to/Data.fs
>    </filestorage>
>  </blobstorage>
> </zodb>
> $ ./bin/multi-zodb-check-refs zeo.conf
> !!! main 328146 ?
> POSKeyError: 'No blob file'
>
> Then I can check the blob with:
> $ fsoids Data.fs 328146
> oid 0x0501d2 ZODB.blob.Blob 1 revision
>    tid 0x03be0bafcbf1f8dd offset=15782912 2017-03-16 17:19:47.799657
>        tid user=''
>        tid description=''
>        new revision ZODB.blob.Blob at 15897728
>        referenced by 0x0501c9 plone.app.blob.field.BlobWrapper at 15846484
>
> and get it from the connection (running a zope instance in debug mode):
> >>> from ZODB.utils import p64
> >>> app._p_jar.get(p64(331142))
> <ZODB.blob.Blob object at 0x7f3f79f91668>
>
>
That's pretty cool. Well done!


> Is this the mainstream way?
>

I don't think there's a "mainstream way", as I don't think this is a common
(commonly-known?) problem.  I haven't encountered or heard of it before.

Jim

-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.

--000000000000a46942057c86f7f4
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Wed=
, Dec 5, 2018 at 6:35 AM Alessandro Pisa &lt;<a href=3D"mailto:alessandro.p=
[email protected]">[email protected]</a>&gt; wrote:<br></div><blockquot=
e class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc sol=
id;padding-left:1ex">On Wed, 5 Dec 2018 at 14:07, Alessandro Pisa &lt;<a hr=
ef=3D"mailto:[email protected]" target=3D"_blank">alessandro.pisa@g=
mail.com</a>&gt; wrote:<br>
&gt;<br>
&gt; Hello everybody,<br>
&gt;=C2=A0 =C2=A0 =C2=A0I have a DB with missing blobs.<br>
&gt; What is the proper way to scan all the blobs and report the missing on=
es?<br>
&gt;<br>
<br>
For the moment I am checking <a href=3D"https://pypi.org/project/zc.zodbdgc=
/" rel=3D"noreferrer" target=3D"_blank">https://pypi.org/project/zc.zodbdgc=
/</a><br>
<br>
Running it on a DB where I removed on purpose on blob tells me:<br>
<br>
$ cat zeo.conf<br>
&lt;zodb main&gt;<br>
=C2=A0&lt;blobstorage&gt;<br>
=C2=A0 =C2=A0blob-dir /path/to/blobs<br>
=C2=A0 =C2=A0&lt;filestorage&gt;<br>
=C2=A0 =C2=A0 =C2=A0 path /path/to/Data.fs<br>
=C2=A0 =C2=A0&lt;/filestorage&gt;<br>
=C2=A0&lt;/blobstorage&gt;<br>
&lt;/zodb&gt;<br>
$ ./bin/multi-zodb-check-refs zeo.conf<br>
!!! main 328146 ?<br>
POSKeyError: &#39;No blob file&#39;<br>
<br>
Then I can check the blob with:<br>
$ fsoids Data.fs 328146<br>
oid 0x0501d2 ZODB.blob.Blob 1 revision<br>
=C2=A0 =C2=A0tid 0x03be0bafcbf1f8dd offset=3D15782912 2017-03-16 17:19:47.7=
99657<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0tid user=3D&#39;&#39;<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0tid description=3D&#39;&#39;<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0new revision ZODB.blob.Blob at 15897728<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0referenced by 0x0501c9 plone.app.blob.field.Blob=
Wrapper at 15846484<br>
<br>
and get it from the connection (running a zope instance in debug mode):<br>
&gt;&gt;&gt; from ZODB.utils import p64<br>
&gt;&gt;&gt; app._p_jar.get(p64(331142))<br>
&lt;ZODB.blob.Blob object at 0x7f3f79f91668&gt;<br>
<br></blockquote><div><br></div><div>That&#39;s pretty cool. Well done!</di=
v><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 =
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Is this the mainstream way?<br></blockquote><div><br></div><div>I don&#39;t=
 think there&#39;s a &quot;mainstream way&quot;, as I don&#39;t think this =
is a common (commonly-known?) problem.=C2=A0 I haven&#39;t encountered or h=
eard of it before.</div><div><br></div><div>Jim</div><div><br></div></div>-=
- <br><div dir=3D"ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_si=
gnature">Jim Fulton<br><a href=3D"http://jimfulton.info" target=3D"_blank">=
http://jimfulton.info</a><br></div></div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;zodb&quot; group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:[email protected]">zodb+unsubscri=
[email protected]</a>.<br />
For more options, visit <a href=3D"https://groups.google.com/d/optout">http=
s://groups.google.com/d/optout</a>.<br />

--000000000000a46942057c86f7f4--