Re: Zeopack fails raising KeyError

On Tue, Mar 7, 2017 at 12:04 PM, Héctor Velarde <[email protected]>
wrote:

> thank you, very much, Jim! database is now 7GB, down from 9GB!


Cool.


>
>
> On 03/07/2017 01:21 PM, Jim Fulton wrote:

...

> Gaaaaa, I just realized that it should be straightforward to create such
>> a tool, if we had OIDS. It's a shame that the error above doesn't give a
>> POSKeyError with a full OID.  I wonder if newer versions of ZODB give
>> POSKeyErrors in this situation.
>>
>
> HV> that would be awesome! how can we help?


IDK :)

...

I also suggest using zc.FileStorage to do the non-gc pack, at it is much
>> faster.
>>
>
> HV> I was checking but the package has absolutely no documentation


Quite possible. :)  It was built in battle at ZC many years ago, but then
was also used in production on almost all of our servers for several years.

Hm, look at:
https://bitbucket.org/zc/zrs-rpm/src/8e04cfcab36e394e80e44dfe34768d9c9d0db0f9/src/zc_zrs_rpm_recipes/README.rst?at=master&fileviewer=file-view-default

and search for zc.FileStorage. :)

It uses a packer hook provided by FileStorage.

In addition to a better pack algorithm:

- It does most work in a subprocess, thus escaping the GIL.

- It uses fadvise to avoid destroying disk cache.

It was helpful for packing our terabyte  (pre blob) databases.

On second thought. Never mind. It doesn't solve a problem you have. :)

It would be nice if I could find time to update FileStorage to use it.
<shug>



> and I was not able to find the source:
>

I just converted it from the old svn.zope.org snapshot I made.
Unfortunately, there was a problem converting tags for some reason, but the
code and history seem to be intact.


>
> https://pypi.python.org/pypi/zc.FileStorage
>
> Sometime, I need to find the time to roll the functionality from those
>> two packages back into FileStorage (unless I replace it with
>> FileStorage2).
>>
>
> HV> is that because you lack the time or resources?
>
> once again, how can we help?


I lack the resources to have the time. ;)

But also, I'm unsure how much effort to put into FileStorage/ZEO.

For a while, I've been planning to replace FileStorage/ZEO with
FileStorage2/byteserver.

Honestly, I'm starting to wonder if most people shouldn't just use
RelStorage + Postgres at this point.  History-free RelStorage + Postgres is
still significantly faster than ZEO.  And a soon (I hope) to be released
version of RelStorage:

https://pypi.python.org/pypi/RelStorage/2.1a1

is another 30% faster.  Postgres as a service offerings make this seem even
more attractive. And Newt!

Jim

-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.