In defense of the ZODB
Christopher Lozinski <[email protected]> Sun, 28 Oct 2018 14:36:53 +0100
| Newsgroups | gmane.comp.web.zope.zodb |
|---|---|
| Message-ID | <[email protected]> |
--Apple-Mail=_877C6F69-43D5-47AA-86D8-3830C7234DC9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" Thank you for the comments. It is most difficult to get good constructive = criticism. So I will go ahead and integrate your comments into the next ve= rsion of my talk.=20 First let me address specific criticisms of the ZODB, and then we will tal= k about our different points of view. Forgive me if I first restate your c= riticisms more politely. And I will add a few other common criticisms.=20 1. The ZODB is a hierarchical database.=20 This is a common misconception. That is why in my talk I start off with t= he canonical Neo4J example, and show how it is implemented in the ZODB by G= raphagus. So that people would understand that the ZODB does graphs as wel= l as hierarchies. Throughout my talk I kept repeating that message, that t= he ZODB also does graphs.=20 Actually web site URL=E2=80=99s are hierarchical, so if one is building a w= ebsite, the ZODB, traversal, and views on objects work great.=20 2. The ZODB stores data in Pickles. =20 Well in order to serialize data and store it on the hard drive, you have to= use something. You recommended ArrangoDB.=20 =20 From Wikipedia: The database... internally ... uses ArangoDB's VelocyPack =E2=80=93 a fast = and compact binary format for serialization and storage.[7] <https://en.wik= ipedia.org/wiki/ArangoDB#cite_note-7> So you think that Velocity Pack is a better choice than Python Pickles? Indeed my talk was clearly labelled =E2=80=9C=E2=80=A6For Python Developers= .=E2=80=9C In fact I am just in the=20 process of debugging a copy command. it uses Python Pickles. Easy for me = to debug entirely in Python. proving that the ZODB is great for Python developers. = If I were to do that in=20 ArrangoDB, I would need access to a C development tool set.=20 Thank you for helping me to better understand this particular advantage of = the ZODB=20 over ArangoDB 3. The ZODB does not serve JSON. =20 And while it is true that the ZODB does not natively include software for c= onverting=20 an object to json, there are a number of Python libraries which do that. I = serve a=20 number of json files over at PythonLinks.info <http://pythonlinks.info/> = It is easy to serve data however you wish. Even better would be Apache Av= ro. There are Python libraries to help me do that. Does ArrangoDB serve A= pache Avro files? I think not.=20 4. The ZODB is old.=20 Python also is old. Are you suggesting that we not use Python? I find th= at software matures. The most recent version of Python keep getting better= . =20 The same is true of the ZODB. It is old, mature, well tested. Very stable= and mature. Its api is not constantly changing.=20 I think of its age as an advantage.=20 5. The ZODB does not scale. Just store your data in PostgreSQL and it will scale as well as any databas= e.=20 6. There is no company behind it.=20 Well indeed the Zope Corporation went bankrupt a long time ago, but the ope= n source ZODB continues to be upgraded and improved. What will happen if A= rrango Corporation goes broke? Are you worried about that? 7. The ZODB does not support indices. =20 Well just write your own, or use NewtDB which integrates the ZODB into Post= greSQL and their high performance indices.=20 8. The ZODB does not have edge objects. =20 Well it does have edges, the references between objects. It does not have = additional properties on those edges. But then I do not need that.=20 And if you need explicit edge objects, Graphagus added them in 500 lines o= f code. Actually at the time, the graphagus benchmarks were faster than th= e Neo4J benchmarks. In the meantime, I have no edge objects, and my code = is simpler.=20 9. The ZODB does not have a query language.=20 There is a project to add GraphQL to Plone. Zope.interface should make tha= t quite easy to do. Probably not that hard to add NetworkX or a python vers= ion of Gremlin to the ZODB. Actually as a Python developer, I am quite hap= py without a query language. Maybe my life is simpler with everything in Py= thon. I use innovative Javascript query tools on the client. ArrangoDB w= ould not help me there.=20 Did I miss any of the major ZODB criticisms? I will add these to the next = version of my talk.=20 POINTS OF VIEWS Now let us get onto the issues of point of view. I targeted this talk at d= evelopers using both Python and a relational database, considering using a = graph database.=20 I think of myself as a Python developer. I want to make things as simple a= s possible. Using the ZODB as a database is so much easier than using a r= elational database. That is the focus of my talk.=20 =20 You are part of a different community You are a seasoned Arrango Graph data= base user. You have certain expectations from a graph database. The ZODB= does not meet your needs. I understand that. =20 I would say that there are two types of databases. Object Graph and Proper= ty Graph. The ZODB is an object graph database. I believe that ArrangoDB = and Neo4J are property graph databases. And yes, the ZODB is not a prope= rty graph database. Nor does it include a query language. Does that mean= that the ZODB is just a hierarchical database? Technically not correct. = Nor are you the language police.=20 There were a number of other mischaracterizations of my statements. ZeroDB= , although no longer maintained is a very interesting concept. I am sure = that the software still works. That is the advantage of a mature database.= I am sorry that people do not care more about privacy. =20 Nor did I create a use case for the ZODB as a replacement for Neo4J or Aran= goDB. That was entirely in your mind. My point was that it is really easy= for Python developers to use compared to a relational database.=20 I am grateful for your feedback. It is always good to be polite. Another i= mportant skill is to understand other people=E2=80=99s points of views. An= d finally it is important to support diversity. Sometimes ArrangoDB is a b= etter choice, sometimes the ZODB is a better choice.=20 Chris --=20 You received this message because you are subscribed to the Google Groups "= zodb" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to [email protected]. For more options, visit https://groups.google.com/d/optout. --Apple-Mail=_877C6F69-43D5-47AA-86D8-3830C7234DC9 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="UTF-8" <html><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webk= it-line-break: after-white-space;" class=3D"">Thank you for the comments. &= nbsp;It is most difficult to get good constructive criticism. So I wi= ll go ahead and integrate your comments into the next version of my talk.&n= bsp;<div class=3D""><br class=3D""></div><div class=3D"">First let me addre= ss specific criticisms of the ZODB, and then we will talk about our d= ifferent points of view. Forgive me if I first restate your criticism= s more politely. And I will add a few other common criticisms. <= /div><div class=3D""><br class=3D""></div><div class=3D"">1. The ZODB= is a hierarchical database. </div><div class=3D""><br class=3D""></di= v><div class=3D"">This is a common misconception. That is why in my = talk I start off with the canonical Neo4J example, and show how i= t is implemented in the ZODB by Graphagus. So that people would under= stand that the ZODB does graphs as well as hierarchies. Throughout my= talk I kept repeating that message, that the ZODB also does graphs.&n= bsp;</div><div class=3D""><br class=3D""></div><div class=3D"">Actually web= site URL=E2=80=99s are hierarchical, so if one is building a website, the = ZODB, traversal, and views on objects work great. </div><div class=3D"= "><br class=3D""></div><div class=3D"">2. The ZODB stores data in Pic= kles. </div><div class=3D""><br class=3D""><div class=3D""><br class= =3D""></div><div class=3D"">Well in order to serialize data and store = it on the hard drive, you have to use something. You recommended Arra= ngoDB. </div><div class=3D""> </div><div class=3D""><div><div>Fro= m Wikipedia:</div><div><span style=3D"color: rgb(34, 34, 34); font-family: = sans-serif; font-variant-ligatures: normal; orphans: 2; widows: 2; backgrou= nd-color: rgb(255, 255, 255);" class=3D"">The database...</span><span style= =3D"color: rgb(34, 34, 34); font-family: sans-serif; font-variant-ligatures= : normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255);" cla= ss=3D""> internally ... uses ArangoDB's VelocyPack =E2=80=93 a fast an= d compact binary format for serialization and storage.</span><sup id=3D"cit= e_ref-7" class=3D"reference" style=3D"line-height: 1; unicode-bidi: isolate= ; white-space: nowrap; color: rgb(34, 34, 34); font-family: sans-serif; fon= t-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(2= 55, 255, 255);"><a href=3D"https://en.wikipedia.org/wiki/ArangoDB#cite_note= -7" style=3D"text-decoration: none; color: rgb(11, 0, 128); background-imag= e: none;" class=3D"">[7]</a></sup></div><div><br class=3D""></div><div>So y= ou think that Velocity Pack is a better choice than Python Pickles?</div><d= iv><span style=3D"color: rgb(34, 34, 34); font-family: sans-serif; white-sp= ace: nowrap; background-color: rgb(255, 255, 255); orphans: 2; widows: 2;" = class=3D"">Indeed my talk was clearly labelled =E2=80=9C=E2=80=A6For P= ython Developers.=E2=80=9C In fact I am just in the </span></div= ><div><div style=3D"orphans: 2; widows: 2;" class=3D""><font color=3D"#2222= 22" face=3D"sans-serif" class=3D""><span style=3D"background-color: rgb(255= , 255, 255);" class=3D""><span style=3D"white-space: nowrap;" class=3D"">pr= ocess of debugging a copy </span></span><span style=3D"white-space: no= wrap;" class=3D"">command. it uses Python Pickles. Easy for me = to debug</span></font></div><div style=3D"orphans: 2; widows: 2;" class=3D"= "><font color=3D"#222222" face=3D"sans-serif" class=3D""><span style=3D"whi= te-space: nowrap;" class=3D"">entirely in Python. proving that the ZO= DB is great for Python developers. If I were to do that in </spa= n></font></div><div style=3D"orphans: 2; widows: 2;" class=3D""><font color= =3D"#222222" face=3D"sans-serif" class=3D""><span style=3D"white-space: now= rap;" class=3D"">ArrangoDB, I would need access to a C developmen= t tool set. </span></font></div><div style=3D"orphans: 2; widows: 2;" = class=3D""><font color=3D"#222222" face=3D"sans-serif" class=3D""><span sty= le=3D"white-space: nowrap;" class=3D""><br class=3D""></span></font></div><= div style=3D"orphans: 2; widows: 2;" class=3D""><font color=3D"#222222" fac= e=3D"sans-serif" class=3D""><span style=3D"white-space: nowrap;" class=3D""= >Thank you for helping me to better understand this particular advanta= ge of the ZODB </span></font></div><div style=3D"orphans: 2; widows: 2= ;" class=3D""><font color=3D"#222222" face=3D"sans-serif" class=3D""><span = style=3D"white-space: nowrap;" class=3D"">over ArangoDB</span></font></div>= <div style=3D"orphans: 2; widows: 2;" class=3D""><font color=3D"#222222" fa= ce=3D"sans-serif" class=3D""><span style=3D"white-space: nowrap;" class=3D"= "><br class=3D""></span></font></div><div style=3D"orphans: 2; widows: 2;" = class=3D""><font color=3D"#222222" face=3D"sans-serif" class=3D""><span sty= le=3D"white-space: nowrap;" class=3D"">3. The ZODB does not serve JSON. &nb= sp;</span></font></div><div style=3D"orphans: 2; widows: 2;" class=3D""><fo= nt color=3D"#222222" face=3D"sans-serif" class=3D""><span style=3D"white-sp= ace: nowrap;" class=3D""><br class=3D""></span></font></div><div style=3D"o= rphans: 2; widows: 2;" class=3D""><font color=3D"#222222" face=3D"sans-seri= f" class=3D""><span style=3D"white-space: nowrap;" class=3D"">And while it = is true that the ZODB does not natively include software for conv= erting </span></font></div><div style=3D"orphans: 2; widows: 2;" class= =3D""><font color=3D"#222222" face=3D"sans-serif" class=3D""><span style=3D= "white-space: nowrap;" class=3D"">an object to json, there are a number of = Python libraries which do that. I serve a </span></font></div><div sty= le=3D"orphans: 2; widows: 2;" class=3D""><font color=3D"#222222" face=3D"sa= ns-serif" class=3D""><span style=3D"white-space: nowrap;" class=3D"">number= of json files over at </span></font><font color=3D"#222222" face=3D"= sans-serif" class=3D""><span style=3D"white-space: nowrap;" class=3D""><a h= ref=3D"http://pythonlinks.info/" class=3D"">PythonLinks.info</a>  = ; I</span></font>t is easy to serve data however you wish. Even bette= r would be Apache Avro. There are Python libraries to help me do that= . Does ArrangoDB serve Apache Avro files? I think not. </d= iv><div class=3D""><br class=3D""></div><div style=3D"orphans: 2; widows: 2= ;" class=3D""><font color=3D"#222222" face=3D"sans-serif" class=3D""><span = style=3D"white-space: nowrap;" class=3D"">4. The ZODB is old. </span><= /font></div><div style=3D"orphans: 2; widows: 2;" class=3D""><br class=3D""= ></div><div style=3D"orphans: 2; widows: 2;" class=3D""><font color=3D"#222= 222" face=3D"sans-serif" class=3D""><span style=3D"white-space: nowrap;" cl= ass=3D""> Python also is old. Are you suggesting that we no= t use </span></font><span style=3D"white-space: nowrap; color: rgb(34,= 34, 34); font-family: sans-serif;" class=3D"">Python? I find that so= ftware matures. The most recent version of Python keep getting b= etter. </span></div><div style=3D"orphans: 2; widows: 2;" class=3D"">= <font color=3D"#222222" face=3D"sans-serif" class=3D""><span style=3D"white= -space: nowrap;" class=3D"">The same is true of the ZODB. It is old, = mature, well tested. Very stable and mature. Its api is not con= stantly changing. </span></font></div><div style=3D"orphans: 2; widows= : 2;" class=3D""><font color=3D"#222222" face=3D"sans-serif" class=3D""><sp= an style=3D"white-space: nowrap;" class=3D""> I think of its age as an= advantage. </span></font></div><div style=3D"orphans: 2; widows: 2;" = class=3D""><br class=3D""></div><div style=3D"orphans: 2; widows: 2;" class= =3D""><br class=3D""></div><div style=3D"orphans: 2; widows: 2;" class=3D""= ><font color=3D"#222222" face=3D"sans-serif" class=3D""><span style=3D"whit= e-space: nowrap;" class=3D"">5. The ZODB does not </span><span s= tyle=3D"white-space: nowrap;" class=3D"">scale.</span></font></div><div sty= le=3D"orphans: 2; widows: 2;" class=3D""><font color=3D"#222222" face=3D"sa= ns-serif" class=3D""><span style=3D"white-space: nowrap;" class=3D""><br cl= ass=3D""></span></font></div><div style=3D"orphans: 2; widows: 2;" class=3D= ""><font color=3D"#222222" face=3D"sans-serif" class=3D""><span style=3D"wh= ite-space: nowrap;" class=3D"">Just store your data in PostgreSQL and it wi= ll scale as well as any database. </span></font></div></div></div><div= ><div style=3D"orphans: 2; widows: 2;" class=3D""><font color=3D"#222222" f= ace=3D"sans-serif" class=3D""><span style=3D"white-space: nowrap;" class=3D= ""><br class=3D""></span></font></div></div></div><div class=3D"">6. There = is no company behind it. </div><div class=3D""><br class=3D""></div><d= iv class=3D"">Well indeed the Zope Corporation went bankrupt a long ti= me ago, but the open source ZODB continues to be upgraded and improved. &nb= sp;What will happen if Arrango Corporation goes broke? Are you worri= ed about that?</div><div class=3D""><br class=3D""></div><div class=3D"">7.= The ZODB does not support indices. </div><div class=3D""><br c= lass=3D""></div><div class=3D"">Well just write your own, or use NewtD= B which integrates the ZODB into PostgreSQL and their high performance indi= ces. </div><div class=3D""><br class=3D""></div><div class=3D""><br cl= ass=3D""></div><div class=3D"">8. The ZODB does not have edge objects= . </div><div class=3D""><br class=3D""></div><div class=3D"">We= ll it does have edges, the references between objects. It does not ha= ve additional properties on those edges. But then I do not need that.= </div><div class=3D"">And if you need explicit edge objects, &nb= sp;Graphagus added them in 500 lines of code. Actually at the ti= me, the graphagus benchmarks were faster than the Neo4J benchmark= s. In the meantime, I have no edge objects, and my code is simp= ler. </div><div class=3D""><br class=3D""></div><div class=3D"">9. The= ZODB does not have a query language. </div><div class=3D""><br class= =3D""></div><div class=3D"">There is a project to add GraphQL to Plone. &nb= sp;Zope.interface should make that quite easy to do. Probably not that= hard to add NetworkX or a python version of Gremlin to the ZODB. Act= ually as a Python developer, I am quite happy without a query language. May= be my life is simpler with everything in Python. I use innovative Ja= vascript query tools on the client. ArrangoDB would not help me there= . </div><div class=3D""><br class=3D""></div><div class=3D""><br class= =3D""></div><div class=3D""><br class=3D""></div><div class=3D"">Did I miss= any of the major ZODB criticisms? I will add these to the next versi= on of my talk. </div><div class=3D""><br class=3D""></div><div class= =3D""><br class=3D""></div><div class=3D"">POINTS OF VIEWS</div><div class= =3D""><br class=3D""></div><div class=3D"">Now let us get onto the issues o= f point of view. I targeted this talk at developers using both P= ython and a relational database, considering using a graph database. <= /div><div class=3D""><br class=3D""></div><div class=3D"">I think of myself= as a Python developer. I want to make things as simple as possi= ble. Using the ZODB as a database is so much easier than using a rel= ational database. That is the focus of my talk. </div><div class= =3D""> </div><div class=3D"">You are part of a different communit= y You are a seasoned Arrango Graph database user. You have certain e= xpectations from a graph database. The ZODB does not meet your n= eeds. I understand that. </div><div class=3D""><br class= =3D""></div><div class=3D""><br class=3D""></div><div class=3D"">I would sa= y that there are two types of databases. Object Graph and Property Gr= aph. The ZODB is an object graph database. I believe that = ArrangoDB and Neo4J are property graph databases. And yes= , the ZODB is not a property graph database. Nor does it include a q= uery language. Does that mean that the ZODB is just a hierarchical da= tabase? Technically not correct. Nor are you the language = police. </div><div class=3D""><br class=3D""></div><div class=3D"">The= re were a number of other mischaracterizations of my statements. = ZeroDB, although no longer maintained is a very interestin= g concept. I am sure that the software still works. That is the= advantage of a mature database. I am sorry that people do not c= are more about privacy. </div><div class=3D""><br class=3D""></div><d= iv class=3D"">Nor did I create a use case for the ZODB as a replacement for= Neo4J or ArangoDB. That was entirely in your mind. My point wa= s that it is really easy for Python developers to use compared to a relatio= nal database. </div><div class=3D""><br class=3D""></div><div class=3D= ""><font class=3D"">I am grateful for your feedback. It is always good to b= e polite. Another important skill is to understand other people= =E2=80=99s points of views. And finally it is important to support di= versity. Sometimes ArrangoDB is a better choice, sometimes the ZODB i= s a better choice. </font></div><div class=3D""><br class=3D""></div><= div class=3D"">Chris</div><div class=3D""><br class=3D""></div><div class= =3D""><div class=3D""><br class=3D""></div><div class=3D""><br class=3D""><= /div><div class=3D""><br class=3D""></div><div class=3D""><br class=3D""></= div><div class=3D""><div class=3D""><br class=3D""></div><div class=3D""><d= iv><div><br class=3D""></div><br class=3D""></div></div></div></div></div><= /body></html> <p></p> -- <br /> You received this message because you are subscribed to the Google Groups &= quot;zodb" group.<br /> To unsubscribe from this group and stop receiving emails from it, send an e= mail to <a href=3D"mailto:[email protected]">zodb+unsubscri= [email protected]</a>.<br /> For more options, visit <a href=3D"https://groups.google.com/d/optout">http= s://groups.google.com/d/optout</a>.<br /> --Apple-Mail=_877C6F69-43D5-47AA-86D8-3830C7234DC9--