Re: [patch] add CLOB datatype
anthony berglas <[email protected]> Fri, 27 Apr 2012 14:39:18 +1000
| Newsgroups | gmane.comp.java.orm.simpleorm |
|---|---|
| Message-ID | <CA+_PZMfHx=Jzuz9_P5UYqmeTNQ7s2+JiK7yoDAgDZWOqWRM=Pg@mail.gmail.com> |
--bcaec5171e876d7cbd04bea1b2d3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello Noel, Thanks for providing the patch, but we really do need a decent test case for all changes. (Not sure what HSQL does with Clobs, but if not supported then the test case should run properly on at least one database!). As it is, Franck will have spent more time on it than the original authorship, for a feature that no one is currently requiring. Remember that people can always use raw SQL for odd cases, and a goal is to keep SimpleOrm simple. And B/Clobs have extra issues that we should be able to retrieve them as a single string/byte array, or as a stream. As to PostgreSql, there is a fair chance that accessing an ordinary text field as a Clob should work, remember that the Java type need not be the same as the SQL type in JDBC. Anthony On Thu, Apr 26, 2012 at 10:55 PM, Franck Routier <[email protected]>wrote: > Ok. So I'll fix this. > > What I have done is : > 1) allow user to choose a maxAutoRetrieveSize, to (eventually) avoid > getting all content when building the SRecordInstance > 2) tweak Prostgresql, Cach=E9DB and Mysql driver (to get the right sql > type, eg TEXT or LONGVARCHAR) > 3) add a test in DataTypeTest > > Notice : > 1) Postgresql does not really support Clobs. > This would map to TEXT fields, but the streaming part is not implemented, > and the whole text will be in the resultset anyway. > 2) I was not able to find a way to allow interaction with the Clob object= . > Maybe with another SFieldXXXX that would have an internal type > java.sql.Clob (but then you could only interact while within the > transaction as I understand it). > Anyway, it's probably not very useful for now. > > I have added a test case, but I have no Oracle database to test it now. > Anyone willing to test is welcome. > > Regards, > Franck > > Le 26/04/2012 11:41, Noel Grandin a =E9crit : > > You know, you are right, that loop does look dodgy. > To be honest, my code only briefly used CLOB's and then I moved onto > something else, so that code is no longer in use here. > > On 2012-04-24 12:18, Franck Routier wrote: > > Ok. > But, then maybe we should give the user (I mean the developper using > Simpleorm) an access to the Clob object, instead of fetching its value on > queryFieldValue ? > And maybe add a custom method to SFieldClob that would effectively fetch > the value, but not on queryFieldValue ? > > > Another point is, did this code work ? I'm asking because the break > condition in queryFieldValue seems erroneous, as reader.read(buf) will > return -1 on end of file, not 0. ? > > Franck > > Le 24/04/2012 11:12, Noel Grandin a =E9crit : > > No good reason, just that it would be a little surprising performance wis= e > - could trigger a very long fetch from the server that is likely to be > uncached. > > But I don't have a principled objection to it. > > On 2012-04-24 09:12, Franck Routier wrote: > > > > Hi No=EBl and Anthony, > > I've been looking at your code, and I was wondering why Clobs wouldn't be > comparable... What is the reason behind that ? > Typically, Postgresql type for Clobs (TEXT) would just compare fine, as > well as java representation of a Clob (a String). > Is there any Oracle specificity around that, or simply a size problem ?? > > Thanks, > Franck > > > > > ------------------------------ > Disclaimer: http://www.peralex.com/disclaimer.html > > > > > > ------------------------------ > Disclaimer: http://www.peralex.com/disclaimer.html > > > --=20 Dr Anthony Berglas, [email protected] Mobile: +61 4 4838 8874 Just because it is possible to push twigs along the ground with ones nose does not necessarily mean that that is the best way to collect firewood. --bcaec5171e876d7cbd04bea1b2d3 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <head> <style type=3D"text/css"> <!-- /* start of attachment style */ .ygrp-photo-title{ clear: both; font-size: smaller; height: 15px; overflow: hidden; text-align: center; width: 75px; } div.ygrp-photo{ background-position: center; background-repeat: no-repeat; background-color: white; border: 1px solid black; height: 62px; width: 62px; } div.photo-title=20 a, div.photo-title a:active, div.photo-title a:hover, div.photo-title a:visited { text-decoration: none;=20 } div.attach-table div.attach-row { clear: both; } div.attach-table div.attach-row div { float: left; /* margin: 2px;*/ } p { clear: both; padding: 15px 0 3px 0; overflow: hidden; } div.ygrp-file { width: 30px; valign: middle; } div.attach-table div.attach-row div div a { text-decoration: none; } div.attach-table div.attach-row div div span { font-weight: normal; } div.ygrp-file-title { font-weight: bold; } /* end of attachment style */ --> </style> </head> <html> <head> <style type=3D"text/css"> <!-- #ygrp-mkp { border: 1px solid #d8d8d8; font-family: Arial; margin: 10px 0; padding: 0 10px; } #ygrp-mkp hr { border: 1px solid #d8d8d8; } #ygrp-mkp #hd { color: #628c2a; font-size: 85%; font-weight: 700; line-height: 122%; margin: 10px 0; } #ygrp-mkp #ads { margin-bottom: 10px; } #ygrp-mkp .ad { padding: 0 0; } #ygrp-mkp .ad p { margin: 0; } #ygrp-mkp .ad a { color: #0000ff; text-decoration: none; } --> </style> </head> <body> <div class=3D"gmail_extra">Hello Noel,</div><div class=3D"gmail_extra"><br>= </div><div class=3D"gmail_extra">Thanks for providing the patch, but we rea= lly do need a decent test case for all changes. =A0(Not sure what HSQL does= with Clobs, but if not supported then the test case should run properly on= at least one database!).</div> <div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">As it is, F= ranck will have spent more time on it than the original authorship, for a f= eature that no one is currently requiring. =A0Remember that people can alwa= ys use raw SQL for odd cases, and a goal is to keep SimpleOrm simple. =A0An= d B/Clobs have extra issues that we should be able to retrieve them as a si= ngle string/byte array, or as a stream.</div> <div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">As to Postg= reSql, there is a fair chance that accessing an ordinary text field as a Cl= ob should work, remember that the Java type need not be the same as the SQL= type in JDBC.</div> <div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">Anthony<br>= <br><div class=3D"gmail_quote">On Thu, Apr 26, 2012 at 10:55 PM, Franck Rou= tier <span dir=3D"ltr"><<a href=3D"mailto:[email protected]" targ= et=3D"_blank">[email protected]</a>></span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> =20=20 =20=20=20=20 =20=20 <div bgcolor=3D"#FFFFFF" text=3D"#000000"> Ok. So I'll fix this.<br> <br> What I have done is :<br> 1) allow user to choose a maxAutoRetrieveSize, to (eventually) avoid getting all content when building the SRecordInstance<br> 2)=A0 tweak Prostgresql, Cach=E9DB and Mysql driver (to get the right sql type, eg TEXT or LONGVARCHAR)<br> 3) add a test in DataTypeTest<br> <br> Notice :<br> 1) Postgresql does not really support Clobs.<br> This would map to TEXT fields, but the streaming part is not implemented, and the whole text will be in the resultset anyway.<br> 2) I was not able to find a way to allow interaction with the Clob object.<br> Maybe with another SFieldXXXX that would have an internal type java.sql.Clob (but then you could only interact while within the transaction as I understand it).<br> Anyway, it's probably not very useful for now.<br> <br> I have added a test case, but I have no Oracle database to test it now. Anyone willing to test is welcome.<br> <br> Regards,<br> Franck<br> <br> Le 26/04/2012 11:41, Noel Grandin a =E9crit=A0: <div><div class=3D"h5"><blockquote type=3D"cite"> =20=20=20=20=20=20 You know, you are right, that loop does look dodgy. <br> To be honest, my code only briefly used CLOB's and then I moved onto something else, so that code is no longer in use here.<br> <br> On 2012-04-24 12:18, Franck Routier wrote: <blockquote type=3D"cite"> =20=20=20=20=20=20=20=20 Ok. <br> But, then maybe we should give the user (I mean the developper using Simpleorm) an access to the Clob object, instead of fetching its value on queryFieldValue ?<br> And maybe add a custom method to SFieldClob that would effectively fetch the value, but not on queryFieldValue ?<br> <br> <br> Another point is, did this code work ? I'm asking because the break condition in queryFieldValue seems erroneous, as reader.read(buf) will return -1 on end of file, not 0. ?<br> <br> Franck <br> <br> Le 24/04/2012 11:12, Noel Grandin a =E9crit=A0: <blockquote type=3D"cite"> =20=20=20=20=20=20=20=20=20=20 No good reason, just that it would be a little surprising performance wise - could trigger a very long fetch from the server that is likely to be uncached.<br> <br> But I don't have a principled objection to it.<br> <br> On 2012-04-24 09:12, Franck Routier wrote: <blockquote type=3D"cite"> <span>=A0</span> =20=20=20=20=20=20=20=20=20=20=20=20 <div> <div> =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 <div> <p> Hi No=EBl and Anthony,<br> <br> I've been looking at your code, and I was wondering why Clobs wouldn't be comparable... What is the reason behind that ?<br> Typically, Postgresql type for Clobs (TEXT) would just compare fine, as well as java representation of a Clob (a String).<br> Is there any Oracle specificity around that, or simply a size problem ??<br> <br> Thanks,<br> Franck<br> <br> </p> </div> </div> </div> =20=20=20=20=20=20=20=20=20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20 </blockquote> <br> <br> <br> <hr><font color=3D"808080" size=3D"-2">Disclaimer: <a href=3D"htt= p://www.peralex.com/disclaimer.html" target=3D"_blank">http://www.peralex.c= om/disclaimer.html</a><br> <br> </font></blockquote> <br> </blockquote> <br> <br> <br> <hr><font color=3D"808080" size=3D"-2">Disclaimer: <a href=3D"http://= www.peralex.com/disclaimer.html" target=3D"_blank">http://www.peralex.com/d= isclaimer.html</a><br> <br> </font></blockquote> <br> </div></div></div> </blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><p> </p><p>Dr Anthony Berglas, <a href=3D"mailto:[email protected]" target=3D= "_blank">[email protected]</a>=A0=A0=A0=A0=A0=A0 Mobile: +61 4 4838 8874<= br> Just because it is possible to push twigs along the ground with ones nose<b= r> does not necessarily mean that that is the best way to collect firewood.</p= ><br> </div> <!-- |**|begin egp html banner|**| --> <br> =20=20=20=20 =20=20=20=20 <br> <!-- |**|end egp html banner|**| --> <div width=3D"1" style=3D"color: white; clear: both;"/>__._,_.___</div> <!-- Start Recommendations --> <!-- End Recommendations --> <!-- |**|begin egp html banner|**| --> <img src=3D"http://geo.yahoo.com/serv?s=3D97476590/grpId=3D7360226/grpspI= d=3D1705006905/msgId=3D2057/stime=3D1335501603" width=3D"1" height=3D"1"> <= br> <!-- |**|end egp html banner|**| --> =20=20 <!-- |**|begin egp html banner|**| --> <br> <div style=3D"font-family: verdana; font-size: 77%; border-top: 1px s= olid #666; padding: 5px 0;" > Your email settings: Individual Email|Traditional <br> <a href=3D"http://groups.yahoo.com/group/SimpleORM/join;_ylc=3DX3oDMT= JmNGQxbjg1BF9TAzk3NDc2NTkwBGdycElkAzczNjAyMjYEZ3Jwc3BJZAMxNzA1MDA2OTA1BHNlY= wNmdHIEc2xrA3N0bmdzBHN0aW1lAzEzMzU1MDE2MDM-">Change settings via the Web</a= > (Yahoo! ID required) <br> Change settings via email: <a href=3D"mailto:SimpleORM-digest@yahoogr= oups.com?subject=3DEmail Delivery: Digest">Switch delivery to Daily Digest<= /a> | <a href =3D "mailto:[email protected]?subject=3D= Change Delivery Format: Fully Featured">Switch to Fully Featured</a> <br> <a href=3D"http://groups.yahoo.com/group/SimpleORM;_ylc=3DX3oDMT= JkMDM4ZDgyBF9TAzk3NDc2NTkwBGdycElkAzczNjAyMjYEZ3Jwc3BJZAMxNzA1MDA2OTA1BHNlY= wNmdHIEc2xrA2hwZgRzdGltZQMxMzM1NTAxNjAz"> Visit Your Group=20 </a> | <a href=3D"http://docs.yahoo.com/info/terms/"> Yahoo! Groups Terms of Use </a> | <a href=3D"mailto:[email protected]?subject=3DUns= ubscribe"> Unsubscribe=20 </a>=20 <br> </div> <br> <!-- |**|end egp html banner|**| --> <div style=3D"color: white; clear: both;"/>__,_._,___</div> </body> </html> --bcaec5171e876d7cbd04bea1b2d3--