Re: Speed improvement using compile time generate proxies
"Fabio Maulo" <[email protected]> Mon, 1 Sep 2008 09:26:47 -0300
| Newsgroups | gmane.comp.windows.dotnet.nhibernate.devel |
|---|---|
| Message-ID | <[email protected]> |
--===============0098346762== Content-Type: multipart/alternative; boundary="----=_Part_24411_27946154.1220272007350" ------=_Part_24411_27946154.1220272007350 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline And please since NH have a proxyfactoryfactory and ByteCodeProvider try to create separate injectable implementation. Then we can test it in the core and hopeful use the new implementation as default. 2008/9/1 "Andr=E9s G. Aragoneses" <[email protected]> > +1 > > I've had similar performance issues and I would vote for this to be > included in NH Core vNext, so let's start creating a JIRA issue for it, > right? > > Johan, wonderful research! Do you have rough benchmark numbers about the > performance increase? A simple percentage number? > > Regards, > > Andr=E9s > > > Johan Kirsten wrote: > > Greetings > > > > I recently had the opportunity to move a project onto NHibernate. The > > project was using Gentle.NET. After moving the project onto NHibernate, > > I noticed that the application had become slower. For those not familia= r > > with Gentle.NET, one of the biggest differences between Gentle.NET and > > NHibernate is the use of runtime generated proxies. Gentle.NET does not > > make use of runtime generated proxies, but rather method calls. > > > > An example of an NHibernate objects would be: > > > > * public class Invoice { > > private int _id; > > private string _reference; > > private Company _company; > > private IList<InvoiceItem> _invoiceItems;* > > ** > > * public Invoice() { > > }* > > ** > > * public int Id { > > get { return _id; } > > set { _id =3D value; } > > }* > > ** > > * public virtual string Reference { > > get { return _reference; } > > set { _reference =3D value; } > > }* > > ** > > * public virtual Company Company { > > get { return _company; } > > set { _company =3D value; } > > }* > > ** > > * public virtual IList<InvoiceItem> InvoiceItems { > > get { return _invoiceItems; } > > set { _invoiceItems =3D value; } > > } > > }* > > ** > > * public class Company { > > private int _id; > > private string _name;* > > ** > > * public Company() { > > }* > > ** > > * public int Id { > > get { return _id; } > > set { _id =3D value; } > > }* > > ** > > * public string Name { > > get { return _name; } > > set { _name =3D value; } > > } > > }* > > > > The equivalent for Gentle.NET would be: > > > > * public class Invoice { > > private int _id; > > private string _reference; > > private int _companyKey;* > > ** > > * public Invoice() { > > }* > > ** > > * public int Id { > > get { return _id; } > > set { _id =3D value; } > > }* > > ** > > * public virtual string Reference { > > get { return _reference; } > > set { _reference =3D value; } > > }* > > ** > > * public int CompanyKey { > > get { return _companyKey; } > > set { _companyKey =3D value; } > > }* > > ** > > * public Company GetCompany() { > > // Code that executes SQL and return company for current invoi= ce > > }* > > ** > > * public static IList<Invoice> GetList() { > > // Code that executes SQL and returns all invoices > > } > > }* > > ** > > * public class Company { > > private int _id; > > private string _name;* > > ** > > * public Company() { > > }* > > ** > > * public int Id { > > get { return _id; } > > set { _id =3D value; } > > }* > > ** > > * public string Name { > > get { return _name; } > > set { _name =3D value; } > > }* > > ** > > * public IList<Invoice> GetInvoiceList() { > > // Code that executes SQL and returns invoices for current > company > > }* > > ** > > * public static IList<Company> GetList() { > > // Code that executes SQL and returns all companies > > } > > }* > > > > It is important to note that - in general and assuming lazy loading - > > Gentle.NET and NHibernate will execute the same SQL at the same point i= n > > code. For example the Gentle.NET code: > > > > * IList invoiceList =3D Invoice.GetList(); > > foreach (Invoice invoice in invoiceList) { > > Company company =3D invoice.GetCompany();* > > ** > > * // Print invoice and company information > > }* > > > > will request a list of invoices and the company for each invoice. > > NHibernate will do the same: > > > > * IList invoiceList =3D InvoiceDAO.GetList(); > > foreach (Invoice invoice in invoiceList) { > > Company company =3D invoice.Company;* > > ** > > * // Print invoice and company information > > }* > > > > What I found was that the Gentle.NET code was faster. The only > > difference was the runtime generated proxies. > > > > The point I am trying to make is NHibernate takes a performance hit > > because of runtime generated proxies. I investigated and realised that > > the queries that were taking the longest were those retrieving lists of > > complex objects (objects containing other objects). These were the > > objects that used the most runtime generated proxies. I dived into the > > NHibernate code to understand why this is happening. What I discovered > > is that if you query a List of Invoices, NHibernate will compile a prox= y > > for each Company of each row. > > > > To make it clear, for the first row in an Invoice results set, > > NHibernate (using Castle's dynamic proxy) generates a Company proxy typ= e > > and then instantiates the proxy. For the second row, NHibernate again > > generates a Company proxy type and then instantiates the proxy. > > Therefore for a 100 invoices the same Company proxy type will be > > generated 100 times. After investigation I confirmed that this holds > > true for NH 1.2.1 and 2.0, although I suspect that NH 2.0 does not > > suffer as much due to optimization in Castle's new dynamic proxy. > > > > To confirm my suspisions I experimented with NH 1.2.1. I altered the > > code of NHibernate.Proxy.CastleProxyFactory, which is responsible for > > proxy generation. I had it store the type on its first pass and then to > > use the stored type to instantiate each instance. I added the data > member: > > > > * private System.Type _proxyType;* > > > > And altered the GetProxy method from: > > > > * public INHibernateProxy GetProxy(object id, ISessionImplementor > > session) { > > try { > > CastleLazyInitializer initializer =3D new > > CastleLazyInitializer(_persistentClass, id, _getIdentifierMethod, > > _setIdentifierMethod, session);* > > ** > > * object generatedProxy =3D null;* > > ** > > * if (IsClassProxy) { > > generatedProxy =3D > > _proxyGenerator.CreateClassProxy(_persistentClass, _interfaces, > > initializer, false); > > } > > else { > > generatedProxy =3D _proxyGenerator.CreateProxy(_interfaces, > > initializer, new object()); > > }* > > ** > > * initializer._constructed =3D true; > > return (INHibernateProxy)generatedProxy; > > } > > catch (Exception e) { > > log.Error("Creating a proxy instance failed", e); > > throw new HibernateException("Creating a proxy instance > > failed", e); > > } > > }* > > > > to: > > > > * public INHibernateProxy GetProxy(object id, ISessionImplementor > > session) { > > try { > > CastleLazyInitializer initializer =3D new > > CastleLazyInitializer(_persistentClass, id, _getIdentifierMethod, > > _setIdentifierMethod, session);* > > ** > > * object generatedProxy =3D null;* > > ** > > * if (IsClassProxy) { > > if (_proxyType =3D=3D null) > > _proxyType =3D > > _proxyGenerator.ProxyBuilder.CreateClassProxy(_persistentClass, > > _interfaces);* > > ** > > * generatedProxy =3D Activator.CreateInstance(_proxyType, > > initializer); > > } > > else { > > if (_proxyType =3D=3D null) > > _proxyType =3D > > _proxyGenerator.ProxyBuilder.CreateInterfaceProxy(_interfaces, > > typeof(object));* > > ** > > * generatedProxy =3D Activator.CreateInstance(_proxyType, ne= w > > object[] { initializer, new object() }); > > }* > > ** > > * initializer._constructed =3D true; > > return (INHibernateProxy)generatedProxy; > > } > > catch (Exception e) { > > log.Error("Creating a proxy instance failed", e); > > throw new HibernateException("Creating a proxy instance > > failed", e); > > } > > }* > > > > This provided a speed improvement, but this isn't a solution. I do not > > want any delays at runtime due to proxy generation. I wanted to > > translate all of the types at startup. But for a lot of classes this > > would take too long and use too much memory. Then I found compile time > > generated proxies and it fit perfectly. Check out > > http://code.google.com/p/nhibernateproxygenerator/ by WC Pierce. I > > downloaded the source code and altered it to compile multiple libraries > > (my domain is spread over multiple libraries). The library it generated > > contains a ProxyFactoryFactory which you can use by setting the > > "proxyfactory.factory_class" property in your config file: > > > > * <property > > name=3D"proxyfactory.factory_class">StaticProxyFactoryFactory, > > Library.Proxies</property>* > > > > It worked amazingly well. The speed improved significantly. It does tak= e > > a while to compile the proxies. Especially if you have a lot of classes= . > > So I only plan to do this on deployment. I will continue to use runtime > > generated proxies in my debug environment. This solution seems to work > > very well. Please provide any thoughts and comments. I am especially > > interested in hearing any comments from people that can point out any > > serious downside to this approach. Are there specific reasons why > > runtime generated proxies are prefered over compile time generated > > proxies that I am not aware of? > > > > Finally I want to state that I hope that the NHibernate community will > > contemplate making compile time proxy generation a standard feature in > > the next release of NHibernate. > > > > Thanks > > > > Johan Kirsten > > > > > > -----------------------------------------------------------------------= - > > > > -----------------------------------------------------------------------= -- > > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > > Build the coolest Linux based applications with Moblin SDK & win great > prizes > > Grand prize is a trip for two to an Open Source event anywhere in the > world > > http://moblin-contest.org/redirect.php?banner_id=3D100&url=3D/ > > > > > > -----------------------------------------------------------------------= - > > > > _______________________________________________ > > Nhibernate-development mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/nhibernate-development > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > Build the coolest Linux based applications with Moblin SDK & win great > prizes > Grand prize is a trip for two to an Open Source event anywhere in the wor= ld > http://moblin-contest.org/redirect.php?banner_id=3D100&url=3D/ > _______________________________________________ > Nhibernate-development mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/nhibernate-development > --=20 Fabio Maulo ------=_Part_24411_27946154.1220272007350 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline <div dir=3D"ltr">And please since NH have a proxyfactoryfactory and ByteCod= eProvider try to create separate injectable implementation.<br>Then we can = test it in the core and hopeful use the new implementation as default.<br> <br><div class=3D"gmail_quote">2008/9/1 "Andr=E9s G. Aragoneses" = <span dir=3D"ltr"><<a href=3D"mailto:[email protected]">[email protected]<= /a>></span><br><blockquote class=3D"gmail_quote" style=3D"border-left: 1= px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"= > +1<br> <br> I've had similar performance issues and I would vote for this to be<br> included in NH Core vNext, so let's start creating a JIRA issue for it,= <br> right?<br> <br> Johan, wonderful research! Do you have rough benchmark numbers about the<br= > performance increase? A simple percentage number?<br> <br> Regards,<br> <font color=3D"#888888"><br> Andr=E9s<br> </font><div><div></div><div class=3D"Wj3C7c"><br> <br> Johan Kirsten wrote:<br> > Greetings<br> ><br> > I recently had the opportunity to move a project onto NHibernate. The<= br> > project was using Gentle.NET. After moving the project onto NHibernate= ,<br> > I noticed that the application had become slower. For those not famili= ar<br> > with Gentle.NET, one of the biggest differences between Gentle.NET and= <br> > NHibernate is the use of runtime generated proxies. Gentle.NET does no= t<br> > make use of runtime generated proxies, but rather method calls.<br> ><br> > An example of an NHibernate objects would be:<br> ><br> > * public class Invoice {<br> > private int _id;<br> > private string _reference;<br> > private Company _company;<br> > private IList<InvoiceItem> _invoiceItems;*<= br> > **<br> > * public Invoice() {<br> > }*<br> > **<br> > * public int Id {<br> > get { return _id; }<br> > set { _id =3D value; }<br> > }*<br> > **<br> > * public virtual string Reference {<br> > get { return _reference; }<br> > set { _reference =3D value; }<br> > }*<br> > **<br> > * public virtual Company Company {<br> > get { return _company; }<br> > set { _company =3D value; }<br> > }*<br> > **<br> > * public virtual IList<InvoiceItem> InvoiceI= tems {<br> > get { return _invoiceItems; }<br> > set { _invoiceItems =3D value; }<br> > }<br> > }*<br> > **<br> > * public class Company {<br> > private int _id;<br> > private string _name;*<br> > **<br> > * public Company() {<br> > }*<br> > **<br> > * public int Id {<br> > get { return _id; }<br> > set { _id =3D value; }<br> > }*<br> > **<br> > * public string Name {<br> > get { return _name; }<br> > set { _name =3D value; }<br> > }<br> > }*<br> ><br> > The equivalent for Gentle.NET would be:<br> ><br> > * public class Invoice {<br> > private int _id;<br> > private string _reference;<br> > private int _companyKey;*<br> > **<br> > * public Invoice() {<br> > }*<br> > **<br> > * public int Id {<br> > get { return _id; }<br> > set { _id =3D value; }<br> > }*<br> > **<br> > * public virtual string Reference {<br> > get { return _reference; }<br> > set { _reference =3D value; }<br> > }*<br> > **<br> > * public int CompanyKey {<br> > get { return _companyKey; }<br> > set { _companyKey =3D value; }<br> > }*<br> > **<br> > * public Company GetCompany() {<br> > // Code that executes SQL and return= company for current invoice<br> > }*<br> > **<br> > * public static IList<Invoice> GetList() {<b= r> > // Code that executes SQL and return= s all invoices<br> > }<br> > }*<br> > **<br> > * public class Company {<br> > private int _id;<br> > private string _name;*<br> > **<br> > * public Company() {<br> > }*<br> > **<br> > * public int Id {<br> > get { return _id; }<br> > set { _id =3D value; }<br> > }*<br> > **<br> > * public string Name {<br> > get { return _name; }<br> > set { _name =3D value; }<br> > }*<br> > **<br> > * public IList<Invoice> GetInvoiceList() {<b= r> > // Code that executes SQL and return= s invoices for current company<br> > }*<br> > **<br> > * public static IList<Company> GetList() {<b= r> > // Code that executes SQL and return= s all companies<br> > }<br> > }*<br> ><br> > It is important to note that - in general and assuming lazy loading -<= br> > Gentle.NET and NHibernate will execute the same SQL at the same point = in<br> > code. For example the Gentle.NET code:<br> ><br> > * IList invoiceList =3D Invoice.GetList();<br> > foreach (Invoice invoice in invoiceList) {<br> > Company company =3D invoice.GetCompany();*<br> > **<br> > * // Print invoice and company information<br> > }*<br> ><br> > will request a list of invoices and the company for each invoice.<br> > NHibernate will do the same:<br> ><br> > * IList invoiceList =3D InvoiceDAO.GetList();<br> > foreach (Invoice invoice in invoiceList) {<br> > Company company =3D invoice.Company;*<br> > **<br> > * // Print invoice and company information<br> > }*<br> ><br> > What I found was that the Gentle.NET code was faster. The only<br> > difference was the runtime generated proxies.<br> ><br> > The point I am trying to make is NHibernate takes a performance hit<br= > > because of runtime generated proxies. I investigated and realised that= <br> > the queries that were taking the longest were those retrieving lists o= f<br> > complex objects (objects containing other objects). These were the<br> > objects that used the most runtime generated proxies. I dived into the= <br> > NHibernate code to understand why this is happening. What I discovered= <br> > is that if you query a List of Invoices, NHibernate will compile a pro= xy<br> > for each Company of each row.<br> ><br> > To make it clear, for the first row in an Invoice results set,<br> > NHibernate (using Castle's dynamic proxy) generates a Company prox= y type<br> > and then instantiates the proxy. For the second row, NHibernate again<= br> > generates a Company proxy type and then instantiates the proxy.<br> > Therefore for a 100 invoices the same Company proxy type will be<br> > generated 100 times. After investigation I confirmed that this holds<b= r> > true for NH 1.2.1 and 2.0, although I suspect that NH 2.0 does not<br> > suffer as much due to optimization in Castle's new dynamic proxy.<= br> ><br> > To confirm my suspisions I experimented with NH <a href=3D"http://1.2.= 1." target=3D"_blank">1.2.1.</a> I altered the<br> > code of NHibernate.Proxy.CastleProxyFactory, which is responsible for<= br> > proxy generation. I had it store the type on its first pass and then t= o<br> > use the stored type to instantiate each instance. I added the data mem= ber:<br> ><br> > * private System.Type _proxyType;*<br> ><br> > And altered the GetProxy method from:<br> ><br> > * public INHibernateProxy GetProxy(object id, ISessionImplement= or<br> > session) {<br> > try {<br> > CastleLazyInitializer initializer = =3D new<br> > CastleLazyInitializer(_persistentClass, id, _getIdentifierMethod,<br> > _setIdentifierMethod, session);*<br> > **<br> > * object generatedProxy =3D null;*<br> > **<br> > * if (IsClassProxy) {<br> > generatedProxy =3D<br> > _proxyGenerator.CreateClassProxy(_persistentClass, _interfaces,<br> > initializer, false);<br> > }<br> > else {<br> > generatedProxy =3D _proxyGen= erator.CreateProxy(_interfaces,<br> > initializer, new object());<br> > }*<br> > **<br> > * initializer._constructed =3D true;<br> > return (INHibernateProxy)generatedPr= oxy;<br> > }<br> > catch (Exception e) {<br> > log.Error("Creating a proxy ins= tance failed", e);<br> > throw new HibernateException("C= reating a proxy instance<br> > failed", e);<br> > }<br> > }*<br> ><br> > to:<br> ><br> > * public INHibernateProxy GetProxy(object id, ISessionImplement= or<br> > session) {<br> > try {<br> > CastleLazyInitializer initializer = =3D new<br> > CastleLazyInitializer(_persistentClass, id, _getIdentifierMethod,<br> > _setIdentifierMethod, session);*<br> > **<br> > * object generatedProxy =3D null;*<br> > **<br> > * if (IsClassProxy) {<br> > if (_proxyType =3D=3D null)<= br> > _proxyType =3D<= br> > _proxyGenerator.ProxyBuilder.CreateClassProxy(_persistentClass,<br> > _interfaces);*<br> > **<br> > * generatedProxy =3D Activato= r.CreateInstance(_proxyType,<br> > initializer);<br> > }<br> > else {<br> > if (_proxyType =3D=3D null)<= br> > _proxyType =3D<= br> > _proxyGenerator.ProxyBuilder.CreateInterfaceProxy(_interfaces,<br> > typeof(object));*<br> > **<br> > * generatedProxy =3D Activato= r.CreateInstance(_proxyType, new<br> > object[] { initializer, new object() });<br> > }*<br> > **<br> > * initializer._constructed =3D true;<br> > return (INHibernateProxy)generatedPr= oxy;<br> > }<br> > catch (Exception e) {<br> > log.Error("Creating a proxy ins= tance failed", e);<br> > throw new HibernateException("C= reating a proxy instance<br> > failed", e);<br> > }<br> > }*<br> ><br> > This provided a speed improvement, but this isn't a solution. I do= not<br> > want any delays at runtime due to proxy generation. I wanted to<br> > translate all of the types at startup. But for a lot of classes this<b= r> > would take too long and use too much memory. Then I found compile time= <br> > generated proxies and it fit perfectly. Check out<br> > <a href=3D"http://code.google.com/p/nhibernateproxygenerator/" target= =3D"_blank">http://code.google.com/p/nhibernateproxygenerator/</a> by WC Pi= erce. I<br> > downloaded the source code and altered it to compile multiple librarie= s<br> > (my domain is spread over multiple libraries). The library it generate= d<br> > contains a ProxyFactoryFactory which you can use by setting the<br> > "proxyfactory.factory_class" property in your config file:<b= r> ><br> > * <property<br> > name=3D"proxyfactory.factory_class">StaticProxyFactoryFac= tory,<br> > Library.Proxies</property>*<br> ><br> > It worked amazingly well. The speed improved significantly. It does ta= ke<br> > a while to compile the proxies. Especially if you have a lot of classe= s.<br> > So I only plan to do this on deployment. I will continue to use runtim= e<br> > generated proxies in my debug environment. This solution seems to work= <br> > very well. Please provide any thoughts and comments. I am especially<b= r> > interested in hearing any comments from people that can point out any<= br> > serious downside to this approach. Are there specific reasons why<br> > runtime generated proxies are prefered over compile time generated<br> > proxies that I am not aware of?<br> ><br> > Finally I want to state that I hope that the NHibernate community will= <br> > contemplate making compile time proxy generation a standard feature in= <br> > the next release of NHibernate.<br> ><br> > Thanks<br> ><br> > Johan Kirsten<br> ><br> ><br> </div></div>> ----------------------------------------------------------= --------------<br> <div><div></div><div class=3D"Wj3C7c">><br> > ----------------------------------------------------------------------= ---<br> > This SF.Net email is sponsored by the Moblin Your Move Developer's= challenge<br> > Build the coolest Linux based applications with Moblin SDK & win g= reat prizes<br> > Grand prize is a trip for two to an Open Source event anywhere in the = world<br> > <a href=3D"http://moblin-contest.org/redirect.php?banner_id=3D100&= url=3D/" target=3D"_blank">http://moblin-contest.org/redirect.php?banner_id= =3D100&url=3D/</a><br> ><br> ><br> > ----------------------------------------------------------------------= --<br> ><br> > _______________________________________________<br> > Nhibernate-development mailing list<br> > <a href=3D"mailto:[email protected]">Nhiber= [email protected]</a><br> > <a href=3D"https://lists.sourceforge.net/lists/listinfo/nhibernate-dev= elopment" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/nh= ibernate-development</a><br> <br> <br> -------------------------------------------------------------------------<b= r> This SF.Net email is sponsored by the Moblin Your Move Developer's chal= lenge<br> Build the coolest Linux based applications with Moblin SDK & win great = prizes<br> Grand prize is a trip for two to an Open Source event anywhere in the world= <br> <a href=3D"http://moblin-contest.org/redirect.php?banner_id=3D100&url= =3D/" target=3D"_blank">http://moblin-contest.org/redirect.php?banner_id=3D= 100&url=3D/</a><br> _______________________________________________<br> Nhibernate-development mailing list<br> <a href=3D"mailto:[email protected]">Nhibernate-= [email protected]</a><br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/nhibernate-developm= ent" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/nhibern= ate-development</a><br> </div></div></blockquote></div><br><br clear=3D"all"><br>-- <br>Fabio Maulo= <br> </div> ------=_Part_24411_27946154.1220272007350-- --===============0098346762== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ --===============0098346762== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Nhibernate-development mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nhibernate-development --===============0098346762==--