Re: Implementation PrevaylerEight

--===============2052170017046770278==
Content-Type: multipart/alternative; boundary=001a1140a8c60d91200505f4ffd2

--001a1140a8c60d91200505f4ffd2
Content-Type: text/plain; charset=UTF-8

Hi Christian,

Thanks for that, Ill try and get it running :)

Best regards,
Marcel

ps.: Im running on an AMD Phenom II X4 960T processor @ 3.00 GHz, 16GB Ram
and an old Seagate ST500DM002
That HD is the achilles heel of my system and Ill be replacing it soon with
a samsung 840 pro ssd. :)



On Tue, Oct 21, 2014 at 10:20 PM, Christian Stein <[email protected]>
wrote:

> Sure, Marcel. Here you go.
>
> Gonna upload a full source package later this week.
>
> Cheers,
> Christian
>
>
> public class ByteBufferOutputStream extends OutputStream {
>
>   private final ByteBuffer buffer;
>
>   public ByteBufferOutputStream(ByteBuffer buffer) {
>     this.buffer = buffer;
>   }
>
>   public void write(int b) {
>     buffer.put((byte) b);
>   }
>
>   public void write(byte[] bytes, int off, int len) {
>     buffer.put(bytes, off, len);
>   }
>
> }
>
>
> public class ByteBufferInputStream extends InputStream {
>
>   private final ByteBuffer buffer;
>
>   public ByteBufferInputStream(ByteBuffer buffer) {
>     this.buffer = buffer;
>   }
>
>   public int read() {
>     if (!buffer.hasRemaining())
>       return -1;
>
>     return buffer.get();
>   }
>
>   public int read(byte[] bytes, int off, int len) {
>     len = Math.min(len, buffer.remaining());
>     buffer.get(bytes, off, len);
>     return len;
>   }
>
> }
>
>
>
> On Tue, Oct 21, 2014 at 10:14 PM, Klaus Wuestefeld <[email protected]> wrote:
>
>> Nice graph!
>>
>> On Tue, Oct 21, 2014 at 5:25 PM, Marcel Oerlemans <
>> [email protected]> wrote:
>>
>>> Hi Christian,
>>>
>>> Those numbers are impressive!
>>> I see you tested default Prevayler up to only 8 threads. You might want
>>> to scale that up a bit?
>>>
>>> After I read your mail, I ran a prevayler scalability test, from 1 to
>>> 1500 threads, 20 seconds per round/
>>> In my test setup Prevayler handled 60 transactions/second @ 8 threads,
>>> but at 1489 threads, Prevayler could handle 5945 transactions / second.
>>> That's ~100 times as many. Results in graph form can be found here:
>>> http://imgur.com/sRIaCVD
>>>
>>> I wanted to do a similar test with Prevayler8, but I see now I need two
>>> classes which I cant seem to find: ByteBufferOutputStream
>>> and ByteBufferInputStream. Can you help me with that?
>>>
>>> Best regards,
>>> Marcel Oerlemans
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Oct 16, 2014 at 12:13 PM, Christian Stein <[email protected]>
>>> wrote:
>>>
>>>> Hi folks,
>>>>
>>>> I recently found some spare time and made the scalability test use
>>>> the PrevaylerEight implementation. See [0] for the outline of its source. A
>>>> single-system-wide lock facade prevayler implementation is used for
>>>> synchronization, at the moment.
>>>>
>>>> The initial results are promising in terms of performance and
>>>> consistency:
>>>>
>>>>    - Default Prevayler: 1511,00 operations/second (8 threads)
>>>>    - PrevaylerEight implementation: 89189,50 operations/second (2
>>>>    threads)
>>>>
>>>>
>>>> These microbenchmarks are, especially when the consistency checks kicks
>>>> in, very garbage collection dependent. I had to set
>>>> the TransactionTestCheckConsistency to NO, because replaying more the
>>>> 1.000.000 transactions into the prevalent system would still be running...
>>>> Ran the benchmark with -verbose:gc and saw after an hour that every 10
>>>> deserialized transactions a full GC.
>>>>
>>>> So, I had to lower the round duration from 20 to 2 seconds. This change
>>>> makes the results even more dependent on the gc, jit, et al. But, now some
>>>> results are shown. See console output at [0] and [1]. Query runs and JDBC
>>>> runs are switched off.
>>>>
>>>> Next time, I'll integrate Caliper[3] to show more details.
>>>>
>>>> Cheers,
>>>> Christian
>>>>
>>>> [0] PrevaylerEight.java
>>>> https://gist.github.com/sormuras/3fea6f6ade327b2a4249
>>>> [1] Result PrevaylerImpl
>>>> https://gist.github.com/sormuras/ef279705d5838a757964
>>>> [2] Result PrevaylerEight
>>>> https://gist.github.com/sormuras/6f61c6af2066d5f8bf87
>>>> [3] Caliper https://code.google.com/p/caliper/
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Comprehensive Server Monitoring with Site24x7.
>>>> Monitor 10 servers for $9/Month.
>>>> Get alerted through email, SMS, voice calls or mobile push
>>>> notifications.
>>>> Take corrective actions from your mobile device.
>>>> http://p.sf.net/sfu/Zoho
>>>> _______________________________________________
>>>> To unsubscribe go to the end of this page:
>>>> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>>>> _______________________________________________
>>>> "Databases in Memoriam" -- http://www.prevayler.org
>>>>
>>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Comprehensive Server Monitoring with Site24x7.
>>> Monitor 10 servers for $9/Month.
>>> Get alerted through email, SMS, voice calls or mobile push notifications.
>>> Take corrective actions from your mobile device.
>>> http://p.sf.net/sfu/Zoho
>>> _______________________________________________
>>> To unsubscribe go to the end of this page:
>>> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>>> _______________________________________________
>>> "Databases in Memoriam" -- http://www.prevayler.org
>>>
>>>
>>
>>
>> --
>> Valeu, Klaus.
>>
>>
>> ------------------------------------------------------------------------------
>> Comprehensive Server Monitoring with Site24x7.
>> Monitor 10 servers for $9/Month.
>> Get alerted through email, SMS, voice calls or mobile push notifications.
>> Take corrective actions from your mobile device.
>> http://p.sf.net/sfu/Zoho
>> _______________________________________________
>> To unsubscribe go to the end of this page:
>> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>> _______________________________________________
>> "Databases in Memoriam" -- http://www.prevayler.org
>>
>>
>
>
> ------------------------------------------------------------------------------
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push notifications.
> Take corrective actions from your mobile device.
> http://p.sf.net/sfu/Zoho
> _______________________________________________
> To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>
>

--001a1140a8c60d91200505f4ffd2
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Christian,<div><br></div><div>Thanks for that, Ill try =
and get it running :)</div><div><br></div><div>Best regards,</div><div>Marc=
el</div><div><br></div><div>ps.: Im running on an AMD Phenom II X4 960T pro=
cessor @ 3.00 GHz, 16GB Ram and an old Seagate=C2=A0ST500DM002</div><div>Th=
at HD is the achilles heel of my system and Ill be replacing it soon with a=
 samsung 840 pro ssd. :)</div><div><br></div><div><br></div></div><div clas=
s=3D"gmail_extra"><br><div class=3D"gmail_quote">On Tue, Oct 21, 2014 at 10=
:20 PM, Christian Stein <span dir=3D"ltr">&lt;<a href=3D"mailto:sormuras@gm=
ail.com" target=3D"_blank">[email protected]</a>&gt;</span> wrote:<br><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c=
cc solid;padding-left:1ex"><div dir=3D"ltr"><div>Sure, Marcel. Here you go.=
</div><div><br></div><div>Gonna upload a full source package later this wee=
k.</div><div><br></div><div>Cheers,</div><div>Christian</div><div><br></div=
><div><br></div><div>public class ByteBufferOutputStream extends OutputStre=
am {</div><div><br></div><div>=C2=A0 private final ByteBuffer buffer;</div>=
<div><br></div><div>=C2=A0 public ByteBufferOutputStream(ByteBuffer buffer)=
 {</div><div>=C2=A0 =C2=A0 this.buffer =3D buffer;</div><div>=C2=A0 }</div>=
<div><br></div><div>=C2=A0 public void write(int b) {</div><div>=C2=A0 =C2=
=A0 buffer.put((byte) b);</div><div>=C2=A0 }</div><div><br></div><div>=C2=
=A0 public void write(byte[] bytes, int off, int len) {</div><div>=C2=A0 =
=C2=A0 buffer.put(bytes, off, len);</div><div>=C2=A0 }</div><div><br></div>=
<div>}</div><div><br></div><div><div><br></div><div>public class ByteBuffer=
InputStream extends InputStream {</div><div><br></div><div>=C2=A0 private f=
inal ByteBuffer buffer;</div><div><br></div><div>=C2=A0 public ByteBufferIn=
putStream(ByteBuffer buffer) {</div><div>=C2=A0 =C2=A0 this.buffer =3D buff=
er;</div><div>=C2=A0 }</div><div><br></div><div>=C2=A0 public int read() {<=
/div><div>=C2=A0 =C2=A0 if (!buffer.hasRemaining())</div><div>=C2=A0 =C2=A0=
 =C2=A0 return -1;</div><div><br></div><div>=C2=A0 =C2=A0 return buffer.get=
();</div><div>=C2=A0 }</div><div><br></div><div>=C2=A0 public int read(byte=
[] bytes, int off, int len) {</div><div>=C2=A0 =C2=A0 len =3D=C2=A0Math.min=
(len, buffer.remaining());</div><div>=C2=A0 =C2=A0 buffer.get(bytes, off, l=
en);</div><div>=C2=A0 =C2=A0 return len;</div><div>=C2=A0 }</div><div><br><=
/div><div>}</div></div><div><br></div><div><br></div></div><div class=3D"HO=
EnZb"><div class=3D"h5"><div class=3D"gmail_extra"><br><div class=3D"gmail_=
quote">On Tue, Oct 21, 2014 at 10:14 PM, Klaus Wuestefeld <span dir=3D"ltr"=
>&lt;<a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>=
&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0=
 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Nice =
graph!<br></div><div class=3D"gmail_extra"><div><div><br><div class=3D"gmai=
l_quote">On Tue, Oct 21, 2014 at 5:25 PM, Marcel Oerlemans <span dir=3D"ltr=
">&lt;<a href=3D"mailto:[email protected]" target=3D"_blank">marcel=
[email protected]</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quo=
te" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"=
><div dir=3D"ltr">Hi Christian,<div><br></div><div>Those numbers are impres=
sive!</div><div>I see you tested default Prevayler up to only 8 threads. Yo=
u might want to scale that up a bit?=C2=A0</div><div><br></div><div>After I=
 read your mail, I ran a prevayler scalability test, from 1 to 1500 threads=
, 20 seconds per round/</div><div>In my test setup Prevayler handled 60 tra=
nsactions/second @ 8 threads, but at 1489 threads, Prevayler could handle 5=
945 transactions / second. That&#39;s ~100 times as many. Results in graph =
form can be found here:=C2=A0<a href=3D"http://imgur.com/sRIaCVD" target=3D=
"_blank">http://imgur.com/sRIaCVD</a></div><div><br></div><div>I wanted to =
do a similar test with Prevayler8, but I see now I need two classes which I=
 cant seem to find:=C2=A0ByteBufferOutputStream and=C2=A0ByteBufferInputStr=
eam. Can you help me with that?<br></div><div><br></div><div>Best regards,<=
/div><div>Marcel Oerlemans</div><div><br></div><div><br></div><div><br></di=
v><div><br></div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_q=
uote">On Thu, Oct 16, 2014 at 12:13 PM, Christian Stein <span dir=3D"ltr">&=
lt;<a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]=
om</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"marg=
in:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"=
>Hi folks,<div><br></div><div>I recently found some spare time and made the=
 scalability test use the=C2=A0PrevaylerEight implementation. See [0] for t=
he outline of its source. A single-system-wide lock facade prevayler implem=
entation is used for synchronization, at the moment.</div><div><br></div><d=
iv>The initial results are promising in terms of performance and consistenc=
y:</div><div><ul><li>Default Prevayler: 1511,00 operations/second (8 thread=
s)</li><li>PrevaylerEight implementation: 89189,50 operations/second (2 thr=
eads)</li></ul></div><div><br></div><div>These microbenchmarks are, especia=
lly when the consistency checks kicks in, very garbage collection dependent=
. I had to set the=C2=A0TransactionTestCheckConsistency to NO, because repl=
aying more the 1.000.000 transactions into the prevalent system would still=
 be running... Ran the benchmark with=C2=A0<span style=3D"color:rgb(0,0,0);=
font-family:Consolas,Menlo,Monaco,&#39;Lucida Console&#39;,&#39;Liberation =
Mono&#39;,&#39;DejaVu Sans Mono&#39;,&#39;Bitstream Vera Sans Mono&#39;,&#3=
9;Courier New&#39;,monospace,serif;font-size:14px;line-height:17.8048000335=
693px;white-space:pre-wrap;background-color:rgb(238,238,238)">-verbose:gc</=
span>=C2=A0and saw after an hour that every 10 deserialized transactions a =
full GC.=C2=A0</div><div><br></div><div>So, I had to lower the round durati=
on from 20 to 2 seconds. This change makes the results even more dependent =
on the gc, jit, et al. But, now some results are shown. See console output =
at [0] and [1]. Query runs and JDBC runs are switched off.</div><div><br></=
div><div>Next time, I&#39;ll integrate Caliper[3] to show more details.=C2=
=A0</div><div><br></div><div>Cheers,</div><div>Christian</div><div><br></di=
v><div>[0] PrevaylerEight.java <a href=3D"https://gist.github.com/sormuras/=
3fea6f6ade327b2a4249" target=3D"_blank">https://gist.github.com/sormuras/3f=
ea6f6ade327b2a4249</a></div><div>[1] Result PrevaylerImpl=C2=A0<a href=3D"h=
ttps://gist.github.com/sormuras/ef279705d5838a757964" target=3D"_blank">htt=
ps://gist.github.com/sormuras/ef279705d5838a757964</a></div><div>[2] Result=
 PrevaylerEight=C2=A0<a href=3D"https://gist.github.com/sormuras/6f61c6af20=
66d5f8bf87" target=3D"_blank">https://gist.github.com/sormuras/6f61c6af2066=
d5f8bf87</a></div><div>[3] Caliper <a href=3D"https://code.google.com/p/cal=
iper/" target=3D"_blank">https://code.google.com/p/caliper/</a></div></div>
<br>-----------------------------------------------------------------------=
-------<br>
Comprehensive Server Monitoring with Site24x7.<br>
Monitor 10 servers for $9/Month.<br>
Get alerted through email, SMS, voice calls or mobile push notifications.<b=
r>
Take corrective actions from your mobile device.<br>
<a href=3D"http://p.sf.net/sfu/Zoho" target=3D"_blank">http://p.sf.net/sfu/=
Zoho</a><br>_______________________________________________<br>
To unsubscribe go to the end of this page: <a href=3D"http://lists.sourcefo=
rge.net/lists/listinfo/prevayler-discussion" target=3D"_blank">http://lists=
.sourceforge.net/lists/listinfo/prevayler-discussion</a><br>
_______________________________________________<br>
&quot;Databases in Memoriam&quot; -- <a href=3D"http://www.prevayler.org" t=
arget=3D"_blank">http://www.prevayler.org</a><br>
<br></blockquote></div><br></div>
<br>-----------------------------------------------------------------------=
-------<br>
Comprehensive Server Monitoring with Site24x7.<br>
Monitor 10 servers for $9/Month.<br>
Get alerted through email, SMS, voice calls or mobile push notifications.<b=
r>
Take corrective actions from your mobile device.<br>
<a href=3D"http://p.sf.net/sfu/Zoho" target=3D"_blank">http://p.sf.net/sfu/=
Zoho</a><br>_______________________________________________<br>
To unsubscribe go to the end of this page: <a href=3D"http://lists.sourcefo=
rge.net/lists/listinfo/prevayler-discussion" target=3D"_blank">http://lists=
.sourceforge.net/lists/listinfo/prevayler-discussion</a><br>
_______________________________________________<br>
&quot;Databases in Memoriam&quot; -- <a href=3D"http://www.prevayler.org" t=
arget=3D"_blank">http://www.prevayler.org</a><br>
<br></blockquote></div><br><br clear=3D"all"><br></div></div><span><font co=
lor=3D"#888888">-- <br>Valeu, Klaus.
</font></span></div>
<br>-----------------------------------------------------------------------=
-------<br>
Comprehensive Server Monitoring with Site24x7.<br>
Monitor 10 servers for $9/Month.<br>
Get alerted through email, SMS, voice calls or mobile push notifications.<b=
r>
Take corrective actions from your mobile device.<br>
<a href=3D"http://p.sf.net/sfu/Zoho" target=3D"_blank">http://p.sf.net/sfu/=
Zoho</a><br>_______________________________________________<br>
To unsubscribe go to the end of this page: <a href=3D"http://lists.sourcefo=
rge.net/lists/listinfo/prevayler-discussion" target=3D"_blank">http://lists=
.sourceforge.net/lists/listinfo/prevayler-discussion</a><br>
_______________________________________________<br>
&quot;Databases in Memoriam&quot; -- <a href=3D"http://www.prevayler.org" t=
arget=3D"_blank">http://www.prevayler.org</a><br>
<br></blockquote></div><br></div>
</div></div><br>-----------------------------------------------------------=
-------------------<br>
Comprehensive Server Monitoring with Site24x7.<br>
Monitor 10 servers for $9/Month.<br>
Get alerted through email, SMS, voice calls or mobile push notifications.<b=
r>
Take corrective actions from your mobile device.<br>
<a href=3D"http://p.sf.net/sfu/Zoho" target=3D"_blank">http://p.sf.net/sfu/=
Zoho</a><br>_______________________________________________<br>
To unsubscribe go to the end of this page: <a href=3D"http://lists.sourcefo=
rge.net/lists/listinfo/prevayler-discussion" target=3D"_blank">http://lists=
.sourceforge.net/lists/listinfo/prevayler-discussion</a><br>
_______________________________________________<br>
&quot;Databases in Memoriam&quot; -- <a href=3D"http://www.prevayler.org" t=
arget=3D"_blank">http://www.prevayler.org</a><br>
<br></blockquote></div><br></div>

--001a1140a8c60d91200505f4ffd2--


--===============2052170017046770278==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
--===============2052170017046770278==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

--===============2052170017046770278==--