Re: Implementation PrevaylerEight
Christian Stein <[email protected]> Tue, 21 Oct 2014 22:20:46 +0200
| Newsgroups | gmane.comp.java.prevayler |
|---|---|
| Message-ID | <CAJi8+g7wZDP9yAH4jqebgE6Syo0Yyzq-k3AC+eJ8CxquX4J2vg@mail.gmail.com> |
--===============5907026719481055368==
Content-Type: multipart/alternative; boundary=001a1132f75efd5c9b0505f4907a
--001a1132f75efd5c9b0505f4907a
Content-Type: text/plain; charset=UTF-8
Sure, Marcel. Here you go.
Gonna upload a full source package later this week.
Cheers,
Christian
public class ByteBufferOutputStream extends OutputStream {
private final ByteBuffer buffer;
public ByteBufferOutputStream(ByteBuffer buffer) {
this.buffer = buffer;
}
public void write(int b) {
buffer.put((byte) b);
}
public void write(byte[] bytes, int off, int len) {
buffer.put(bytes, off, len);
}
}
public class ByteBufferInputStream extends InputStream {
private final ByteBuffer buffer;
public ByteBufferInputStream(ByteBuffer buffer) {
this.buffer = buffer;
}
public int read() {
if (!buffer.hasRemaining())
return -1;
return buffer.get();
}
public int read(byte[] bytes, int off, int len) {
len = Math.min(len, buffer.remaining());
buffer.get(bytes, off, len);
return len;
}
}
On Tue, Oct 21, 2014 at 10:14 PM, Klaus Wuestefeld <[email protected]> wrote:
> Nice graph!
>
> On Tue, Oct 21, 2014 at 5:25 PM, Marcel Oerlemans <
> [email protected]> wrote:
>
>> Hi Christian,
>>
>> Those numbers are impressive!
>> I see you tested default Prevayler up to only 8 threads. You might want
>> to scale that up a bit?
>>
>> After I read your mail, I ran a prevayler scalability test, from 1 to
>> 1500 threads, 20 seconds per round/
>> In my test setup Prevayler handled 60 transactions/second @ 8 threads,
>> but at 1489 threads, Prevayler could handle 5945 transactions / second.
>> That's ~100 times as many. Results in graph form can be found here:
>> http://imgur.com/sRIaCVD
>>
>> I wanted to do a similar test with Prevayler8, but I see now I need two
>> classes which I cant seem to find: ByteBufferOutputStream
>> and ByteBufferInputStream. Can you help me with that?
>>
>> Best regards,
>> Marcel Oerlemans
>>
>>
>>
>>
>>
>> On Thu, Oct 16, 2014 at 12:13 PM, Christian Stein <[email protected]>
>> wrote:
>>
>>> Hi folks,
>>>
>>> I recently found some spare time and made the scalability test use
>>> the PrevaylerEight implementation. See [0] for the outline of its source. A
>>> single-system-wide lock facade prevayler implementation is used for
>>> synchronization, at the moment.
>>>
>>> The initial results are promising in terms of performance and
>>> consistency:
>>>
>>> - Default Prevayler: 1511,00 operations/second (8 threads)
>>> - PrevaylerEight implementation: 89189,50 operations/second (2
>>> threads)
>>>
>>>
>>> These microbenchmarks are, especially when the consistency checks kicks
>>> in, very garbage collection dependent. I had to set
>>> the TransactionTestCheckConsistency to NO, because replaying more the
>>> 1.000.000 transactions into the prevalent system would still be running...
>>> Ran the benchmark with -verbose:gc and saw after an hour that every 10
>>> deserialized transactions a full GC.
>>>
>>> So, I had to lower the round duration from 20 to 2 seconds. This change
>>> makes the results even more dependent on the gc, jit, et al. But, now some
>>> results are shown. See console output at [0] and [1]. Query runs and JDBC
>>> runs are switched off.
>>>
>>> Next time, I'll integrate Caliper[3] to show more details.
>>>
>>> Cheers,
>>> Christian
>>>
>>> [0] PrevaylerEight.java
>>> https://gist.github.com/sormuras/3fea6f6ade327b2a4249
>>> [1] Result PrevaylerImpl
>>> https://gist.github.com/sormuras/ef279705d5838a757964
>>> [2] Result PrevaylerEight
>>> https://gist.github.com/sormuras/6f61c6af2066d5f8bf87
>>> [3] Caliper https://code.google.com/p/caliper/
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Comprehensive Server Monitoring with Site24x7.
>>> Monitor 10 servers for $9/Month.
>>> Get alerted through email, SMS, voice calls or mobile push notifications.
>>> Take corrective actions from your mobile device.
>>> http://p.sf.net/sfu/Zoho
>>> _______________________________________________
>>> To unsubscribe go to the end of this page:
>>> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>>> _______________________________________________
>>> "Databases in Memoriam" -- http://www.prevayler.org
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Comprehensive Server Monitoring with Site24x7.
>> Monitor 10 servers for $9/Month.
>> Get alerted through email, SMS, voice calls or mobile push notifications.
>> Take corrective actions from your mobile device.
>> http://p.sf.net/sfu/Zoho
>> _______________________________________________
>> To unsubscribe go to the end of this page:
>> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>> _______________________________________________
>> "Databases in Memoriam" -- http://www.prevayler.org
>>
>>
>
>
> --
> Valeu, Klaus.
>
>
> ------------------------------------------------------------------------------
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push notifications.
> Take corrective actions from your mobile device.
> http://p.sf.net/sfu/Zoho
> _______________________________________________
> To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>
>
--001a1132f75efd5c9b0505f4907a
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div>Sure, Marcel. Here you go.</div><div><br></div><div>G=
onna upload a full source package later this week.</div><div><br></div><div=
>Cheers,</div><div>Christian</div><div><br></div><div><br></div><div>public=
class ByteBufferOutputStream extends OutputStream {</div><div><br></div><d=
iv>=C2=A0 private final ByteBuffer buffer;</div><div><br></div><div>=C2=A0 =
public ByteBufferOutputStream(ByteBuffer buffer) {</div><div>=C2=A0 =C2=A0 =
this.buffer =3D buffer;</div><div>=C2=A0 }</div><div><br></div><div>=C2=A0 =
public void write(int b) {</div><div>=C2=A0 =C2=A0 buffer.put((byte) b);</d=
iv><div>=C2=A0 }</div><div><br></div><div>=C2=A0 public void write(byte[] b=
ytes, int off, int len) {</div><div>=C2=A0 =C2=A0 buffer.put(bytes, off, le=
n);</div><div>=C2=A0 }</div><div><br></div><div>}</div><div><br></div><div>=
<div><br></div><div>public class ByteBufferInputStream extends InputStream =
{</div><div><br></div><div>=C2=A0 private final ByteBuffer buffer;</div><di=
v><br></div><div>=C2=A0 public ByteBufferInputStream(ByteBuffer buffer) {</=
div><div>=C2=A0 =C2=A0 this.buffer =3D buffer;</div><div>=C2=A0 }</div><div=
><br></div><div>=C2=A0 public int read() {</div><div>=C2=A0 =C2=A0 if (!buf=
fer.hasRemaining())</div><div>=C2=A0 =C2=A0 =C2=A0 return -1;</div><div><br=
></div><div>=C2=A0 =C2=A0 return buffer.get();</div><div>=C2=A0 }</div><div=
><br></div><div>=C2=A0 public int read(byte[] bytes, int off, int len) {</d=
iv><div>=C2=A0 =C2=A0 len =3D=C2=A0Math.min(len, buffer.remaining());</div>=
<div>=C2=A0 =C2=A0 buffer.get(bytes, off, len);</div><div>=C2=A0 =C2=A0 ret=
urn len;</div><div>=C2=A0 }</div><div><br></div><div>}</div></div><div><br>=
</div><div><br></div></div><div class=3D"gmail_extra"><br><div class=3D"gma=
il_quote">On Tue, Oct 21, 2014 at 10:14 PM, Klaus Wuestefeld <span dir=3D"l=
tr"><<a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]<=
/a>></span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:=
0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Ni=
ce graph!<br></div><div class=3D"gmail_extra"><div><div class=3D"h5"><br><d=
iv class=3D"gmail_quote">On Tue, Oct 21, 2014 at 5:25 PM, Marcel Oerlemans =
<span dir=3D"ltr"><<a href=3D"mailto:[email protected]" target=
=3D"_blank">[email protected]</a>></span> wrote:<br><blockquote =
class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid=
;padding-left:1ex"><div dir=3D"ltr">Hi Christian,<div><br></div><div>Those =
numbers are impressive!</div><div>I see you tested default Prevayler up to =
only 8 threads. You might want to scale that up a bit?=C2=A0</div><div><br>=
</div><div>After I read your mail, I ran a prevayler scalability test, from=
1 to 1500 threads, 20 seconds per round/</div><div>In my test setup Prevay=
ler handled 60 transactions/second @ 8 threads, but at 1489 threads, Prevay=
ler could handle 5945 transactions / second. That's ~100 times as many.=
Results in graph form can be found here:=C2=A0<a href=3D"http://imgur.com/=
sRIaCVD" target=3D"_blank">http://imgur.com/sRIaCVD</a></div><div><br></div=
><div>I wanted to do a similar test with Prevayler8, but I see now I need t=
wo classes which I cant seem to find:=C2=A0ByteBufferOutputStream and=C2=A0=
ByteBufferInputStream. Can you help me with that?<br></div><div><br></div><=
div>Best regards,</div><div>Marcel Oerlemans</div><div><br></div><div><br><=
/div><div><br></div><div><br></div></div><div class=3D"gmail_extra"><br><di=
v class=3D"gmail_quote">On Thu, Oct 16, 2014 at 12:13 PM, Christian Stein <=
span dir=3D"ltr"><<a href=3D"mailto:[email protected]" target=3D"_blank=
">[email protected]</a>></span> wrote:<br><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex=
"><div dir=3D"ltr">Hi folks,<div><br></div><div>I recently found some spare=
time and made the scalability test use the=C2=A0PrevaylerEight implementat=
ion. See [0] for the outline of its source. A single-system-wide lock facad=
e prevayler implementation is used for synchronization, at the moment.</div=
><div><br></div><div>The initial results are promising in terms of performa=
nce and consistency:</div><div><ul><li>Default Prevayler: 1511,00 operation=
s/second (8 threads)</li><li>PrevaylerEight implementation: 89189,50 operat=
ions/second (2 threads)</li></ul></div><div><br></div><div>These microbench=
marks are, especially when the consistency checks kicks in, very garbage co=
llection dependent. I had to set the=C2=A0TransactionTestCheckConsistency t=
o NO, because replaying more the 1.000.000 transactions into the prevalent =
system would still be running... Ran the benchmark with=C2=A0<span style=3D=
"color:rgb(0,0,0);font-family:Consolas,Menlo,Monaco,'Lucida Console'=
;,'Liberation Mono','DejaVu Sans Mono','Bitstream Vera =
Sans Mono','Courier New',monospace,serif;font-size:14px;line-he=
ight:17.8048000335693px;white-space:pre-wrap;background-color:rgb(238,238,2=
38)">-verbose:gc</span>=C2=A0and saw after an hour that every 10 deserializ=
ed transactions a full GC.=C2=A0</div><div><br></div><div>So, I had to lowe=
r the round duration from 20 to 2 seconds. This change makes the results ev=
en more dependent on the gc, jit, et al. But, now some results are shown. S=
ee console output at [0] and [1]. Query runs and JDBC runs are switched off=
.</div><div><br></div><div>Next time, I'll integrate Caliper[3] to show=
more details.=C2=A0</div><div><br></div><div>Cheers,</div><div>Christian</=
div><div><br></div><div>[0] PrevaylerEight.java <a href=3D"https://gist.git=
hub.com/sormuras/3fea6f6ade327b2a4249" target=3D"_blank">https://gist.githu=
b.com/sormuras/3fea6f6ade327b2a4249</a></div><div>[1] Result PrevaylerImpl=
=C2=A0<a href=3D"https://gist.github.com/sormuras/ef279705d5838a757964" tar=
get=3D"_blank">https://gist.github.com/sormuras/ef279705d5838a757964</a></d=
iv><div>[2] Result PrevaylerEight=C2=A0<a href=3D"https://gist.github.com/s=
ormuras/6f61c6af2066d5f8bf87" target=3D"_blank">https://gist.github.com/sor=
muras/6f61c6af2066d5f8bf87</a></div><div>[3] Caliper <a href=3D"https://cod=
e.google.com/p/caliper/" target=3D"_blank">https://code.google.com/p/calipe=
r/</a></div></div>
<br>-----------------------------------------------------------------------=
-------<br>
Comprehensive Server Monitoring with Site24x7.<br>
Monitor 10 servers for $9/Month.<br>
Get alerted through email, SMS, voice calls or mobile push notifications.<b=
r>
Take corrective actions from your mobile device.<br>
<a href=3D"http://p.sf.net/sfu/Zoho" target=3D"_blank">http://p.sf.net/sfu/=
Zoho</a><br>_______________________________________________<br>
To unsubscribe go to the end of this page: <a href=3D"http://lists.sourcefo=
rge.net/lists/listinfo/prevayler-discussion" target=3D"_blank">http://lists=
.sourceforge.net/lists/listinfo/prevayler-discussion</a><br>
_______________________________________________<br>
"Databases in Memoriam" -- <a href=3D"http://www.prevayler.org" t=
arget=3D"_blank">http://www.prevayler.org</a><br>
<br></blockquote></div><br></div>
<br>-----------------------------------------------------------------------=
-------<br>
Comprehensive Server Monitoring with Site24x7.<br>
Monitor 10 servers for $9/Month.<br>
Get alerted through email, SMS, voice calls or mobile push notifications.<b=
r>
Take corrective actions from your mobile device.<br>
<a href=3D"http://p.sf.net/sfu/Zoho" target=3D"_blank">http://p.sf.net/sfu/=
Zoho</a><br>_______________________________________________<br>
To unsubscribe go to the end of this page: <a href=3D"http://lists.sourcefo=
rge.net/lists/listinfo/prevayler-discussion" target=3D"_blank">http://lists=
.sourceforge.net/lists/listinfo/prevayler-discussion</a><br>
_______________________________________________<br>
"Databases in Memoriam" -- <a href=3D"http://www.prevayler.org" t=
arget=3D"_blank">http://www.prevayler.org</a><br>
<br></blockquote></div><br><br clear=3D"all"><br></div></div><span class=3D=
"HOEnZb"><font color=3D"#888888">-- <br>Valeu, Klaus.
</font></span></div>
<br>-----------------------------------------------------------------------=
-------<br>
Comprehensive Server Monitoring with Site24x7.<br>
Monitor 10 servers for $9/Month.<br>
Get alerted through email, SMS, voice calls or mobile push notifications.<b=
r>
Take corrective actions from your mobile device.<br>
<a href=3D"http://p.sf.net/sfu/Zoho" target=3D"_blank">http://p.sf.net/sfu/=
Zoho</a><br>_______________________________________________<br>
To unsubscribe go to the end of this page: <a href=3D"http://lists.sourcefo=
rge.net/lists/listinfo/prevayler-discussion" target=3D"_blank">http://lists=
.sourceforge.net/lists/listinfo/prevayler-discussion</a><br>
_______________________________________________<br>
"Databases in Memoriam" -- <a href=3D"http://www.prevayler.org" t=
arget=3D"_blank">http://www.prevayler.org</a><br>
<br></blockquote></div><br></div>
--001a1132f75efd5c9b0505f4907a--
--===============5907026719481055368==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
--===============5907026719481055368==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org
--===============5907026719481055368==--