Re: Prewikka performance

we have around 1 million events in a mysql database and response from prewikka is far from sparkling.  I too would be interested in knowing how well prelude scales.   Looking at the DB schema there are a lot of tables and I suspect that you end up doing a *lot* of joins for most queries.

Russell


On 6/08/2010, at 9:23 PM, Honza Mach wrote:

> Hi everybody,
> 
> does anyone has any information about prewikka performance to share with
> me?
> 
> Regards
> 
> ---
> Jan Mach
> Cesnet, z.s.p.o.
> www.cesnet.cz
> 
> From: Honza Mach <[email protected]>
> Date: 30 July 2010 1:40:43 AM NZST
> To: <[email protected]>
> Subject: [prelude-user] Prewikka performance
> Reply-To: <[email protected]>
> 
> 
> Hi everybody,
> 
> I was wondering, if anyone could give me some information regarding
> Prewikka performance.
> 
> I have recently installed Prelude on Debian Lenny system and connected
> some heavy sensors we have. After the weekend, the MySQL database
> contained about 6 million entries and the Prewikka interface was
> absolutely unusable - it could not even render the page after tens of
> minutes. 
> 
> My first guess was that the database has poor performance. I made some
> tuning and optimization of InnoDB tables according to the tips here
> http://www.wzdftpd.net/trac/wiki/Misc/PreludeTips and here
> http://www.mysqlperformanceblog.com/2007/11/03/choosing-innodb_buffer_pool_size/
> 
> , but with absolutely no positive results. Database responded normally
> via the linux command line, incoming new entries were written in real
> time and did not get stalled anywhere.
> 
> So it brought me to the Prewikka. I disconnected some of the heaviest
> senzors, turned on the MySQL logging and discovered, that for Prewikka
> to display the alerts listing page with about 2000 entries in the
> database it has to do about 2000 QUERIES to the database !!! This is
> absolutely out of my expectations and no wonder that it was impossible
> to render the alert listing page with 6 million entries in DB. Prewikka
> reads the entries one by one.
> 
> Have you any experience with using Prewikka with large amount of data?
> Are there any options to optimize the queries, so the Prewikka will be
> usable even for huge amounts of data? Or are there any lightweight
> alternatives to Prewikka? Have you any experience with Piwi? It seems
> dead to me.
> 
> I have considered the pre-filtering of the incoming data in the
> prelude-manager, so the database will not contain so much of junk, but
> still, I would like to keep as many of the data as possible.
> 
> Perhaps this is not the issue of the Prewikka, but instead of the
> underlaying libpreludedb library.
> 
> I have to admit, that we are testing the Prelude on the lower-end system
> (Dell PowerEdge 2650, 4GB RAM), so I did not expected flashing speed,
> but this in my opinion is too much.
> 
> Thanks for your time in advance
> 
> Jan Mach
> Cesnet, z.s.p.o.
> www.cesnet.cz
> _______________________________________________
> Prelude-user site list
> [email protected]
> http://lists.prelude-technologies.com/mailman/listinfo/prelude-user
> 
> _______________________________________________
> Prelude-user site list
> [email protected]
> http://lists.prelude-technologies.com/mailman/listinfo/prelude-user

_______________________________________________
Prelude-user site list
[email protected]
http://lists.prelude-technologies.com/mailman/listinfo/prelude-user