Re: installation for large volume

Hi Paul,


On Sat, Apr 18, 2009 at 1:31 AM, Paul Richards <paul-P/[email protected]> wrote:

> Thanks Atif,
>
> I've tested up till about 4000 projects (with some level of subprojects) on
> my test data sets, so this is slightly larger.
>
> If you are just evaluating, it might be worth looking at git trunk. On top
> of that I've got a couple of development branches at the moment that are
> works in progress that might have a knock-on effect of optimising some
> things ( http://git.mantisforge.org/w/mantisbt/paul.git - two branches -
> one that starts to change the way we handle dates, and the other BugData
> objects)
>

I will get your git repository and try it out.


>
> In terms of your data set, I think you might currently be going to hit a
> couple of issues:
> a) we kick off a call to project_cache_all (which does a SELECT * FROM
> mantis_project_table) for any page hit
> --> Reason: that's 'quicker' as the dropdown box to select projects
> generally requires all projects to be loaded, but for this case...
>

Would it not be better to cache only those projects that the logged in user
is allowed to see.


>
>
> b) The project select box - I assume currently only shows 2-3 projects per
> user?
>

Yup. thats correct.


>
>
> c) Filters (view issues page) - I believe the AJAX dropdown selectors to
> search by reporters includes *ALL* users (i.e. 17,000). Someone has
> submitted a patch into www.mantisbt.org/bugs suggesting that we limit the
> reporters to reporters linked to the projects a user is related to, or
> similar.
>

Good idea. I havent tried this one yet.


>
>
> At the moment, the things i'm aware of that i'd like to try to optimise,
> but haven't done are:
> a) large number of viewable projects
> b) subprojects hierachies
> c) generated emails with large numbers of linked people.
>
> I'd be interested to know if the latest development code also shows out of
> memory errors assuming your just testing this.
>
> Out of interest, given you are running php 5, what is your php memory limit
> set to? I know that php group changed the default limit from 8MB to 128MB as
> php5 itself requires more ram than php4.
>

The default memory_limit was set to 16MB which I have raised to 140MB just
to get it working but for production I will set it back to 16MB.

Like I said before, for most use-cases the software works fine but in my
test-cases its pushing the limits.

I am not sure if it is necessary to do these optimizations for so large
datasets but if you think it may be useful, I can give a helping hand. Its
the least we can do to give back to the software we have been using for
years.
We run a lot of php based softwares and i see and understand many problems
that are being faced.
some new software that we are developing recently (in these cases the memory
limit it set to very low (8-12MB) and max_time for script is set to 3-5
seconds :-). After that the script deserves to die)
some old software that were done years ago and we have to maintain ( and in
those cases the memory limit is set very high and we keep dumping big
machines to bail-us-out until we move those softwares to new ones)





>
> Paul
>
>
>
>
>
> ----- Original Message -----
> From: "Atif Ghaffar" <[email protected]>
> To: "Help with Mantis" <[email protected]>
> Sent: Friday, April 17, 2009 11:23:45 PM GMT +00:00 GMT Britain, Ireland,
> Portugal
> Subject: Re: [mantisbt-help] installation for large volume
>
> Here are the new stats after loading the data again.
> Mantis Database Statisticsmantis_bug_file_table = 1 records
> mantis_bug_history_table = 8 records
> mantis_bug_monitor_table = 0 records
> mantis_bug_relationship_table = 0 records
> mantis_bug_table = 3 records
> mantis_bug_tag_table = 0 records
> mantis_bug_text_table = 3 records
> mantis_bugnote_table = 0 records
> mantis_bugnote_text_table = 0 records
> mantis_category_table = 3 records
> mantis_news_table = 0 records
> mantis_plugin_table = 1 records
> mantis_project_file_table = 0 records
> mantis_project_table = 26682 records
> mantis_project_user_list_table = 50743 records
> mantis_project_version_table = 0 records
> mantis_tag_table = 0 records
> mantis_user_table = 17051 records
> mantis_user_profile_table = 0 records
> mantis_user_pref_table = 4 records
> mantis_user_print_pref_table = 0 records
> mantis_custom_field_project_table = 0 records
> mantis_custom_field_table = 1 records
> mantis_custom_field_string_table = 0 records
> mantis_filters_table = 0 records
> mantis_sponsorship_table = 0 records
> mantis_tokens_table = 3 records
> mantis_project_hierarchy_table = 0 records
> mantis_config_table = 1 records
> mantis_email_table = 3 records
>
>
> On Sat, Apr 18, 2009 at 12:03 AM, Atif Ghaffar <[email protected]>wrote:
>
>>
>>
>> On Sat, Apr 18, 2009 at 12:01 AM, Atif Ghaffar <[email protected]>wrote:
>>
>>> Hi Paul,
>>>
>>> a) Please find the file attached for the stats output.
>>>
>> PS: This report is after deleteing all projects/users and then loading
>> just a few thousand records.
>> If you want that I fully load the database and provide you the report,
>> please ask.
>>
>>
>>
>
>
> --
> best regards
> Atif Ghaffar
>
> ------------------------------------------------------------------------------
> Stay on top of everything new and different, both inside and around Java
> (TM) technology - register by April 22, and save $200 on the JavaOne (SM)
> conference, June 2-5, 2009, San Francisco. 300 plus technical and hands-on
> sessions. Register today. Use priority code J9JMT32. http://p.sf.net/sfu/p
> _______________________________________________ mantisbt-help mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/mantisbt-help
>
>
> ------------------------------------------------------------------------------
> Stay on top of everything new and different, both inside and
> around Java (TM) technology - register by April 22, and save
> $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
> 300 plus technical and hands-on sessions. Register today.
> Use priority code J9JMT32. http://p.sf.net/sfu/p
> _______________________________________________
> mantisbt-help mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/mantisbt-help
>
>


-- 
best regards
Atif Ghaffar

------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and 
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today. 
Use priority code J9JMT32. http://p.sf.net/sfu/p

_______________________________________________
mantisbt-help mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mantisbt-help