Re: Bug 712, 914..... and now 1027

Hello,

[email protected] wrote:
<cut>

> Alexander Keremidarski indicated that he can not repeat the issue bought to
> light by Paul's test case (bug #1027), yet he has not once attempted to run
> the test case on RedHat 7.3 (nor even on MySQL 4.0.14) which the bug report
> calls for as he considers testing performance issues between RedHat releases
> to be 'irrelevant'.  

Yes irrelevant.

I said TEST case is irrelevant, not Speed Difference Issue.

I also stated that if there is test which demonstrates difference in speed between
MySQL vresions running on same platform (OS + hardware) this is to be considered 
MySQL problem.

Quoting my previous email:

> Send us please test case which demonstrates difference in speed between
> MySQL 3.23 under RedHat RedHat 7.3 and MySQL 4.0 under RedHat 7.3
> and also test case which demonstrates difference in speed between
> MySQL 3.23 RedHat 9 and MySQL 4.0 on RedHat 9

I insyst on fact that Benchmarking between different Versions of MySQL (actually 
any software) runnign on different platforms (OS + Hardware) is irrelevant.

Benchmarking MySQL 3.23 on RedHat 7.3 against MySQL 4.0 on RedHat 9 is irrelevant.
Benchmarking MySQL 3.23 on RedHat 7.3 against MySQL 4.0 on RedHat 7.3 is relevant 
as well as benchmarking MySQL 3.23 on RedHat 9 against MySQL 4.0 on RedHat 9

Add to above equation "with diffrent mysqld startup options". 3.23.49 and 4.0.14 
have different default startup options.

> Using identical hardware (previously a master/slave backup system) I ran
> BOTH test cases provided by Paul Coldrey on MySQL 3.23.49, RedHat 7.3 and
> MySQL 4.0.14, RedHat 9.0 and can verify that I received similar poor
> performance easily and repetedly.
 >
> Under MySQL 4.0.14, RedHat 9.0 both insert.sh and insert.sql took around 3
> minutes and 30 seconds to insert 10,000 rows into an un-ndexed MyISAM table.
> 
> Under MySQL 3.23.49, RedHat 7.3 insert.sh took around 1 minute 30 seconds
> while insert.sql took a mere 4 seconds.


> Given the test cases insert a pitiful 10,000 rows into an un-indexed MyISAM
> table which of the following does the MySQL team consider acceptable?

I disagree. Paul Coldrey test case creates table as:

CREATE DATABASE BugMe3;
USE BugMe3;

CREATE TABLE GroupMemberLookup (
   GroupId int(11) default NULL,
   PersonId int(11) default NULL,
   SequenceId int(11) default NULL,
   KEY GroupId (GroupId),
   KEY PersonId (PersonId),
   KEY groupmemberlookup_sequenceid_i (SequenceId)
) TYPE=InnoDB;

Therefore it tests InnoDB INSERT speed not MyISAM test speed.

MyISAM INSERT speed and InnoDB INSERT speed are very different due to lot of 
reasons. There are cases where one is faster than another and vice versa.


By the way it is important to run above test against MySQL-Max-3.23 as in 3.23 
InnoDB is compiled only in Max releases while in 4.0 it is compiled in standard 
releases too. There is one more huge difference. Max versions are dynamically 
linked while standard are statically linked.

>  1) The 3 minutes 30 seconds I experienced under MySQL 4.0.14, RedHat 9.0 ?
> 
>  2) The 1 minute 45 seconds Alexander Keremidarski experienced under MySQL
> 4.0, RedHat 9.0 ?
> 
>  3) The 4 seconds I experienced under MySQL 3.23.49, RedHat 7.3 ?

We don't run same test under same conditions. 1) and 3) are comparable, 2) is not 
comparable to any of 1) and 3).


> As Alexander rightly points out Pauls test case demonstrates that MySQL
> performs differently on different versions of RedHat which is
> understandable.  But what baffles me is why when such a marked DECREASE in
> performance is bought to the attention of the MySQL developers is it
> dismissed as an OS issue and investigated no further?  Even if it is a
> RedHat 9.0 issue it manifests itself most obviously in MySQL which will
> leave the majority of users blaming MySQL for the bad performance.
> 
> Be it a MySQL issue, be it a Red Hat issue, whatever it is, isn't it
> important to the MySQL developers to maintain performance on releases
> regardless of operating system version?  Its not like we're comparing RedHat
> 9.0 with Windows 98 here.

Yes it is important. No we don't have RedHat 9 tested well.

> When my clients come to me asking to upgrade their servers what advice shall
> I give? That the MySQL development team acknowledges that "different
> versions of MySQL appear to run differently on different versions of RedHat"
> but refuses to do anything about it so stick with MySQL 3.23.49 on RedHat
> 7.3????

Why 3.23.49? Why not benchmarking it against 4.0.14 on the very same RedHat 7.3?
Why not benchmarking it against current stable 3.23 which is 3.23.57?

Don't you take into consideration the fact that 3.23.49 is year and half old 
release and there is bug amount of bugs fixed since then?

> In understand Paul Coldrey's frustration completely.  This is an issue that
> warrants serious attention and unless there is a solid commitment by the
> MySQL development team to ensure no performance degradation between releases
> regardless of operating system versions all of us who recommend MySQL
> products to our clients are putting our reputation on the line each time we
> do so ... and thats something I am not prepared to do.

I also understand and share Paul Coldrey's frustration. I have spend lot of time 
trying to pinpoint the problem, but so far I don't see anything which can help us 
find what exactly is going wrong.

> Yes, different versions of MySQL do run differently on different versions of
> RedHat.  But when upgrading both MySQL and the OS one expects an increase
> in, or similar performance not a decrease, and to say openly that is not an
> issue is a slap in the face to all the developers who have loyally supported
> your product over the years.

I don't agree with your point. Upgrading OS unfortunately in most cases slows down 
machines as new versions are usually more demanding than old ones. True for Linux 
distributions too. Yes upgrading from kernel 2.2 to kernel 2.4 improves your CPU 
performance, but first thing you read is that 2.4 is more demanding to amount of 
RAM and so on.

My personal experience shows that upgrading from RH 7.3 to RH 9 slows down all and 
every program.
But this is just my impression. I have no tests to prove it.

Therefore I don't state that every program which slows down has bugs and it's 
programmers must take it into consideration. I just assume that RedHat 9 is most 
demanding to resources and this is something to live with.





Here are results of Paul Coldrey test case against 3.23.49, 3.23.57 and 4.0.15 on 
same machine:


inserts.sh

mysql-max-3.23.49a 1 min 39 sec
mysql-max-3.23.57  1 min 40 sec

mysql-debug-4.0.15 1 min 5 sec

inserts.sql

mysql-max-3.23.49a 3.9 sec
mysql-max-3.23.57  3.3 sec

mysql-debug-4.0.15 3.9 sec

RedHat 9
kernel 2.4.20-19.9
CPU  Intel(R) Pentium(R) 4 CPU 1.60GHz
528Mb RAM


As you see I can't repeat "degradation in speed between MySQL versions".


>   Byron Kidd.

Best regards

-- 
  Are you MySQL certified? -> http://www.mysql.com/certification
  For technical support contracts, visit https://order.mysql.com/?ref=msal
     __  ___     ___ ____  __
    /  |/  /_ __/ __/ __ \/ /    Mr. Alexander Keremidarski <[email protected]>
   / /|_/ / // /\ \/ /_/ / /__   MySQL AB, Full-Time Developer
  /_/  /_/\_, /___/\___\_\___/   Sofia, Bulgaria
          <___/  www.mysql.com




-- 
MySQL Bugs Mailing List
For list archives: http://lists.mysql.com/bugs
To unsubscribe:    http://lists.mysql.com/[email protected]