Upgrading to PHP 5.2, MySQL no longer bundled

I'm trying to upgrade to PHP 5.2 on a Linux machine (RedHat). For the 
"./configure" step, I copied the configure command exactly as it appears 
in the results of "php -i". When I run it in the new installation 
folder, I get a message telling me that the program can't find the MySQL 
header files in the location specified, and warns me that MySQL is no 
longer bundled with PHP. (The location specified is "/usr".)

I've looked around on the Internet, and apparently one key file is 
"mysql.h". Indeed, there is no such file under /usr/include/, though 
there are tons of other header files. I used "find" to locate the file, 
and it's nowhere on the server. (I didn't setup the server, the hosting 
company did that.) I guess the header files don't exist if MySQL comes 
bundled with the server, or something like that? I'm not sure. My server 
is running MySQL 5.0.45.

Some have suggested that, rather than re-installing MySQL (ugh), you 
simply run this command: "yum install mysql-devel". When I do that, the 
program analyzes my system and tells me it needs to update two packages: 
"mysql" and "mysql-server". I don't like the sound of that. I don't want 
to change anything to do with MySQL, I just want my new version of PHP 
to work with it. What should I do?