Re: [Security-Discuss] Exploits

* Bob Puff <bob-6dd4Sf22++lWk0Htik3J/[email protected]> [2006-04-03 21:04:36 -0400]:

> I realize that 9.2 is not supported, and is not the latest and greatest.  I
> asked the questions because in the bit of googling around, the symptoms I saw
> pointed to older exploits that were fixed according to the security update
> advisories, but at least at this point -appear- to be the same exploits used
> to compromise the box.
> 
> That said, let's dig in!

Works for me... =)

> Vincent Wrote:
> 
> > What were the factors in determining these to be the causes?  "A ptrace
> > exploit" is fairly generic... considering how long 9.2 has been out 
> > of support, and the large number of vulnerabilities since then, it 
> > could be any one of a number of things.
> > 
> > I'm curious as to how you determined they got access to the box via
> > openssl, however.  What do you have to substantiate that?  The 
> > reason I ask is I've had a number of people assume one thing when it 
> > was, in fact, something much much simpler.  Sounds to me like 
> > openssl isn't the easiest thing to exploit unless you've already got 
> > local access.
> 
> Because I see the following entries in the syslog and kernel logs:
> /var/log/syslog:Apr  2 21:22:27 web1 kernel: request_module[net-pf-14]: waitpid(
> 5083,...) failed, errno 512
> /var/log/syslog:Apr  2 21:22:27 web1 modprobe: modprobe: Can't locate module net
> -pf-14

These are indictive of the BH01 worm, or the ptrace exploit.  These
aren't indictive of any openssl-related issues that I'm aware of.

> The initial googling revealed several messages that pointed to the Apache
> mod_ssl exploit that caused this.

No.  I did a quick google for "net-pf-14 kernel linux exploit" and my
first hit was this:

http://www.sans.org/resources/malwarefaq/Ptrace.php

Reading through it, what you're seeing in the logs is the trail that
myptrace.c leaves.  So this indicates the ptrace (or, *a* ptrace)
vulnerability.  This says nothing about openssl.  Near the top of this
document I saw:

"The attacker used two exploits to gain control over the system. These
are
openssl-too-open - Apache SSL key Arg buffer overflow [ref: 10]
myptrace.c - ptrace and kernel level vulnerability in Linux OS by Snooq
http://www.angelfire.com/linux/snooq/ [ref: 1]"

So what one can infer from this is that the openssl issue was used to
exploit apache in order to gain local privs which are a pre-requisite to
exploit the ptrace issue.  They are *not* tied together.  If I could use
the php xmlrpc vuln to get local privs, I could still use the exploit to
root the box.  I see you referred to the exploit the same way the paper
did... the "openssl-too-open" issue.

If I do another search using the same google terms but tacking "openssl"
to the end, I get a *lot* less hits (7, in fact).  The SANS page shows
up at the top of the list, but the next hit (and the only other one in
english) shows up because someone posted the versions of what they had
installed, and the appropriate bits of the syslog.  But nowhere is the
conclusion drawn that openssl is to blame.

So, again, I know the basic cause of the problem (a ptrace issue) but
the initial entrance vector is still unknown.  You're assuming it's
openssl because (now I'm assuming) you read the SANS paper and saw that
was the entrance vector there but the two are not tied together by any
means.

I think you missed the real entrance vector.  Honestly.

> Interestingly, I made the change to the ssl config (yes, I am running ssl),
> and also did one fix for the ptrace thing (doing a echo /no/such/file >
> /proc/sys/kernel/modprobe at the end of the init scripts), and see recent
> attempts:
> 
> /var/log/syslog:Apr  3 20:18:48 web1 kernel: kmod: failed to exec /no/such/file
> -s -k net-pf-10, errno = 2
> /var/log/syslog:Apr  3 20:27:49 web1 kernel: kmod: failed to exec /no/such/file
> -s -k net-pf-10, errno = 2

So you're still running something vulnerable which means that either
your attempts to fix ssl failed (which could imply the "fix" was
ineffective and it's still vulnerable or, more likely, there is another
entrance vector that you missed.  *Or* there is a cronjob or something
running that is periodically attempting to root your box and setup the
listener because you haven't cleaned up properly.

Either way, I'm astounded you still have this box running.  You're
pretty much due a re-install here.  You can't trust this box anymore.

At least not without some serious cleaning and forensics, which means
finding the real entrance vector.

> > > Could someone please confirm if these 9.2 modules are indeed vulnerable (as 
> > > the advisorys seem to indicate that everything was fixed, but...):
> > > 
> > > openssl-0.9.7b-5.1.92mdk
> > > libopenssl0.9.7-devel-0.9.7b-5.1.92mdk
> > > libopenssl0.9.7-0.9.7b-5.1.92mdk
> > > kernel-2.4.22.32mdk-1-1mdk
> > 
> > 2.4.22?  Everything fixed?  You're kidding, right?  The last kernel
> > for 9.2 was released over a year ago.  Not to be snide here, but if you
> > insist on running a 9.2 box, the *least* you should be doing is manually
> > compiling the latest kernel.org kernel and using it.
> 
> Of course I know no new things would be fixed!  I was referring to the
> documented fixes.  From reading the old mandrake security update notices, I
> see where both non-root ptrace exploits and mod_ssl exploits were fixed.  I
> wanted to verify that these didn't resurface.

The openssl issue hasn't resurfaced AFAIK.  But if you do a search for
"linux ptrace" on the CVE site, you'll see that there have been
ptrace-related issues even in the 2.6 kernel in the last year.  I count
7 CVE numbers for 2005 and 2006 that mention ptrace.  None of them
mention gaining privilege tho, and seem to mostly be for 64bit archs.

As well, doing a bit more digging, the CVE reference in the SANS paper
was for CAN-2003-0127 which was fixed in the kernels even before 9.1 was
released.  9.2 would have had this fix from upstream already.  In fact,
the SANS paper indicates it was fixed after 2.4.21pre1 which, clearly,
pre-dates 2.4.22 which is what you're running.

Which makes me think that while this vulnerability may use the same
mechanism and leave the same "signature", it's not the same thing.  Oh,
it may take advantage of a ptrace vuln, but it's not *that* *same*
ptrace vuln.  The best way to know is for you to find the code in use
and evaluate it (via gdb, strings, etc.).

I think you're too hooked on similarities to look beyond the fact that
these things get re-factored to deal with other issues and you might be
looking in the entirely wrong place.

Anyways, to answer your questions: I don't think mod_ssl is your issue.
And I don't think the kernel ptrace issue you're thinking of has
resurfaced... I think the worm/exploit code has and has been engineered
to take advantage of something more recent.

> > > in to see that I couldn't restart apache.  I saw this running on port 80:
> > > # lsof -i:80
> > > COMMAND   PID   USER   FD   TYPE    DEVICE SIZE NODE NAME
> > > ptrace24 5251 apache    4u  IPv4 137712543       TCP *:http (LISTEN)
> > > hatori   5299 apache    4u  IPv4 137712543       TCP *:http (LISTEN)
> > > CROND    5599 apache    4u  IPv4 137712543       TCP *:http (LISTEN)
> > 
> > Ok, this tells me a few things.  First, the entry vector was apache.
> > Are you using apache and mod_ssl?  If you're not using mod_ssl, you can
> > scrap the openssl issue as being the entry point right now.  Because
> > it's run as user apache, the entry point was apache itself... now, this
> > could have been a vulnerability in apache itself (there have been a 
> > few) or in some web apps you're using (ie. arbitrary php code 
> > execution in packages like phpbb, phppuke... err.. phpnuke, etc.). 
> >  Regardless, apache (or something it's serving up) is how they got in.
> 
> Exactly.  The bad part is, I don't see anything in the standard apache logs
> that would point to accesses at the same time that I see these net-pf errors.

Look before then.  It could easily have been someone who got in and
setup a cronjob for the user apache to do it later in an attempt to
obfuscate the entrance vector.

> > Now, you probably did more harm than good in restarting apache.  I don't
> > believe user apache can restart it's own service (ie. apache can't issue
> > "service apache restart"), so by stopping apache on your own, you let
> > something that was actively waiting to grab port 80 to do so (unless,
> >  of course, they got root and killed it that way... did you notice anything
> > funny connecting to your website to determine whether it was apache 
> > or something else actually listening on port 80?)
> 
> ...and that's exactly what happened.  Parts of Apache wouldn't die; then when
> I did kill it, these files kicked in on port 80.

I've seen that before.

> > > Can't seem to find ptrace24, CROND, or hatori, but they were running.
> > > Example:
> > > # ps ax | grep CRON
> > > 19263 ?        S      0:00 ./CROND
> > 
> > cat /proc/19263/cmdline and do an "ls -al /proc/19263" looking at the
> > exe and cwd entries.  That should tell you where to find this file.
> 
> Great info, I'll hold on to that.
> 
> > > Any ideas on fixing the ability to run the ptrace exploit?
> > 
> > Upgrade?  If you don't want to run the OS, ok, fine... you're playing
> > with fire but that's your call.  At the very least download the 
> > latest linux kernel, compile it, and reboot.
> 
> Ok, here's a question: how much depends on a certain version of the kernel? 
> In other words, am I simply able to take say a 10.2 kernel and install it? 
> Does it have to be compiled from scratch?  Wouldn't other programs all need to
> be compiled again?

It's using a 2.4 kernel already, so you should be able to upgrade to the
latest 2.4 kernel without a problem.  Now, if you've got some really
fancy hardware that had stuff backported to it, then you might have an
issue but considering when 9.2 came out there was no 2.6 kernel, I think
it would be pretty safe to roll your own 2.4 (latest) kernel and at
least try it.

I have no idea whether or not the 2.4-based 10.2 kernel would work.  You
don't want to go to a 2.6 kernel as then you'll have to upgrade some
userland tools.

But... can I give you a piece of advice?  Anything you do will be a
bandaid and you most likely *will* have this happen again.  Running
something public on 9.2 is not something I would recommend to anyone.
It's just not safe.

> Someone here suggested upgrading to a newer version.  I purchased CS 3.0 a
> little while back with the intent of upgrading a couple of my boxes to that,
> but after deploying one just recently, I found a show-stopping Apache bug that
> I just can't run on a production server like this box.

Well, I don't know what the bug is, but have you filed a bugzilla
report?  And, failing that, is there anything stopping you from just
upgrading apache?  Or, if you know what the fix is, to fix it yourself?

Either way, you have some work ahead of you.  You can either attempt to
fix (or help fix or facilitate the fix) of this apache issue, or you can
roll your own kernel and pray that nothing breaks (which it will because
nothing else on that system is being upgraded and issues are being found
almost daily that affect old stuff like that).

Frankly, the easier fix (to me) is to try to get that apache issue dealt
with.  Fiddling with the kernel is a whole lot harder.

-- 
"lynx -source http://linsec.ca/vdanen.asc | gpg --import"
{FEE30AD4 : 7F6C A60C 06C2 4811 FA1C  A2BC 2EBC 5E32 FEE3 0AD4}
Wasting time like it was free...

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Darwin)

iD8DBQFEMgdMLrxeMv7jCtQRAj4AAJ92Uk+4W0EHN5xwx0O9QPyA1Lw+4gCghY+B
OMPOqaGdCywNriMwdZ7NspQ=
=T4aD
-----END PGP SIGNATURE-----