Re: Error messages from syslogd

Mordechai T. Abzug wrote:
>I would love to live in a world where all code put into production is
>bug-free.  In practice, we live in a world where code put into
>production usually contains bugs.

Hang on - you asked me "what's wrong with this" and I told
you. Then you reply "but we don't live in an ideal world" - that's
cheating. If your precondition for wrongness is that things
stay wrong, then - of course - nothing ever gets better.  :)

You're also conflating a couple problems - we're not talking
about buggy code, here, we're talking about error handling.
That's an entirely different problem because many errors
are external to the program that's being invoked (i.e.: the
program is told to read a file that's not there, etc) - the
people who write code need to realize that
"bind"
is not a useful error message. It becomes a mildly
useful log message only by virtue of the fact that
syslogd puts the program name and the error
code around it - something that any competent
programmer would have already done.

Don't expect me to be an apologist for the huge
numbers of crappy "programmers" out there. That
is just not going to happen. :)

>  Which is why vendors release
>patches, right?

Don't expect me to be an apologist for the huge
numbers of crappy "programmers" out there. That
is just not going to happen. :)

The reason vendors release patches is because:
a) their code is crap to begin with and their customers
        paid for something that actually does something
        and they're trying to get their customers money
        by eventually providing it
b) it justifies their 20% "annual maintenance charge"
        which means that you're paying for the crap
        all over again every 5 years in spite of the fact
        that it barely worked in the first place and probably
        never will.

>This is especially true when working with $vendor's buggy COTS
>program, for which I cannot apply a debugger, make it more reliable,
>or prevent it from crashing or otherwise misbehaving without more
>information.  When a COTS product has a problem, looking at logs,
>including syslogs, can be instrumental in fixing said problem.

This is pointless - now you're just making excuses for
mediocrity. OK, so you bought some crap and you have to
fix it. The problem is that if you presuppose that it's crap
then you can also assume that the only "useful" messages
you'll have in syslog are things like:
(datetime) program: uh, wow: socket operation on non socket


> I don't want to send the vendor a crashdump, and wait 6 months for a
>patch, I want to see a debug message saying that the product had
>successfully started and finished X, then crashed while doing Y, so I
>can figure out how Y differs from X and maybe workaround the problem.
>This is not theoretical -- I do this kind of thing all the time, and
>it's part of why I care about logs.

Hang on, if you're assuming that the vendor is too stupid to write
a decent crash dump, you have to assume that they're too stupid
to write a decent syslog message, too. Stop making excuses
for lameness, OK? That makes you part of the problem, not
part of the solution. The only way to get rid of the crap that is out
there is to be absolutely, ruthlessly, utterly intolerant of crap.

My point in the previous posting is that if an application identifies
a system error it ABSOLUTELY should put that in the log because
things like "file system full" or "file table full" are going to affect
other processes. But stuff like program internal consistency
error checks? Writing that to a crash dump is a whole lot
better because then all the information is right there in one
place, not scattered around in syslog messages that say
(datetime) program: help! programmer sucks! WTF?

> For example, the firewall is up, but it has stopped
>allowing telnet connections.  The logs have tn-gw whining that it has
>too many policies -- apparently, the vendor in question had a
>hardcoded limit of 100 tn-gw policies in netperm.

I've learned a thing or two about coding in the last decade, yep.
I'm not saying I'm perfect, either - obviously - and I'm intolerant
of crap even if I wrote it. That's why I started telling people to
stop using fwtk 4 years after it was out....  But this isn't about
scoring debaters' points, is it?

> And I'm sure you
>know which firewall vendor I'm talking about.  ;) Thanks to having a
>useful failure message in the logs, I could then workaround the
>problem (merge some policies temporarily) and request a patch from the
>vendor more easily than by just saying "it's broke, I don't know why,
>fix it."

As I said before: that program:
a) should not have had such an amateurish error - obviously it was
        coded by someone young and foolish. ;)
b) should have offered a succint diagnosis of the problem to the
        vendor so you didn't have to run around trying to figure it
        out

(BTW - the reason, in that case, why syslog was used was because
of the assumption that most firewalls would not have a useable
console. Unfortunately, that turned out to be a correct assumption)

>No, it's pure genius.  raw syslog has been so successful precisely
>because it takes so little effort to get value.  Speaking from a
>sysadmin perspective, I would *way* rather have syslog spewage than a
>more austere logging system that is better-structured but might not
>contain the critical clue I need to actually fix a problem quickly.

What you're saying, in a nutshell is: "the suckage that I understand
is better, because I understand it."

That's sometimes true. But that's the mindset that makes people
use hammers to drive in perfectly good screws.

mjr.