RE: How to define Log, Event, and Alert?
Eric Fitzgerald <[email protected]> Wed, 30 Jul 2008 00:41:08 -0700
| Newsgroups | gmane.comp.sysutils.loganalysis |
|---|---|
| Message-ID | <7E6E73F6-2E06-444B-BB39-3010835F8866@mimectl> |
--===============1008890044== Content-Class: urn:content-classes:message Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_7E6E73F62E06444BBB393010835F8866mimectl_" --_000_7E6E73F62E06444BBB393010835F8866mimectl_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Rainer, I have really enjoyed reading your last few posts. I would invite you to j= oin the CEE working group. Your discussion of state was very similar to a private one that I sent offl= ine in CEE. I think that from a developer standpoint, events are typically= raised on internal state changes, but users of a system typically view it = as a black box and don't focus on the state changes so much as the observab= le macroscopic occurrence, which might appear to have been a state change b= ut might not appear to be so. My working definition of "event" is: "An observable occurrence in an IT system." * observable - If we can't observe it, then we can't raise an event on it, = so it's not interesting. * occurrence - We raise an event about an instantaneous occurrence, not abo= ut something ongoing (the beginning, end, and perhaps state changes within = a long duration process might be events). * IT system - This restricts the problem domain to computer science, seemed= like a reasonable restriction at the time. Perhaps there is a better term= to restrict this to the realm of computers and software. For completeness, I define the following: Event record- a persistable data structure containing information about an = event Event log- an ordered database of event records (typically but not always a= sequential access file) [this also captures your and Bill's [Scherr] point= about the temporal aspect of events, but does not restrict us to any speci= fic technology] Now a few comments regarding sequence: Sequence information in the log is also often insufficient for determining = the actual order of events; I agree that it is better than timestamp for th= e reasons that you state but I also feel that most logs are significantly l= ess useful (if not useless) without timestamps, being the primary correlato= r with any other observable or discoverable information in the IT environme= nt. Consider a multi-processor system where the log server is in a different pr= ocess than the processes which are performing logged activities. If multiple loggable activities occur at roughly the same time, then whiche= ver process' inter-process call to the log server is serviced first, will g= et the lower sequence number. Since logging often involves allocating memo= ry for and marshaling string data, etc., often the "smaller" event makes it= to the log first, or in some cases, the "better prepared" event- if a proc= ess that logs frequently leaves allocated buffers then it might be able to = call the log server without delay, while another process is still marshalin= g log data, even though the logged activities occurred simultaneously. It gets worse. Process prioritization might influence the order in which t= he kernel services the calls to the log server, as will the time that the l= ogged activity and the call to the logging service happened within the quan= tum of execution. So an activity that occurred at an earlier time on a lower-priority process= , or an activity that occurred at an earlier time, but near the end of the = execution quantum for the process raising the event, will frequently be log= ged sequentially after events raised by higher-priority processes. To an external observer, log ordering on a busy multiprocessor server might= appear essentially random. A group here at Microsoft did some research recently into reliable ordering= of events. I am currently unable to locate a published paper on this but = I am still looking. If I find it I will forward the link. In short it is = possible, even across multiple systems, to be able to reliably reconstruct = order in limited cases but there are severe constraints on the scenarios. = I can't go into more detail until I determine the IP status of the work. Sequence information IS useful for gap detection and for a rough idea of or= dering. I'm not sure that high-precision timestamping on logs is very usef= ul except perhaps on single-processor machines with cooperative rather than= pre-emptive multitasking OS, or on real-time OS. I do think that adding a "collection timestamp" at a log collection server = is very useful as this solves almost all the problems of timestamp normaliz= ation, and it doesn't make ordering any worse than it already is. Eric ________________________________ From: [email protected] [loganalysis-bounces@loganalysis.= org] On Behalf Of Rainer Gerhards [[email protected]] Sent: Tuesday, July 29, 2008 8:31 AM To: [email protected] Cc: [email protected] Subject: RE: [logs] How to define Log, Event, and Alert? <inline...> On Fri, 2008-07-25 at 22:48 -0400, Bill Scherr IV wrote: > Wow! Comments interlaced! > > Circa 11:23, 25 Jul 2008, a note, claiming source Rainer Gerhards <rgerha= [email protected]>, was sent to me: > > Subject: RE: [logs] How to define Log, Event, and Alert? > Date sent: Fri, 25 Jul 2008 11:23:40 +0200 > From: "Rainer Gerhards" <[email protected]> > To: "Anton Chuvakin" <[email protected]>, <bschnzl@cot= se.net> > Copies to: <[email protected]> > > > If I may generalize things a bit... > > > > I'd replace TIME by "sequence identifier", with sequence identifier > > defined as being something that is monotonically increasing. A timestam= p > > is an object that we think to be a natural example of a monotonically > > increasing function. HOWEVER, if we look at existing technology, this i= s > > not always the case. In fact, it is more often NOT the case than it > > is... If we have two systems a and b and these systems do not have time > > synchronized, and have a system c which is the event collector (and > > collects only events from a and b), then c may record time stamps insid= e > > its log that do not monotonically increase. For example, it may record: > > > > 02:00:00 event a1 > > 01:00:00 event b1 > > 02:01:00 event a2 > > 01:01:00 event b2 > > > If we keep the original log intact, the out of sync timestamps are inform= ation in themselves. My point was just in relation to that log. You even do not know - with only the information you see - if the log is out of sync. For example, some time zone may be improperly configured. As many devices do not convey timezone info, you need to reset to metadata to do that. So in short: the timestamp alone is insufficient. > > > Of course, this is still a TIMED record of occurrences. However, in thi= s > > sense this is used, "TIMED" includes a sense of temporal order (at leas= t > > to me). In the above log, we may not have the correct temporal order. > > We may be able to reconstruct it by sorting on the timestamp. That woul= d > > be a valid approach if the timestamps are indeed correct (compared to > > universal time). But if a and/or b has incorrect time, we would create = a > > wrong temporal order. Indeed, in this sense the monotonically increasin= g > > identity of the log in question would actually not be the timestamp but > > rather the *sequence of recording*, kind of a meta-property not directl= y > > contained in the property set of the individual event record (but rathe= r > > obtained by its relationship to its predecessor in the log file). > > > If a file is deleted before it is created, then we locate the shift in ti= mestamps. Different context has different meaning. For > instance, a timestamp shift coincident with the file deletion says real b= adness, while a timeshift occuring an odd time before > (3hrs, 24mins, 5 secs) may just indicate that an admin noticed that ntpd = had died. Rather than try to sync the time in the > logs, what can we do with variations? Automatically? I am not talking about a shift in timestamp. I am talking about insufficient resolution. The system I describe is working fine, but it emits only second-resolution timestamps. For a file delete and file create, you can not differentiate any sequence without resorting to meta-information (like sequence of records inside the event log). > > > Now let's assume a log without a timestamp. These things happens, e.g. > > in debug logs (and all too often in others I have seen). > > > > If we define > > > > > Log =3D a TIMED record of the above occurence. > > > > such a "log" would obviously not be a log, because it does not fulfill > > the requirement to be timed. > > > > If a log instead is "a record of events with a sequence identifier", > > that problem does not exist. The sequence identifier in that case would > > be the derived property I mentioned above. > > > The sequence is the order in which it appears in the log, plus or minus n= etwork latency. (wait for it) What happens if the logs were queued for some time at an interim system? Again, you need to have the meta-information, just the timestamp is insufficient. > > > The question remains if such a definition is actually useful. The > > sequence identifier is obviously something with very vague semantics. > > They depend on the observer as well as the correctness of the "sequence > > identifier generating function" on all systems in question. > > > > Let's get back to the simple case of timestamps: as outlined above, the > > semantics of timestamps depend on time sync. Even if there is ntp > > timesync, timestamps (with reasonable precision) are always > > questionable. They are approximate, even on the same system. With > > standard syslog timestamps (second precision!) the problem is easy to > > see: one may receive hundreds of events within the same second. So even > > if time is correct, an observer is unable to detect any order of events= . > > If looking just at the timestamps, one must conclude that all events > > happened at once. If looking at the semantics of the messages, one most > > often also can conclude this is impossible (e.g. how to delete a file > > before it is created?). Obviously, the timestamp alone is never > > sufficient to detect order of events, even on a single system. Granted, > > for practical purposes a high resolution timestamp (with good time > > synchronization) is most often a sufficiently well approximation of the > > time an event happened. But do you really trust it? ...always? Have a > > look at your own correlation engines: do they work on pure timestamps - > > or do they include some other properties, like the order of event log > > records inside the log? > > > Why are you looking at logs? Is the machine broke? There should be plen= ty of context to render a correct determination. Is > the machine compromised? Then you must find an independent log that says= roughly the same thing. Of course. But, and this is my point, you can not just rely on the timestamp (as you say). > > > Now let me try to define what I think a log actually is: > > > > An EVENT is a set of properties that describe a state change (in the > > sense I have described state change yesterday). The contents of this se= t > > is depending on the entity who's state changes as well as on the > > observer. [so it may actually be a set of two sets: entity-related > > properties and observer-related properties] > > > > An event is generated when a state changes. > > > "when" being the temporal aspect, recorded from the view of the program s= ensing the state change. I fully agree. IMO a "state change" is also a temporal aspect. I think "state *change*" implies that something has *now* state y while it had state x a (maybe very) short while ago. So I think there is always a temporal component inside a state change. <snip> > > > There is no inherent order inside event logs. In practice we experience > > a "natural order" (one record begins before another), but that actually > > is a meta-property of the event record. So we can order event records > > based on their meta-properties. It just happens that a text log is > > physically ordered by the sequence meta property. > > <snip> > > > > > > On Thu, Jul 24, 2008 at 7:23 PM, Bill Scherr IV <[email protected]> > > > wrote: > > > > So... > > > > > > > > I gather a temporal mention to be appropriate beyond the definition > > > of the Log. Also, most systems break off their logs by > > > > size, not time. Although there is a definite time to each log, the= y > > > are not consistent, even with the same log gatherer. Right or > > > > wrong, that is how I find them. Suggestions below (if I may be so > > > bold): > > > > > > > > Circa 11:26, 23 Jul 2008, a note, claiming source Heinbockel, Bill > > > <[email protected]>, was sent to me: > > > > > > > > From: "Heinbockel, Bill" <[email protected]> > > > > To: <[email protected]> > > > > Subject: [logs] How to define Log, Event, and Alert? > > > > > > > >> > > > >> > > > >> Here is our initial shot at defining these terms: > > > >> > > > >> > > > >> Event: > > > >> A discrete, distinct, and discernible state change in an > > > >> environment. > > > > > > > > A discrete, distinct, and discernible state change in an environmen= t > > > at a recorded (or given) time. > > > >> --_000_7E6E73F62E06444BBB393010835F8866mimectl_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <HTML dir=3Dltr><HEAD> <STYLE>.EmailQuote { PADDING-LEFT: 4pt; MARGIN-LEFT: 1pt; BORDER-LEFT: #800000 2px solid } </STYLE> <STYLE title=3DowaParaStyle>P { MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px } </STYLE> </HEAD> <BODY ocsi=3D"x"> <DIV dir=3Dltr><FONT face=3DTahoma color=3D#000000 size=3D2>Rainer,</FONT><= /DIV> <DIV dir=3Dltr><FONT face=3DTahoma color=3D#000000 size=3D2></FONT> </= DIV> <DIV dir=3Dltr><FONT face=3DTahoma color=3D#000000 size=3D2><FONT face=3Dta= homa>I have really enjoyed reading your last few posts. I would invit= e you to join the CEE working group.</FONT></FONT></DIV> <DIV dir=3Dltr><FONT face=3DTahoma color=3D#000000 size=3D2><FONT face=3Dta= homa></FONT></FONT> </DIV> <DIV dir=3Dltr><FONT face=3DTahoma color=3D#000000 size=3D2><FONT face=3Dta= homa>Your discussion of state was very similar to a private one that I sent= offline in CEE. I think that from a developer standpoint, events are= typically raised on internal state changes, but users of a system typicall= y view it as a black box and don't focus on the state changes so much as th= e observable macroscopic occurrence, which might appear to have been a stat= e change but might not appear to be so.</FONT></FONT></DIV> <DIV dir=3Dltr><FONT face=3DTahoma color=3D#000000 size=3D2><FONT face=3Dta= homa></FONT></FONT> </DIV> <DIV dir=3Dltr><FONT face=3DTahoma color=3D#000000 size=3D2><FONT face=3Dta= homa>My working definition of "event" is:</FONT></FONT></DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2></FONT> </DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>"An observable occurrence in an= IT system."</FONT></DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>* observable - If we can't= observe it, then we can't raise an event on it, so it's not interesting.</= FONT></DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>* occurrence - We raise an= event about an instantaneous occurrence, not about something ongoing (the = beginning, end, and perhaps state changes within a long duration process mi= ght be events).</FONT></DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>* IT system - This restricts th= e problem domain to computer science, seemed like a reasonable restriction = at the time. Perhaps there is a better term to restrict this to the r= ealm of computers and software.</FONT></DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2></FONT> </DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>For completeness, I define the = following:</FONT></DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>Event record- a persistable dat= a structure containing information about an event</FONT></DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>Event log- an ordered database = of event records (typically but not always a sequential access file) [= this also captures your and Bill's [Scherr] point about the temporal aspect= of events, but does not restrict us to any specific technology]</FONT></DI= V> <DIV dir=3Dltr><FONT face=3DTahoma color=3D#000000 size=3D2></FONT> </= DIV> <DIV dir=3Dltr><FONT face=3DTahoma color=3D#000000 size=3D2>Now a few comme= nts regarding sequence:</FONT></DIV> <DIV dir=3Dltr><FONT face=3DTahoma color=3D#000000 size=3D2></FONT> </= DIV> <DIV dir=3Dltr><FONT face=3DTahoma color=3D#000000 size=3D2>Sequence inform= ation in the log is also often insufficient for determining the actual orde= r of events; I agree that it is better than timestamp for the reasons that = you state but I also feel that most logs are significantly less useful (if = not useless) without timestamps, being the primary correlator with any= other observable or discoverable information in the IT environment.</FONT>= </DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2></FONT> </DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>Consider a multi-processor syst= em where the log server is in a different process than the processes which = are performing logged activities.</FONT></DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2></FONT> </DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>If multiple loggable activities= occur at roughly the same time, then whichever process' inter-process call= to the log server is serviced first, will get the lower sequence number.&n= bsp; Since logging often involves allocating memory for and marshaling stri= ng data, etc., often the "smaller" event makes it to the log first, or in s= ome cases, the "better prepared" event- if a process that logs frequently l= eaves allocated buffers then it might be able to call the log server withou= t delay, while another process is still marshaling log data, even though th= e logged activities occurred simultaneously.</FONT></DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2></FONT> </DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>It gets worse. Process pr= ioritization might influence the order in which the kernel services the cal= ls to the log server, as will the time that the logged activity and the cal= l to the logging service happened within the quantum of execution.</FONT></= DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2></FONT> </DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>So an activity that occurred at= an earlier time on a lower-priority process, or an activity that occurred = at an earlier time, but near the end of the execution quantum for the proce= ss raising the event, will frequently be logged sequentially after events r= aised by higher-priority processes.</FONT></DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2></FONT> </DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>To an external observer, log or= dering on a busy multiprocessor server might appear essentially random.</FO= NT></DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2></FONT> </DIV><FONT face= =3Dtahoma size=3D2> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>A group here at Microsoft did s= ome research recently into reliable ordering of events. I a= m currently unable to locate a published paper on this but I am still = looking. If I find it I will forward the link. In short it is p= ossible, even across multiple systems, to be able to reliably reconstruct o= rder in limited cases but there are severe constraints on the scenarios.&nb= sp; I can't go into more detail until I determine the IP status of the work= .</FONT></DIV> <P><FONT face=3Dtahoma></FONT> </P> <DIV dir=3Dltr>Sequence information IS useful for gap detection and for a r= ough idea of ordering. I'm not sure that high-precision timestamping = on logs is very useful except perhaps on single-processor machines wit= h cooperative rather than pre-emptive multitasking OS, or on real-time= OS.</FONT></DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2></FONT> </DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>I do think that adding a "colle= ction timestamp" at a log collection server is very useful as this solves a= lmost all the problems of timestamp normalization, and it doesn't make orde= ring any worse than it already is.</FONT></DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2></FONT> </DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2>Eric</FONT></DIV> <DIV dir=3Dltr><FONT face=3Dtahoma size=3D2></FONT> </DIV> <DIV id=3DdivRpF936612 style=3D"DIRECTION: ltr"> <HR tabIndex=3D-1> <FONT face=3DTahoma size=3D2><B>From:</B> [email protected]= rg [[email protected]] On Behalf Of Rainer Gerhards [rger= [email protected]]<BR><B>Sent:</B> Tuesday, July 29, 2008 8:31 AM<BR><B>= To:</B> [email protected]<BR><B>Cc:</B> [email protected]<BR><B>S= ubject:</B> RE: [logs] How to define Log, Event, and Alert?<BR></FONT><BR><= /DIV> <DIV></DIV><FONT size=3D2> <DIV class=3DPlainText><inline...><BR><BR>On Fri, 2008-07-25 at 22:48= -0400, Bill Scherr IV wrote:<BR>> Wow! Comments interlaced!= <BR>><BR>> Circa 11:23, 25 Jul 2008, a note, claiming source Rainer G= erhards <[email protected]>, was sent to me:<BR>><BR>> S= ubject: &n= bsp; RE: [logs] How to define Log, Event, and Alert?<BR>> Date sen= t: Fri, 2= 5 Jul 2008 11:23:40 +0200<BR>> From: = "Rainer Gerhar= ds" <[email protected]><BR>> To: &nb= sp; = "Anton Chuvakin" <[email protected]>, <[email protected]&g= t;<BR>> Copies to: = <[email protected]><BR>><BR>> > If I = may generalize things a bit...<BR>> ><BR>> > I'd replace TIME b= y "sequence identifier", with sequence identifier<BR>> > defined as b= eing something that is monotonically increasing. A timestamp<BR>> > i= s an object that we think to be a natural example of a monotonically<BR>>= ; > increasing function. HOWEVER, if we look at existing technology, thi= s is<BR>> > not always the case. In fact, it is more often NOT the ca= se than it<BR>> > is... If we have two systems a and b and these syst= ems do not have time<BR>> > synchronized, and have a system c which i= s the event collector (and<BR>> > collects only events from a and b),= then c may record time stamps inside<BR>> > its log that do not mono= tonically increase. For example, it may record:<BR>> ><BR>> > 0= 2:00:00 event a1<BR>> > 01:00:00 event b1<BR>> > 02:01:00 event= a2<BR>> > 01:01:00 event b2<BR>> ><BR>> If we keep the orig= inal log intact, the out of sync timestamps are information in themselves.<= BR><BR>My point was just in relation to that log. You even do not know - wi= th<BR>only the information you see - if the log is out of sync. For example= ,<BR>some time zone may be improperly configured. As many devices do not<BR= >convey timezone info, you need to reset to metadata to do that. So in<BR>s= hort: the timestamp alone is insufficient.<BR><BR>><BR>> > Of cour= se, this is still a TIMED record of occurrences. However, in this<BR>> &= gt; sense this is used, "TIMED" includes a sense of temporal order (at leas= t<BR>> > to me). In the above log, we may not have the correct = temporal order.<BR>> > We may be able to reconstruct it by sorting on= the timestamp. That would<BR>> > be a valid approach if the timestam= ps are indeed correct (compared to<BR>> > universal time). But if a a= nd/or b has incorrect time, we would create a<BR>> > wrong temporal o= rder. Indeed, in this sense the monotonically increasing<BR>> > ident= ity of the log in question would actually not be the timestamp but<BR>> = > rather the *sequence of recording*, kind of a meta-property not direct= ly<BR>> > contained in the property set of the individual event recor= d (but rather<BR>> > obtained by its relationship to its predecessor = in the log file).<BR>> ><BR>> If a file is deleted before it is cr= eated, then we locate the shift in timestamps. Different context has = different meaning. For<BR>> instance, a timestamp shift coincident= with the file deletion says real badness, while a timeshift occuring an od= d time before<BR>> (3hrs, 24mins, 5 secs) may just indicate that an admi= n noticed that ntpd had died. Rather than try to sync the time in the= <BR>> logs, what can we do with variations? Automatically?<BR><BR>I am n= ot talking about a shift in timestamp. I am talking about<BR>insufficient r= esolution. The system I describe is working fine, but it<BR>emits only seco= nd-resolution timestamps. For a file delete and file<BR>create, you can not= differentiate any sequence without resorting to<BR>meta-information (like = sequence of records inside the event log).<BR><BR>><BR>> > Now let= 's assume a log without a timestamp. These things happens, e.g.<BR>> >= ; in debug logs (and all too often in others I have seen).<BR>> ><BR>= > > If we define<BR>> ><BR>> > > Log =3D a TIMED recor= d of the above occurence.<BR>> ><BR>> > such a "log" would obvi= ously not be a log, because it does not fulfill<BR>> > the requiremen= t to be timed.<BR>> ><BR>> > If a log instead is "a record of e= vents with a sequence identifier",<BR>> > that problem does not exist= . The sequence identifier in that case would<BR>> > be the derived pr= operty I mentioned above.<BR>> ><BR>> The sequence is the order in= which it appears in the log, plus or minus network latency. (wait fo= r it)<BR><BR>What happens if the logs were queued for some time at an inter= im system?<BR>Again, you need to have the meta-information, just the timest= amp is<BR>insufficient.<BR><BR>><BR>> > The question remains if su= ch a definition is actually useful. The<BR>> > sequence identifier is= obviously something with very vague semantics.<BR>> > They depend on= the observer as well as the correctness of the "sequence<BR>> > iden= tifier generating function" on all systems in question.<BR>> ><BR>>= ; > Let's get back to the simple case of timestamps: as outlined above, = the<BR>> > semantics of timestamps depend on time sync. Even if there= is ntp<BR>> > timesync, timestamps (with reasonable precision) are a= lways<BR>> > questionable. They are approximate, even on the same sys= tem. With<BR>> > standard syslog timestamps (second precision!) the p= roblem is easy to<BR>> > see: one may receive hundreds of events with= in the same second. So even<BR>> > if time is correct, an observer is= unable to detect any order of events.<BR>> > If looking just at the = timestamps, one must conclude that all events<BR>> > happened at once= . If looking at the semantics of the messages, one most<BR>> > often = also can conclude this is impossible (e.g. how to delete a file<BR>> >= ; before it is created?). Obviously, the timestamp alone is never<BR>> &= gt; sufficient to detect order of events, even on a single system. Granted,= <BR>> > for practical purposes a high resolution timestamp (with good= time<BR>> > synchronization) is most often a sufficiently well appro= ximation of the<BR>> > time an event happened. But do you really trus= t it? ...always? Have a<BR>> > look at your own correlation engines: = do they work on pure timestamps -<BR>> > or do they include some othe= r properties, like the order of event log<BR>> > records inside the l= og?<BR>> ><BR>> Why are you looking at logs? Is the machine = broke? There should be plenty of context to render a correct determin= ation. Is<BR>> the machine compromised? Then you must find a= n independent log that says roughly the same thing.<BR><BR>Of course. But, = and this is my point, you can not just rely on the<BR>timestamp (as you say= ).<BR><BR>><BR>> > Now let me try to define what I think a log act= ually is:<BR>> ><BR>> > An EVENT is a set of properties that de= scribe a state change (in the<BR>> > sense I have described state cha= nge yesterday). The contents of this set<BR>> > is depending on the e= ntity who's state changes as well as on the<BR>> > observer. [so it m= ay actually be a set of two sets: entity-related<BR>> > properties an= d observer-related properties]<BR>> ><BR>> > An event is genera= ted when a state changes.<BR>> ><BR>> "when" being the temporal as= pect, recorded from the view of the program sensing the state change.<BR><B= R>I fully agree. IMO a "state change" is also a temporal aspect. I think<BR= >"state *change*" implies that something has *now* state y while it had<BR>= state x a (maybe very) short while ago. So I think there is always a<BR>tem= poral component inside a state change.<BR><BR><snip></DIV> <DIV class=3DPlainText><FONT face=3D"times new roman"></FONT><BR>><BR>&g= t; > There is no inherent order inside event logs. In practice we experi= ence<BR>> > a "natural order" (one record begins before another), but= that actually<BR>> > is a meta-property of the event record. So we c= an order event records<BR>> > based on their meta-properties. It just= happens that a text log is<BR>> > physically ordered by the sequence= meta property.<BR>> ><BR></DIV> <DIV class=3DPlainText><FONT face=3D"times new roman"><snip></FONT><B= R><BR>> > ><BR>> > > On Thu, Jul 24, 2008 at 7:23 PM, Bil= l Scherr IV <[email protected]><BR>> > > wrote:<BR>> >= > > So...<BR>> > > ><BR>> > > > I gather a t= emporal mention to be appropriate beyond the definition<BR>> > > o= f the Log. Also, most systems break off their logs by<BR>> > &g= t; > size, not time. Although there is a definite time to each log= , they<BR>> > > are not consistent, even with the same log gathere= r. Right or<BR>> > > > wrong, that is how I find them.&nb= sp; Suggestions below (if I may be so<BR>> > > bold):<BR>> >= > ><BR>> > > > Circa 11:26, 23 Jul 2008, a note, claimin= g source Heinbockel, Bill<BR>> > > <[email protected]>, w= as sent to me:<BR>> > > ><BR>> > > > From: &nb= sp; = "Heinbockel, Bill" <[email protected]><BR= >> > > > To: &nb= sp; <l= [email protected]><BR>> > > > Subject: &= nbsp; &nbs= p; [logs] How to define Log, Event, and Alert?<BR>> > > ><BR>&g= t; > > >><BR>> > > >><BR>> > > >>= Here is our initial shot at defining these terms:<BR>> > > >&g= t;<BR>> > > >><BR>> > > >> Event:<BR>> >= ; > >> A discrete, distinct, a= nd discernible state change in an<BR>> > > >> environment.<B= R>> > > ><BR>> > > > A discrete, distinct, and disc= ernible state change in an environment<BR>> > > at a recorded (or = given) time.<BR>> > > >><BR></DIV></FONT></BODY></HTML> --_000_7E6E73F62E06444BBB393010835F8866mimectl_-- --===============1008890044== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ LogAnalysis mailing list [email protected] http://www.loganalysis.org/mailman/listinfo/loganalysis --===============1008890044==--