Re: Question: DARPA's Common Intrusion Detection Framework (CIDF) and the IDWG effort to refine the CIDF and get it commercially accepted

Robert,

Your conclusion is mostly correct.

CIDF was a DARPA funded research effort leaded by Stuart Staniford and
many other DARPA PI's in late 90. The objective was to standardize it
via the IETF process. However, the first BOF in LA, I still remember,
was not very well received by the audience due to the potential
complexity and expressiveness of s-expressions. But, it was clear from
the BOF that there was a significant interest in having an industry
standard.

Therefore, the IDWG working group was formed, and Stuart and Mike were
the first chairs for the working group. IDWG basically abandoned CIDF
and S-expression (at least initially) and started from "requirement
study". We then ran into the debate of issues such as SNMP versus XML.
S-expression was not discussed very much after that point.

I personally believe that an industry standard is still needed as we
have many more powerful networ/host IDS systems today. And, from the
research community, I feel that we know a lot better about what should
be standardized than say 7~8 years ago. And, I see a critical need for
interoperability and inter-domain security operations now and in the
future. I think we should think about all the issues again.

-Felix

Dr. Robert G. Rains wrote:
> Gentlemen,
> 
> I have a question regarding a "standard" that apparently originated at 
> DARPA in the late 90's and then transitioned from DARPA to the IDWG so 
> that the IDWG could refine the specification and further its 
> acceptance.  The "standard" is the Defense Advanced Research Program 
> Agency (DARPA) "Common Intrusion Detection Framework" (CIDF).  I'm 
> working on standards for Anti-Terrorism/Force Protection (AT/FP), and an 
> integration contractor has proposed that the DoD adopt the "DARPA 
> Intrusion Detection Systems" aka "Common Intrusion Detection Framework" 
> (CIDF) as a "standard" to be imposed on AT/FP systems.
> 
> MY CURRENT UNDERSTANDING
> It appears that the standard titled, "DARPA Intrusion Detection Systems 
> (Network)" is not a standard.  It seems to be an area of research.  
> Although DARPA has funded some leading edge research into concepts for 
> intrusion detection, DARPA is not currently the lead for development of 
> standards related to intrusion detection.  An April 1999 DARPA report 
> stated, "... intrusion detection systems must be able to interoperate 
> with security screens, such as network firewalls, and with response 
> mechanisms that can be used to contain and/or mitigate the effects of an 
> attack.  To enable such interoperation, DARPA has developed a Common 
> Intrusion Detection Framework that allows network components to exchange 
> information and to work together to respond to network-based attacks. 
> The Internet Engineering Task Force, the organization that maintains the 
> standards that govern the Internet, will refine the specification and 
> further its acceptance."  I understand that the standards portion of t!
> he activity was being pursued under the auspices of the IETF.  A 
> somewhat dated history of the CIDF effort may be found at 
> http://www.isi.edu/gost/cidf.  However, many of the organizational 
> references, e-mail addresses, and URLs on that page are now out of 
> date.   Development of intrusion detection standards was being pursued 
> by the Intrusion Detection Exchange Format Working Group (idwg) of the 
> Internet Engineering Task Force (IETF).  However, in the e-mail chain 
> below, Sam Hartman states, "Based on my own conclusions and on the 
> strong opinions of other area directors I've consulted with, IDWG no 
> longer constitutes an active working group.  There is an insufficient 
> core of active participants--particularly participants not involved in 
> the document--to generate an informed consensus on changes."
> 
> QUOTE FROM THE CONTRACTOR SUBMITTAL PROPOSING THE CIDF AS A STANDARD
> The Defense Advanced Research Program Agency (DARPA) "Common Intrusion 
> Detection Framework" (CIDF) specifies a set of concepts and guidelines 
> that provide a framework within which Intrusion Detection Systems (IDS) 
> can be understood, compared and designed. An IDS system's architecture, 
> capabilities and features can be defined in relation to relationships 
> with the DARPA CIDF.  The IETF has formed an Intrusion Detection Working 
> Group (IDWG) that is also providing guidance in regard to IDS 
> standards.  The Gartner Groups "Seven Key Selection Criteria for Network 
> Intrusion Prevention Systems (IPS)" is also a valuable framework for IDS 
> and IPS. The OASIS Security Assertion Markup Language (SAML 2.0) also 
> relates to standard security measures.
> 
> REFERENCES
> 1)    http://www.toplayer.com/pdf/GartnerBrief.pdf
> 2)    http://www.isi.edu/gost/cidf/
> 3)    http://www.isi.edu/gost/cidf/drafts/communication.txt
> 4)    http://www.isi.edu/gost/cidf/drafts/language.txt
> 5)    http://www.isi.edu/gost/cidf/tutorial.html
> 6)    http://www.isi.edu/gost/cidf/drafts/api.txt
> 7)    
> http://www.ietf.org/internet-drafts/draft-ietf-idwg-requirements-10.txt
> 8)    http://www.ietf.org/internet-drafts/draft-ietf-idwg-idmef-xml-12.txt
> 9)    http://www.ietf.org/internet-drafts/draft-ietf-idwg-beep-idxp-07.txt
> 10)    http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=security
> 
> 
> MY CURRENT CONCLUSION
> In conclusion... I don't see any evidence that the DARPA CIDF ever 
> became a commercially accepted standard.  None of the above references 
> seem to refer to a DARPA web site or to a mature, commercially accepted 
> standard.  I understand that, at this time, the IDWG is for all 
> practical purposes, not an active working group.  It appears that this 
> is an area of research that DARPA was interested in for awhile back in 
> the late 90's, and for which DARPA eventually gave responsibility to the 
> IETF to refine the "standard" and seek its acceptance.  It isn't clear 
> that the CIDF ever became a mature, commercially accepted "standard" 
> which would lead to commercial products that comply with the standard 
> being developed and placed on the market.
> 
> I would sincerely welcome any additional information that you might have 
> which would shed light on the status of the CIDF.
> 
> ONE LAST QUESTION:
> Did draft-ietf-idwg-idmef-xml-14 expire on 31 July 2005, or has it been 
> adopted as a standard?  To the best of your knowledge, are at least two 
> different commercial organizations developing products based on this 
> draft standard?
> 
> Thanks in advance,
> 
> Robert Rains
> 
> *****************************************************************
> Robert G. Rains, Ph.D.
> Force Protection C2 Systems Engineer
> Electronic Systems Center
> Force Protection Systems Squadron, FPSS/FPTE
> 5 Eglin St, Bldg 1624
> Hanscom AFB, MA 01731-2100
> Phone: 781-377-4744
> E-Mail:  [email protected]
> *****************************************************************
> 
> =============================================================
> 
>