Re: draft-cloud-log-00 / CEE - why not IPFIX?

--===============1573721730==
Content-Language: en-US
Content-Type: multipart/signed; micalg=SHA1;
	protocol="application/x-pkcs7-signature";
	boundary="----=_NextPart_000_0033_01CBCE96.C12A9240"

------=_NextPart_000_0033_01CBCE96.C12A9240
Content-Type: text/plain;
	charset="UTF-8"
Content-Transfer-Encoding: 7bit

Thanks for that, I did not realize that IPFIX had been extended beyond its netflow 
past.

I don't have the time now, but I am interested in looking into it further. It does 
kind of remind me of ASN.1/SNMP, where we need to worry about the names/OID 
translation

Also, a lot of vendors and users seem to prefer the ease of text-based protocols 
like Syslog for logging. I am not saying this is good or bad, but it seems to be 
the sweetspot -- binary is not natively human readable and XML has too much 
overhead.


William Heinbockel
The MITRE Corporation


>-----Original Message-----
>From: Jeroen Massar [mailto:[email protected]]
>Sent: Wednesday, 16 February 2011 05:35
>To: Heinbockel, Bill
>Cc: [email protected]; Chris Lonvick; Gene Golovinsky; Sam Johnston; Common
>Event Expression
>Subject: Re: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX?
>
>On 2011-02-16 06:21, Heinbockel, Bill wrote:
>> From what I understand, IPFIX is for expression of IP flows from network
>sensing
>> devices.
>
>For a short bit forget about the history of IPFIX, it indeed comes from
>NetFlow, and thus is used quite in a network centric way, but
>effectively it is a structured streaming data format.
>
>> Could you please explain how IPFIX is relevant to event and cloud logging
>data?
>> I understand how CEE and IPFIX may overlap for describing networking
>events, but
>> it is unclear to me how IPFIX could handle things like Windows Event Logs
>and
>> RHEL audit logs.
>
>There are two parts to IPFIX: Templates + Data
>
>The template describes how the data looks like, for instance, lets take
>an Apache CLF log entry:
>
>66.249.66.174 - - [16/Feb/2011:10:48:11 +0100] "GET /robots.txt
>HTTP/1.1" 200 2629 "-" "Googlebot-Image/0"
>
>We can make an IPFIX template for that
>
>[
>	{4, IPv4_SRC },
>	{4, TIMESTAMP},
>	{4, HTTP_METHOD},
>	{v, URL},
>	{v, HTTP_PROTOCOL},
>	{2, HTTP_RESULT},
>	{8, OCTETS},
>	{v, HTTP_REFER},
>	{v, HTTP_USERAGENT},
>]
>
>The 'v' markers indicate variable fieldlengths, the others indicates the
>number of bytes such a field takes. The data is then just encoded in the
>above format, presto.
>
>The above is a simple example, one can also have repeating lists and of
>course you could make a variable template which just includes the fields
>that you actually want to look at or you could already do some
>aggregation and add other fields. Templates are only sent every now and
>then, as they should not change. The data is the important bit.
>
>The fieldnames are actually numbers in the data, thus very compact, and
>are mapped to descriptions, data types etc, per a nice XML file
> http://www.iana.org/assignments/ipfix/ipfix.xml (or .xhtml or .txt for
>a more human readable version ;) for the official IANA list and with the
>help of Enterprise IDs any others can easily be added.
>
>The big advantage is that you can more or less do static templates if
>you want and you only need one single parser on the collector side, thus
>one does not have to create another parser and collector again for
>decoding other protocols, just one, the IPFIX one, and you can optimize
>that really well for all kinds of scenarios.
>
>Greets,
> Jeroen

------=_NextPart_000_0033_01CBCE96.C12A9240
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIKxTCCA2Qw
ggJMoAMCAQICAQEwDQYJKoZIhvcNAQEFBQAwWjESMBAGA1UEChMJbWl0cmUub3JnMR4wHAYDVQQL
ExVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxJDAiBgNVBAMTG01JVFJFIENvcnBvcmF0aW9uIFJvb3Qg
Q0EtMTAeFw0wNjA2MDEwNDAwMDBaFw0xODA2MDEwNDAwMDBaMFoxEjAQBgNVBAoTCW1pdHJlLm9y
ZzEeMBwGA1UECxMVQ2VydGlmaWNhdGUgQXV0aG9yaXR5MSQwIgYDVQQDExtNSVRSRSBDb3Jwb3Jh
dGlvbiBSb290IENBLTEwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCva1qWPZiEJv5v
MtCbjt0cTu0Nbn15Q1cKqQBXKi8VSH9zZPmPxfWizJJ7JSqFJ5/sLUz3NsnUVjpLYBdFcxNXnOLj
XtmDPFOewm5T98NZc9wRRCiDzt4f8qsHFI19ShPiK3cN5UqtJf+i66QVLA1S6CNL6o2eGAsAl5Wn
xwOh2BfcWU5fNlHDVc9KKAlDDWpHjC2LLHAUbLP4ZzMIJKcLgLKFMtgM2AEfaSHzmi7WUdUHRCtC
blrF7qzPsy/jBLFrr8VcX+mb7saq95pEOilgcix0/naW7kJfM5ph7UBB+S1O/OhH+ZjQ4MjWnwE8
A/YDrQx1OVLAOi29Bsho/l8lAgMBAAGjNTAzMBIGA1UdEwEB/wQIMAYBAf8CAQMwHQYDVR0OBBYE
FMdwUQDYTf7kAdRolsU9n5qX/nQvMA0GCSqGSIb3DQEBBQUAA4IBAQAa+fVfCljimBlcfWwkfJXu
XNKWun9xloFKjnq6SPGgAIKi5LUDil60a0NaNGoGSO3I1xzYt7ncayh21qXulcVTDFqubSJdv51a
HTuJYcYUX72LN/gSq03UVLBCJzYm7ZLUlkb2YLo7xUeZ3coLFcT5AHR36kjG4cYHqXgH0liBl8jx
pN0gwgaci4sgPLUj1w4t8zoKH+zxGFwXwTP/P+etQqiJZ5T00fLLm5kz9mmnxxmmIvUGNdsCAhGh
dnF24pcrR43LNgyOBJ9DPUHBNq3kUQRO48WBKxBxflOtKzsICx/HEtIABcZn7deADHcY9spULZfB
nQYdEpyz5tgh7Y2qMIIDcTCCAlmgAwIBAgICTU0wDQYJKoZIhvcNAQEFBQAwXTESMBAGA1UEChMJ
bWl0cmUub3JnMR4wHAYDVQQLExVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxJzAlBgNVBAMTHk1JVFJF
IENvcnBvcmF0aW9uIFByaW1hcnkgQ0EtMTAeFw0xMDEyMjQxNzAyMDhaFw0xMjA2MDMxNzEzMjJa
MGExEjAQBgNVBAoTCW1pdHJlLm9yZzEPMA0GA1UECxMGUGVvcGxlMRowGAYKCZImiZPyLGQBARMK
aGVpbmJvY2tlbDEeMBwGA1UEAxMVSGVpbmJvY2tlbCBXaWxsaWFtIEouMIGfMA0GCSqGSIb3DQEB
AQUAA4GNADCBiQKBgQD1fnZZxPrtC6BEKYX33ph9tumgzYQTNNIWCUCR+h0AudF0Pxb+wRDeDZps
VIylRejdkCQ4td+J2WZI/QdeqMREpYWHVDR64TQ7kgIImOPgzYa5Ttu7byFBtGkaBJnRvcqgK23j
pBiPCfIY6k0GXGdQjKpEn5kTVv1zm2HpyPZcewIDAQABo4G6MIG3MA4GA1UdDwEB/wQEAwIF4DAd
BgNVHQ4EFgQU6RMrsHGS4C3d+ljqPBM8ENcv+lIwHwYDVR0jBBgwFoAUh7QPSI1iM0LBLVEaSB7C
nrsKsa0wRAYDVR0fBD0wOzA5oDegNYYzaHR0cDovL3d3dy5taXRyZS5vcmcvdGVjaC9taWkvcGtp
L2NhMV9taXRyZV9vcmcuY3JsMB8GA1UdEQQYMBaBFGhlaW5ib2NrZWxAbWl0cmUub3JnMA0GCSqG
SIb3DQEBBQUAA4IBAQAvr07pYzYYAx+28PYGnbmgA32RiE8NAzc64pGLXDNa34xFWTjDFIY9levD
TQhoZ1fXo1dNtXvB45Ilh+Jsf3+9XTe2vYNf8vcEnxu+buddzsdxz2U2KvGbQNlcxat+zEtY0eWn
qZqmMXbhyTiVyziGGJSWnFKDyFOJFkq3cuK29IOYU+9/vZDQE2QBAzCViMvWcAGs6QfTlWS4gqV3
kdkBAmqeV6MgFr+BhUPjjdmYYgSU0CPH6ql0j1j/y2Yd4IK/Er+SFaU6T/ri+7ytSHDjAI+Cub5x
Pv8AiiaErv7KH+3RhG+suNJ/QlNjzuXLgr5RcufHUt38MT5ibca6xBebMIID5DCCAsygAwIBAgIB
BTANBgkqhkiG9w0BAQUFADBaMRIwEAYDVQQKEwltaXRyZS5vcmcxHjAcBgNVBAsTFUNlcnRpZmlj
YXRlIEF1dGhvcml0eTEkMCIGA1UEAxMbTUlUUkUgQ29ycG9yYXRpb24gUm9vdCBDQS0xMB4XDTA2
MDYwMzE3MTMyMloXDTEyMDYwMzE3MTMyMlowXTESMBAGA1UEChMJbWl0cmUub3JnMR4wHAYDVQQL
ExVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxJzAlBgNVBAMTHk1JVFJFIENvcnBvcmF0aW9uIFByaW1h
cnkgQ0EtMTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMjwe1ZdEIKoELdLvENnbkbO
3mVD54ZX5SjxTzFxhvoqhqSJmLOp32xT7J9KvBapJRbJv1BFdiSUN3O0q8H66nvQqwm1RYe+tTsW
SO351FolGrDT9iDRtfWgH60PCKCbABLRsx3iGnEvjOQjeAtMn1AugqQWU3PWZXYy1Grbya+NOytY
vu1p60bDFSoSAn4Jojv14lUfWHl8s3kahay8umLSQibiXdEEX8CrSkaXpOaFOuyM6+wpRw7TybO1
Dk4zGAUtn0887QvsMDk6evgN2WxMprkHAGUcJhpI1QXtkfDIl9ukdNiIoM/vdN2QC/+m6b8dA55K
5UdlBa9SgRnwapkCAwEAAaOBsTCBrjASBgNVHRMBAf8ECDAGAQH/AgECMA4GA1UdDwEB/wQEAwIB
hjAdBgNVHQ4EFgQUh7QPSI1iM0LBLVEaSB7CnrsKsa0wHwYDVR0jBBgwFoAUx3BRANhN/uQB1GiW
xT2fmpf+dC8wSAYDVR0fBEEwPzA9oDugOYY3aHR0cDovL3d3dy5taXRyZS5vcmcvdGVjaC9taWkv
cGtpL3Jvb3RjYTFfbWl0cmVfb3JnLmNybDANBgkqhkiG9w0BAQUFAAOCAQEATW5u664p7N0iAj27
Xl/akjdfkSQpaosf6cNyAHu7utCytFfY1WfRNmvnNDGYkqI3XMFOa18SNjiNsMCH+sFQaO+oyDnP
iIkEZQvlfGGrRpqIm6j//Fgz85bnf1kAM5I61Np7ofCnciRvp9ZB/+u+9i262tgiJPJrvBcqXmge
T9riCc3RPjxqPNmYslOvNLpIifchelJhF7nIge+7RkAUcTJenj8yKwK0J3+PEpgYRQ+V2C62rnjo
huxPgMw/fYoNTOlh3MVl7adwyK1ahPw2a9eOjSWglqoPTaBNeHJqRJZZ6Vi7S55+VAWCfkAqM5m3
tUiVzjsp2dFcTJxnYezaoDGCAr0wggK5AgEBMGMwXTESMBAGA1UEChMJbWl0cmUub3JnMR4wHAYD
VQQLExVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxJzAlBgNVBAMTHk1JVFJFIENvcnBvcmF0aW9uIFBy
aW1hcnkgQ0EtMQICTU0wCQYFKw4DAhoFAKCCAbAwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAc
BgkqhkiG9w0BCQUxDxcNMTEwMjE3MTYzNTE5WjAjBgkqhkiG9w0BCQQxFgQU4hk/CwNI23V0m+GV
fEcPtyJUX6gwZwYJKoZIhvcNAQkPMVowWDAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYI
KoZIhvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwBwYFKw4DAhowCgYIKoZIhvcNAgUw
cgYJKwYBBAGCNxAEMWUwYzBdMRIwEAYDVQQKEwltaXRyZS5vcmcxHjAcBgNVBAsTFUNlcnRpZmlj
YXRlIEF1dGhvcml0eTEnMCUGA1UEAxMeTUlUUkUgQ29ycG9yYXRpb24gUHJpbWFyeSBDQS0xAgJN
TTB0BgsqhkiG9w0BCRACCzFloGMwXTESMBAGA1UEChMJbWl0cmUub3JnMR4wHAYDVQQLExVDZXJ0
aWZpY2F0ZSBBdXRob3JpdHkxJzAlBgNVBAMTHk1JVFJFIENvcnBvcmF0aW9uIFByaW1hcnkgQ0Et
MQICTU0wDQYJKoZIhvcNAQEBBQAEgYC2urY2WkzIQJpCWNNiODQrvmsz0R7mUIWEnMsM2ADRUsWQ
wKxBXJ91tSwU2jGvVttSbalKseabyyTqDw3OzYhhM7w9fUexKlPZZJTCjJcXzqsTdaMjZ+oJm2+K
RlzG2PLlxMatU5HdPX/ZB20RPu21VxWLcYy4Mmq0n4j9XsTekwAAAAAAAA==

------=_NextPart_000_0033_01CBCE96.C12A9240--

--===============1573721730==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Syslog mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/syslog

--===============1573721730==--