Re: Question about loss and retransmission reporting
Carter Bullard <[email protected]> Thu, 13 Dec 2018 08:56:33 -0500
| Newsgroups | gmane.network.argus |
|---|---|
| Message-ID | <[email protected]> |
--Apple-Mail=_97DB8316-5B44-43BC-B731-56C39C6B8354 Content-Type: multipart/alternative; boundary="Apple-Mail=_AC77B83A-62B2-44F2-91C8-273ADE02F359" --Apple-Mail=_AC77B83A-62B2-44F2-91C8-273ADE02F359 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hey Edward, Can you share the pcap file so that I can see what=E2=80=99s up ??? I = did suggest that you try to see which direction is involved, as I think = we do a bit of a different job than others on the return traffic loss = determination (sloss, dloss). Did that reveal anything ??? Carter > On Dec 11, 2018, at 11:52 AM, Balas, Edward G <[email protected]> wrote: >=20 > Carter, >=20 > Ive dug a bit deeper reviewing the original pcap that caught my eye. = The concern / confusion at this point is that most other utilities = report similar values ~12,097 whereas Argus is reporting value thats ~ = 500% larger.=20 >=20 >=20 > 1. I ran tshark and =E2=80=98manually=E2=80=9D counted multiple = instances of sequence numbers when tcp.len is greater than 1 ie not an = ack >=20 > accuracy-test]# tshark -r no-vlan.pcap -nn -Y 'tcp.srcport=3D=3D35940 = and tcp.len > 1' -T fields -e ip.src -e ip.dst -e tcp.dstport -e = tcp.seq | sort | uniq -dc | wc -l. > 12087=20 >=20 > (this is unique instances not total count, though each sequence number = showed up twice in spot checking) >=20 >=20 > 2. next I looked at what tshark's own tcp analysis would result in.=20 > tshark -r no-vlan.pcap -nn -Y 'tcp.port=3D=3D35940 and = tcp.analysis.retransmission' -T fields -e ip.src -e ip.dst -e = tcp.dstport -e tcp.seq | wc -l=20 > 13610 >=20 > -this number is is different to be sure but within same order of = magnitude >=20 > 3. next we ran this pcap through tcptrace and it also reported 12087=20= >=20 > 4. finally ran argus / racluster is showing the following: =20 >=20 > accuracy-test]# racluster -n -r no-vlan.argus -M = dsrs=3D+metric,+asn,+net,+agr,+ipattr -s +trans, +cause, = +retrans,+loss,+appbytes, dur > StartTime Flgs Proto SrcAddr Sport Dir = DstAddr Dport TotPkts TotBytes State Trans Cause = Retrans Loss TotAppByte Dur=20 > 18:32:30.061403 e s tcp xxx.xxx.123.97.35940 -> = xxx.xxx.106.139.5201 309477 2089885325 RST 3 Start = 0 66189 2068372573 10.294667 >=20 > - it should be noted that the metrics for total app bytes, and total = packets are consistent with other utilities and all were generated from = the same pcap file. > - ra is unable to return retrans values even though in the source code = loss is calculated from retrans? >=20 >=20 >=20 > The code path that trigged the 66,189 reported retransmissions is as = follows: >=20 > ArgusTcp.c >=20 >=20 > if (ArgusThisTCPsrc->win) { > int dipid; > if (*tipid && ((dipid =3D (ipid - *tipid)) < = 0) && (dipid > -5000)) { > ArgusThisTCPsrc->status |=3D = ARGUS_OUTOFORDER; > } else > if (ArgusThisTCPsrc->winbytes > ((maxseq - 1) = - ArgusThisTCPdst->ack)) { > ArgusThisTCPsrc->retrans++; > ArgusThisTCPsrc->status |=3D = ARGUS_PKTS_RETRANS; > ArgusThisTCPsrc->winbytes -=3D len; > printf(=E2=80=9CThe code path we are = hitting is here\n"); > model->ArgusInProtocol =3D 0; >=20 > } else { >=20 >=20 > Im not super familiar with the source, could you share a bit as to = whats going on in this situation? and also any insights as to why loss = is showing values but not retransmissions is always 0? >=20 >=20 >=20 > Thanks Much! >=20 >=20 >=20 >=20 >> On Dec 7, 2018, at 1:45 PM, Balas, Edward G <[email protected] = <mailto:[email protected]>> wrote: >>=20 >>=20 >>=20 >>> On Dec 7, 2018, at 1:21 PM, [email protected] = <mailto:[email protected]> wrote: >>>=20 >>> Hey Edward, >>> Its always important to indicate which version your running, so we = can deal with known bugs etc =E2=80=A6 >>=20 >> Sorry about that=E2=80=A6 I hadn't gotten to point of thinking there = was a bug =E2=80=A6 >> 3.0.8.2 is what I running now >>=20 >>>=20 >>> Argus tracks loss in 3 basic conditions, connection oriented = protocols such as TCP, connection-less protocols that have sequence = numbers, such as RTP, UDT, and IPSEC, and strict request / response = protocols where you should see the same out as back. >>>=20 >>> Argus has a complete TCP state machine so that it can identify = requests for missing packets, retransmissions and out of order packets. = But argus is designed to recognize loss regardless of where it is along = the path. As a result, the algorithm is a little complex, mainly = because TCP is reliable and regardless of the loss rate you should = always see at least one copy of all the packets. Because argus is a = bi-directional flow monitor, argus can do things like look for requests = for retransmission as an indication of loss. It can infer that = observing multiple packets is an indication of loss (you don=E2=80=99t = retransmit unless there was loss), and it can needs to do this in the = event of stripping and asymmetric routing. >>=20 >> yeah we are defn interesting in the asymmetic situation, for now we = are validating using point of observation where traffic appears = symmetric. >>=20 >>>=20 >>> Because loss can occur before and or after argus see=E2=80=99s the = packet stream, argus will use retransmissions and retransmission = requests from the far side as an indication of loss. If the far side = requests more than once, we assume that the packet was lost more than = once, or that the retransmission request was lost. This is a possible = source for argus saying there is more loss than other tools. >>=20 >> Ah ok that is in part what I was curious about, so if one were to = look at the individual packets in a flow and count the number of times = any sequence number shows up more than once, they should presumably come = up with the same value as argus loss calc? >>=20 >>=20 >>> Now with that as a starting point, where is argus in relationship = with the the other tools, and what methods are they using to determine = loss ??? >>=20 >> Yeah this is a classic dilemma have having too many watches and not = knowing which is correct. What we have for a reference is ultimately = what iperf3 is reporting, and we are starting to look at what the kernel = can tell us as presumably the tcp implementation should have a pretty = authoritiative view of its own behavior, but I havent dug into proc = filesystem etc enough to know whats available. Im not really sure yet = what tsharks methodology is for determining retransmission, I have been = presuming it is similar to what you described, Ill take a look at = sequence numbers directly to see whats up and report back. >>=20 >>>=20 >>> Carter >>> =09 >>> Carter Bullard =E2=80=A2 CTO >>> 150 E 57th Street, Suite 12D >>> New York, New York 10022-2795 >>> Phone +1.212.588.9133 =E2=80=A2 Mobile +1.917.497.9494 >>>=20 >>>=20 >>>=20 >>>> On Dec 7, 2018, at 12:28 PM, Balas, Edward G <[email protected] = <mailto:[email protected]>> wrote: >>>>=20 >>>> Hey all, >>>>=20 >>>> Ive run into an issue Im struggling to understand, and thus far = googling has failed to right me. I am trying to use Argus to track = retransmissions / loss in flows and I am getting values that are = inconsistent with other tools including the sending application. As I = recurse into the various rabbit holes contributing to this on our end, I = was wondering if someone could guide me on the following: >>>>=20 >>>> 1. within ra etc there is the ability to report loss and retrans. = When I look at the documentation loss seems to imply it contains both = retransmissions and dropped packets, if Im looking at a TCP flow, is it = correct to assume there will be no drops and thus loss is synonymous = with retransmission? >>>>=20 >>>> 2. I am able to get ra and racluster to report loss values for my = flows, however retrans is always 0, is there a special -M or other = options or argus option I need to use to see retrans? Im making the = possibly bad assumption that because I can see loss values the tunings = of argus are sufficient. >>>>=20 >>>> 3. The Loss numbers are always higher than what I am seeing with = other applications, is there a document or place in the code I should go = look at that describes how this is calculated? >>>>=20 >>>> Motivating these questions is the following small test: >>>> ------------------------------------------------------- >>>> I transfered a file to my test host while doing full snaplen packet = capture, and then compared argus with tshark reports of loss and = retransmission.=20 >>>>=20 >>>> accuracy-test2]# argus -JA -r raw.pcap -w raw.argus >>>>=20 >>>> accuracy-test2]# racluster -n -r raw.argus -s = stime,dur,pkts,retrans,loss,appbytes,cause -- port 51170 >>>> StartTime Dur TotPkts Retrans Loss TotAppByte = Cause=20 >>>> 16:47:58.176366 15.206044 21513 0 24 20195392 = Start >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>>>> Total packets between tshark and argus agree: >>>>=20 >>>> accuracy-test2]# tshark -r raw.pcap -nn -Y 'tcp.port=3D=3D51170 ' = | wc -l=20 >>>> 21513 >>>>=20 >>>>>> Retransmissions / loss do not agree between tshark and argus: >>>>=20 >>>> accuracy-test2]# tshark -r raw.pcap -nn -Y 'tcp.port=3D=3D51170 = and tcp.analysis.retransmission ' | wc -l=20 >>>> 17 >>>>=20 >>>> 17 vs 24 >>>>=20 >>>>=20 >>>> Was curious if folks had insights they could share in these = regards? >>>>=20 >>>> Thanks, >>>>=20 >>>> Edward Balas >>>> [email protected] <mailto:[email protected]> >>>=20 >>=20 >=20 --Apple-Mail=_AC77B83A-62B2-44F2-91C8-273ADE02F359 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D"">Hey = Edward,<div class=3D"">Can you share the pcap file so that I can see = what=E2=80=99s up ??? I did suggest that you try to see which = direction is involved, as I think we do a bit of a different job than = others on the return traffic loss determination (sloss, dloss). = Did that reveal anything ???</div><div class=3D""><br = class=3D""><div class=3D"">Carter</div><div class=3D""><br = class=3D""><div><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D"">On Dec 11, 2018, at 11:52 AM, Balas, Edward G <<a = href=3D"mailto:[email protected]" class=3D"">[email protected]</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div = style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; line-break: = after-white-space;" class=3D""><font face=3D"Courier" = class=3D"">Carter,</font><div class=3D""><font face=3D"Courier" = class=3D""><br class=3D""></font></div><div class=3D""><font = face=3D"Courier" class=3D"">Ive dug a bit deeper reviewing the original = pcap that caught my eye. The concern / confusion at this point is = that most other utilities report similar values ~12,097 whereas Argus is = reporting value thats ~ 500% larger. </font></div><div = class=3D""><font face=3D"Courier" class=3D""><br = class=3D""></font></div><div class=3D""><font face=3D"Courier" = class=3D""><br class=3D""></font></div><div class=3D""><font = face=3D"Courier" class=3D"">1. I ran tshark and =E2=80=98manually=E2= =80=9D counted multiple instances of sequence numbers when tcp.len is = greater than 1 ie not an ack</font></div><div class=3D""><font = face=3D"Courier" class=3D""><br class=3D""></font></div><div = class=3D""><font face=3D"Courier" class=3D"">accuracy-test]# = tshark -r no-vlan.pcap -nn -Y 'tcp.srcport=3D=3D35940 and tcp.len = > 1' -T fields -e ip.src -e ip.dst -e tcp.dstport -e tcp.seq = | sort | uniq -dc | wc -l.<br = class=3D"">12087 </font></div><div class=3D""><font face=3D"Courier" = class=3D""><br class=3D""></font></div><div class=3D""><font = face=3D"Courier" class=3D"">(this is unique instances not total count, = though each sequence number showed up twice in spot = checking)</font></div><div class=3D""><font face=3D"Courier" = class=3D""><br class=3D""><br class=3D""></font></div><div = class=3D""><font face=3D"Courier" class=3D"">2. next I looked at what = tshark's own tcp analysis would result in. </font></div><div = class=3D""><font face=3D"Courier" class=3D"">tshark -r no-vlan.pcap -nn = -Y 'tcp.port=3D=3D35940 and tcp.analysis.retransmission' -T fields -e = ip.src -e ip.dst -e tcp.dstport -e tcp.seq | wc -l <br = class=3D"">13610</font></div><div class=3D""><font face=3D"Courier" = class=3D""><br class=3D""></font></div><div class=3D""><font = face=3D"Courier" class=3D"">-this number is is different to be sure but = within same order of magnitude</font></div><div class=3D""><font = face=3D"Courier" class=3D""><br class=3D""></font></div><div = class=3D""><font face=3D"Courier" class=3D"">3. next we ran this = pcap through tcptrace and it also reported 12087 </font></div><div = class=3D""><font face=3D"Courier" class=3D""><br = class=3D""></font></div><div class=3D""><font face=3D"Courier" = class=3D"">4. finally ran argus / racluster is showing the following: = </font></div><div class=3D""><font face=3D"Courier" class=3D""><br = class=3D""></font></div><div class=3D""><font face=3D"Courier" = class=3D"">accuracy-test]# racluster -n -r no-vlan.argus -M = dsrs=3D+metric,+asn,+net,+agr,+ipattr -s +trans, +cause, = +retrans,+loss,+appbytes, dur<br class=3D""> = StartTime Flgs Proto = SrcAddr Sport Dir = DstAddr Dport TotPkts = TotBytes State Trans Cause = Retrans Loss TotAppByte = Dur <br class=3D""> 18:32:30.061403 e s = tcp = xxx.xxx.123.97.35940 -> = xxx.xxx.106.139.5201 309477 2089885325 RST = 3 Start 0 = 66189 2068372573 10.294667<br class=3D""><br = class=3D""></font></div><div class=3D""><font face=3D"Courier" = class=3D"">- it should be noted that the metrics for total app bytes, = and total packets are consistent with other utilities and all were = generated from the same pcap file.</font></div><div class=3D""><font = face=3D"Courier" class=3D"">- ra is unable to return retrans values even = though in the source code loss is calculated from = retrans?</font></div><div class=3D""><font face=3D"Courier" class=3D""><br= class=3D""></font></div><div class=3D""><font face=3D"Courier" = class=3D""><br class=3D""></font></div><div class=3D""><font = face=3D"Courier" class=3D""><br class=3D""></font></div><div = class=3D""><font face=3D"Courier" class=3D"">The code path that trigged = the 66,189 reported retransmissions is as follows:</font></div><div = class=3D""><font face=3D"Courier" class=3D""><br = class=3D""></font></div><div class=3D""><font face=3D"Courier" = class=3D"">ArgusTcp.c</font></div><div class=3D""><font face=3D"Courier" = class=3D""><br class=3D""></font></div><div class=3D""><div = style=3D"margin: 0px; font-stretch: normal; font-size: 15px; = line-height: normal; font-family: Menlo; background-color: rgb(235, 233, = 210); min-height: 18px;" class=3D""><span style=3D"font-variant-ligatures:= no-common-ligatures" class=3D""></span><br class=3D""></div><div = style=3D"margin: 0px; font-stretch: normal; font-size: 15px; = line-height: normal; font-family: Menlo; background-color: rgb(235, 233, = 210);" class=3D""><span style=3D"font-variant-ligatures: = no-common-ligatures" class=3D""> = </span><span = style=3D"font-variant-ligatures: no-common-ligatures; color: #bf6d15" = class=3D"">if</span><span style=3D"font-variant-ligatures: = no-common-ligatures" class=3D""> (ArgusThisTCPsrc->win) = {</span></div><div style=3D"margin: 0px; font-stretch: normal; = font-size: 15px; line-height: normal; font-family: Menlo; = background-color: rgb(235, 233, 210);" class=3D""><span = style=3D"font-variant-ligatures: no-common-ligatures" class=3D""> = = </span><span style=3D"font-variant-ligatures: = no-common-ligatures; color: #19ad11" class=3D"">int</span><span = style=3D"font-variant-ligatures: no-common-ligatures" class=3D""> = dipid;</span></div><div style=3D"margin: 0px; font-stretch: normal; = font-size: 15px; line-height: normal; font-family: Menlo; = background-color: rgb(235, 233, 210);" class=3D""><span = style=3D"font-variant-ligatures: no-common-ligatures" class=3D""> = = </span><span style=3D"font-variant-ligatures: = no-common-ligatures; color: #bf6d15" class=3D"">if</span><span = style=3D"font-variant-ligatures: no-common-ligatures" class=3D""> = (*tipid && ((dipid =3D (ipid - *tipid)) < </span><span = style=3D"font-variant-ligatures: no-common-ligatures; color: #b82c18" = class=3D"">0</span><span style=3D"font-variant-ligatures: = no-common-ligatures" class=3D"">) && (dipid > -</span><span = style=3D"font-variant-ligatures: no-common-ligatures; color: #b82c18" = class=3D"">5000</span><span style=3D"font-variant-ligatures: = no-common-ligatures" class=3D"">)) {</span></div><div style=3D"margin: = 0px; font-stretch: normal; font-size: 15px; line-height: normal; = font-family: Menlo; background-color: rgb(235, 233, 210);" = class=3D""><span style=3D"font-variant-ligatures: no-common-ligatures" = class=3D""> = ArgusThisTCPsrc->status |=3D = ARGUS_OUTOFORDER;</span></div><div style=3D"margin: 0px; font-stretch: = normal; font-size: 15px; line-height: normal; font-family: Menlo; = background-color: rgb(235, 233, 210);" class=3D""><span = style=3D"font-variant-ligatures: no-common-ligatures" class=3D""> = = } </span><span style=3D"font-variant-ligatures: = no-common-ligatures; color: #bf6d15" class=3D"">else</span></div><div = style=3D"margin: 0px; font-stretch: normal; font-size: 15px; = line-height: normal; font-family: Menlo; background-color: rgb(235, 233, = 210);" class=3D""><span style=3D"font-variant-ligatures: = no-common-ligatures" class=3D""> = </span><span = style=3D"font-variant-ligatures: no-common-ligatures; color: #bf6d15" = class=3D"">if</span><span style=3D"font-variant-ligatures: = no-common-ligatures" class=3D""> (ArgusThisTCPsrc->winbytes > = ((maxseq - </span><span style=3D"font-variant-ligatures: = no-common-ligatures; color: #b82c18" class=3D"">1</span><span = style=3D"font-variant-ligatures: no-common-ligatures" class=3D"">) - = ArgusThisTCPdst->ack)) {</span></div><div style=3D"margin: 0px; = font-stretch: normal; font-size: 15px; line-height: normal; font-family: = Menlo; background-color: rgb(235, 233, 210);" class=3D""><span = style=3D"font-variant-ligatures: no-common-ligatures" = class=3D""> = = ArgusThisTCPsrc->retrans++;</span></div><div style=3D"margin: 0px; = font-stretch: normal; font-size: 15px; line-height: normal; font-family: = Menlo; background-color: rgb(235, 233, 210);" class=3D""><span = style=3D"font-variant-ligatures: no-common-ligatures" = class=3D""> = ArgusThisTCPsrc->status |=3D = ARGUS_PKTS_</span><span style=3D"font-variant-ligatures: = no-common-ligatures; background-color: #e6e600" = class=3D"">RETRANS</span><span style=3D"font-variant-ligatures: = no-common-ligatures" class=3D"">;</span></div><div style=3D"margin: 0px; = font-stretch: normal; font-size: 15px; line-height: normal; font-family: = Menlo; background-color: rgb(235, 233, 210);" class=3D""><span = style=3D"font-variant-ligatures: no-common-ligatures" = class=3D""> = ArgusThisTCPsrc->winbytes -=3D = len;</span></div><div style=3D"margin: 0px; font-stretch: normal; = font-size: 15px; line-height: normal; font-family: Menlo; = background-color: rgb(235, 233, 210);" class=3D""><span = style=3D"font-variant-ligatures: no-common-ligatures" = class=3D""> = printf(</span><span = style=3D"font-variant-ligatures: no-common-ligatures; color: #b82c18" = class=3D"">=E2=80=9CThe code path we are hitting is here</span><span = style=3D"font-variant-ligatures: no-common-ligatures; color: #c82bc7" = class=3D"">\n</span><span style=3D"font-variant-ligatures: = no-common-ligatures; color: #b82c18" class=3D"">"</span><span = style=3D"font-variant-ligatures: no-common-ligatures" = class=3D"">);</span></div><div style=3D"margin: 0px; font-stretch: = normal; font-size: 15px; line-height: normal; font-family: Menlo; = background-color: rgb(235, 233, 210);" class=3D""><span = style=3D"font-variant-ligatures: no-common-ligatures" = class=3D""> = model->ArgusInProtocol =3D = </span><span style=3D"font-variant-ligatures: no-common-ligatures; = color: #b82c18" class=3D"">0</span><span style=3D"font-variant-ligatures: = no-common-ligatures" class=3D"">;</span></div><div style=3D"margin: 0px; = font-stretch: normal; font-size: 15px; line-height: normal; font-family: = Menlo; background-color: rgb(235, 233, 210); min-height: 18px;" = class=3D""><span style=3D"font-variant-ligatures: no-common-ligatures" = class=3D""></span><br class=3D""></div><div style=3D"margin: 0px; = font-stretch: normal; font-size: 15px; line-height: normal; font-family: = Menlo; background-color: rgb(235, 233, 210);" class=3D""><span = style=3D"font-variant-ligatures: no-common-ligatures" class=3D""> = = } </span><span style=3D"font-variant-ligatures: = no-common-ligatures; color: #bf6d15" class=3D"">else</span><span = style=3D"font-variant-ligatures: no-common-ligatures" class=3D""> = {</span></div></div><div class=3D""><span style=3D"font-variant-ligatures:= no-common-ligatures" class=3D""><br class=3D""></span></div><div = class=3D""><span style=3D"font-variant-ligatures: no-common-ligatures" = class=3D""><br class=3D""></span></div><div class=3D"">Im not super = familiar with the source, could you share a bit as to whats going on in = this situation? and also any insights as to why loss is showing = values but not retransmissions is always 0?</div><div class=3D""><font = face=3D"Courier" class=3D""><br class=3D""></font></div><div = class=3D""><font face=3D"Courier" class=3D""><br = class=3D""></font></div><div class=3D""><font face=3D"Courier" = class=3D""><br class=3D""></font></div><div class=3D"">Thanks Much!<br = class=3D""><br class=3D""></div><div class=3D""><br class=3D""></div><div = class=3D""><br class=3D""></div><div class=3D""><br class=3D""><blockquote= type=3D"cite" class=3D"">On Dec 7, 2018, at 1:45 PM, Balas, Edward G = <<a href=3D"mailto:[email protected]" class=3D"">[email protected]</a>> = wrote:<br class=3D""><br class=3D""><br class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D"">On Dec 7, 2018, at 1:21 = PM, <a href=3D"mailto:[email protected]" = class=3D"">[email protected]</a> wrote:<br class=3D""><br class=3D"">Hey = Edward,<br class=3D"">Its always important to indicate which version = your running, so we can deal with known bugs etc =E2=80=A6<br = class=3D""></blockquote><br class=3D"">Sorry about that=E2=80=A6 I = hadn't gotten to point of thinking there was a bug =E2=80=A6<br = class=3D"">3.0.8.2 is what I running now<br class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D""><br class=3D"">Argus = tracks loss in 3 basic conditions, connection oriented protocols such as = TCP, connection-less protocols that have sequence numbers, such as = RTP, UDT, and IPSEC, and strict request / response protocols where you = should see the same out as back.<br class=3D""><br class=3D"">Argus has = a complete TCP state machine so that it can identify requests for = missing packets, retransmissions and out of order packets. But = argus is designed to recognize loss regardless of where it is along the = path. As a result, the algorithm is a little complex, mainly = because TCP is reliable and regardless of the loss rate you should = always see at least one copy of all the packets. Because argus is = a bi-directional flow monitor, argus can do things like look for = requests for retransmission as an indication of loss. It can infer = that observing multiple packets is an indication of loss (you = don=E2=80=99t retransmit unless there was loss), and it can needs to do = this in the event of stripping and asymmetric routing.<br = class=3D""></blockquote><br class=3D"">yeah we are defn interesting in = the asymmetic situation, for now we are validating using point of = observation where traffic appears symmetric.<br class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D""><br class=3D"">Because = loss can occur before and or after argus see=E2=80=99s the packet = stream, argus will use retransmissions and retransmission requests from = the far side as an indication of loss. If the far side requests = more than once, we assume that the packet was lost more than once, or = that the retransmission request was lost. This is a possible = source for argus saying there is more loss than other tools.<br = class=3D""></blockquote><br class=3D"">Ah ok that is in part what I was = curious about, so if one were to look at the individual packets in a = flow and count the number of times any sequence number shows up more = than once, they should presumably come up with the same value as argus = loss calc?<br class=3D""><br class=3D""><br class=3D""><blockquote = type=3D"cite" class=3D"">Now with that as a starting point, where is = argus in relationship with the the other tools, and what methods are = they using to determine loss ???<br class=3D""></blockquote><br = class=3D"">Yeah this is a classic dilemma have having too many watches = and not knowing which is correct. What we have for a reference is = ultimately what iperf3 is reporting, and we are starting to look at what = the kernel can tell us as presumably the tcp implementation should have = a pretty authoritiative view of its own behavior, but I havent dug = into proc filesystem etc enough to know whats available. Im not = really sure yet what tsharks methodology is for determining = retransmission, I have been presuming it is similar to what you = described, Ill take a look at sequence numbers directly to = see whats up and report back.<br class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D""><br class=3D"">Carter<br = class=3D""> <span = class=3D"Apple-tab-span" style=3D"white-space:pre"> = </span> <span class=3D"Apple-tab-span" style=3D"white-space:pre"> = </span><br class=3D"">Carter Bullard =E2=80=A2 CTO<br = class=3D"">150 E 57th Street, Suite 12D<br class=3D"">New York, New York = 10022-2795<br class=3D"">Phone +1.212.588.9133 =E2=80=A2 Mobile = +1.917.497.9494<br class=3D""><br class=3D""><br class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D"">On Dec 7, 2018, at 12:28 = PM, Balas, Edward G <<a href=3D"mailto:[email protected]" = class=3D"">[email protected]</a>> wrote:<br class=3D""><br class=3D"">Hey = all,<br class=3D""><br class=3D"">Ive run into an issue Im struggling to = understand, and thus far googling has failed to right me. I am = trying to use Argus to track retransmissions / loss in flows and I am = getting values that are inconsistent with other tools including the = sending application. As I recurse into the various = rabbit holes contributing to this on our end, I was wondering if = someone could guide me on the following:<br class=3D""><br class=3D"">1. = within ra etc there is the ability to report loss and retrans. = When I look at the documentation loss seems to imply it contains = both retransmissions and dropped packets, if Im looking at a TCP flow, = is it correct to assume there will be no drops and thus loss is = synonymous with retransmission?<br class=3D""><br class=3D"">2. = I am able to get ra and racluster to report loss values for my = flows, however retrans is always 0, is there a special -M or other = options or argus option I need to use to see retrans? Im making = the possibly bad assumption that because I can see loss values the = tunings of argus are sufficient.<br class=3D""><br class=3D"">3. = The Loss numbers are always higher than what I am seeing with = other applications, is there a document or place in the code I should go = look at that describes how this is calculated?<br class=3D""><br = class=3D"">Motivating these questions is the following small test:<br = class=3D"">-------------------------------------------------------<br = class=3D"">I transfered a file to my test host while doing full snaplen = packet capture, and then compared argus with tshark reports of loss and = retransmission. <br class=3D""><br class=3D"">accuracy-test2]# = argus -JA -r raw.pcap -w raw.argus<br class=3D""><br = class=3D"">accuracy-test2]# racluster -n -r raw.argus -s = stime,dur,pkts,retrans,loss,appbytes,cause -- port 51170<br = class=3D""> StartTime = Dur TotPkts Retrans Loss TotAppByte = Cause <br class=3D""> 16:47:58.176366 15.206044 = 21513 0 24 = 20195392 Start<br class=3D""><br class=3D""><br = class=3D""><br class=3D""><br class=3D""><blockquote type=3D"cite" = class=3D""><blockquote type=3D"cite" class=3D"">Total packets between = tshark and argus agree:<br class=3D""></blockquote></blockquote><br = class=3D"">accuracy-test2]# tshark -r raw.pcap -nn -Y = 'tcp.port=3D=3D51170 ' | wc -l <br class=3D"">21513<br = class=3D""><br class=3D""><blockquote type=3D"cite" class=3D""><blockquote= type=3D"cite" class=3D"">Retransmissions / loss do not agree between = tshark and argus:<br class=3D""></blockquote></blockquote><br = class=3D"">accuracy-test2]# tshark -r raw.pcap -nn -Y = 'tcp.port=3D=3D51170 and tcp.analysis.retransmission ' | wc = -l <br class=3D"">17<br class=3D""><br class=3D"">17 vs 24<br = class=3D""><br class=3D""><br class=3D"">Was curious if folks had = insights they could share in these regards?<br class=3D""><br = class=3D"">Thanks,<br class=3D""><br class=3D"">Edward Balas<br = class=3D""><a href=3D"mailto:[email protected]" = class=3D"">[email protected]</a><br class=3D""></blockquote><br = class=3D""></blockquote><br class=3D""></blockquote><br = class=3D""></div></div></div></blockquote></div><br = class=3D""></div></div></body></html>= --Apple-Mail=_AC77B83A-62B2-44F2-91C8-273ADE02F359-- --Apple-Mail=_97DB8316-5B44-43BC-B731-56C39C6B8354 Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCCDH8w ggY5MIIFIaADAgECAgIvSjANBgkqhkiG9w0BAQUFADBwMQswCQYDVQQGEwJVUzEYMBYGA1UEChMP VS5TLiBHb3Zlcm5tZW50MQwwCgYDVQQLEwNFQ0ExIjAgBgNVBAsTGUNlcnRpZmljYXRpb24gQXV0 aG9yaXRpZXMxFTATBgNVBAMTDE9SQyBFQ0EgU1cgNTAeFw0xNjA0MjExNDUyMjBaFw0xOTA0MjAx NDUyMjBaMIGGMQswCQYDVQQGEwJVUzEYMBYGA1UEChMPVS5TLiBHb3Zlcm5tZW50MQwwCgYDVQQL EwNFQ0ExDDAKBgNVBAsTA09SQzEUMBIGA1UECxMLUW9TaWVudCBMTEMxKzApBgNVBAMTIkJ1bGxh cmQuV2lsbGlhbS5DLk9SQzEwMDAwMzcxMDEuSUQwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK AoIBAQDR+sBZYyKk0htuIgoLkSkV7lUnfo/inem6rqWzWZerA26V5lh6j1LQIxLA00NfiHVJ2nkv p9/2CcRljNV81LignnEi0Pp7QXYAgd+wYfHqy0Q2ZuFHnbEmqd8djJxGKGdYvD8krVeVJeouS5iu 2YcC4Me7jO6za8TFmwcTf/lr9oMl+xbXIVtk8YPsyLTZBmUWvdb9/wfipTTX6Om3NzTNCvSl6wdz ct56alGAi/dk+n3fUUo9SNethjr7LFWM/VTePlyLDkavdXINFs/H8uyWr+SjgwnfLnKv+zfxbw4k by1Ah/D18l36IdLEHPSIEDCtQGUJXF0qwWKEd2ZF/wLDAgMBAAGjggLEMIICwDAfBgNVHSMEGDAW gBS8D3cLjaOzhUPCNpNmrAKpd8M9UjAdBgNVHQ4EFgQUNnz7myAzDSiIqtxRyqe4NoLE4hcwggEj BggrBgEFBQcBAQSCARUwggERMB4GCCsGAQUFBzABhhJodHRwOi8vZXZhLm9yYy5jb20wMgYIKwYB BQUHMAKGJmh0dHA6Ly9lY2Eub3JjLmNvbS9jYUNlcnRzL0VDQS1TVzUucDdjMIG6BggrBgEFBQcw AoaBrWxkYXA6Ly9lY2EtZHMub3JjLmNvbS9jbiUzZE9SQyUyMEVDQSUyMFNXJTIwNSUyY291JTNk Q2VydGlmaWNhdGlvbiUyMEF1dGhvcml0aWVzJTJjb3UlM2RFQ0ElMmNvJTNkVS5TLiUyMEdvdmVy bm1lbnQlMmNjJTNkVVM/Y0FDZXJ0aWZpY2F0ZTtiaW5hcnksY3Jvc3NDZXJ0aWZpY2F0ZVBhaXI7 YmluYXJ5MA4GA1UdDwEB/wQEAwIGwDAdBgNVHREEFjAUgRJjYXJ0ZXJAcW9zaWVudC5jb20wFwYD VR0gBBAwDjAMBgpghkgBZQMCAQwBMIHxBgNVHR8EgekwgeYwK6ApoCeGJWh0dHA6Ly9lY2Eub3Jj LmNvbS9DUkxzL09SQ0VDQVNXNS5jcmwwgbaggbOggbCGga1sZGFwOi8vZWNhLWRzLm9yYy5jb206 Mzg5L2NuJTNET1JDJTIwRUNBJTIwU1clMjA1JTJDJTIwb3UlM0RDZXJ0aWZpY2F0aW9uJTIwQXV0 aG9yaXRpZXMlMkMlMjBvdSUzREVDQSUyQyUyMG8lM0RVLlMuJTIwR292ZXJubWVudCUyQyUyMGMl M0RVUz9jZXJ0aWZpY2F0ZVJldm9jYXRpb25MaXN0O2JpbmFyeTAbBgNVHQkEFDASMBAGCCsGAQUF BwkEMQQTAlVTMA0GCSqGSIb3DQEBBQUAA4IBAQAJDKDhb7K8Gr2zCF4ItwIiA/mxi1H7CscUq76N EtZHoyoUnbfi15fgQnX/ZFPEsAVxx/Isupmi8RChTj67FxqfWZ8jsdb38PifUtI2Kqf31PSBU0jz 5vCobxb/K3y9dRGwLWx5ufOq6zLD3thYcFmUyJto4tEKWJds4/xjurAulr8EESeZvm5KmzzxohfL CbjJTlIGrVpUmeTYT3AwVdnNrNCdsVSCjyZWvAVId6LNheWpkPBVaHUO7VleWGqmiW9E79kQJ2V0 dn8Fg1kB85jv2lzrOpB43LiIK354h69mhAQZiR4QahY1gQFVnKb+Yspd53J8lg7yBWuNene9xlz1 MIIGPjCCBSagAwIBAgICL0swDQYJKoZIhvcNAQEFBQAwcDELMAkGA1UEBhMCVVMxGDAWBgNVBAoT D1UuUy4gR292ZXJubWVudDEMMAoGA1UECxMDRUNBMSIwIAYDVQQLExlDZXJ0aWZpY2F0aW9uIEF1 dGhvcml0aWVzMRUwEwYDVQQDEwxPUkMgRUNBIFNXIDUwHhcNMTYwNDIxMTQ1MzQwWhcNMTkwNDIw MTQ1MzQwWjCBizELMAkGA1UEBhMCVVMxGDAWBgNVBAoTD1UuUy4gR292ZXJubWVudDEMMAoGA1UE CxMDRUNBMQwwCgYDVQQLEwNPUkMxFDASBgNVBAsTC1FvU2llbnQgTExDMTAwLgYDVQQDEydCdWxs YXJkLldpbGxpYW0uQy5PUkMxMDAwMDM3MTAxLkVuY3J5cHQwggEiMA0GCSqGSIb3DQEBAQUAA4IB DwAwggEKAoIBAQDIiywwq6YQSFJnhXYGQ+bwZUmjYQaNXWTNrEfnhw+L3aFBc5z4fO8wbSCsLigm JpVNDF/+ScNcB54IpVisD505AuXmY4AlF8/5n17+RWTWLXCXJ87YwKSPyD2orEGEoSF1oK3t3twA J4R+6zOMnG3LvFmXxeF2XuHRx9hpiE3XDo4PmdErNFFOfgtDhTHKXattg7rLEfxSFVYoheBaC2O1 C4ZWsel/T38dH/LFIMuQQdKyJODLCD+pBhDBTr4Ds7w98VOyq0HTmJ9vRZJffwzQnp25bGiSJXDW OU+mz9DRg7TrqlufWIjUbNd6ZhMDoup0yQ3Hr+IEGmi/qG8uz4GdAgMBAAGjggLEMIICwDAfBgNV HSMEGDAWgBS8D3cLjaOzhUPCNpNmrAKpd8M9UjAdBgNVHQ4EFgQUb/317FYu4hc3AFJyq7wwfWLC bFcwggEjBggrBgEFBQcBAQSCARUwggERMB4GCCsGAQUFBzABhhJodHRwOi8vZXZhLm9yYy5jb20w MgYIKwYBBQUHMAKGJmh0dHA6Ly9lY2Eub3JjLmNvbS9jYUNlcnRzL0VDQS1TVzUucDdjMIG6Bggr BgEFBQcwAoaBrWxkYXA6Ly9lY2EtZHMub3JjLmNvbS9jbiUzZE9SQyUyMEVDQSUyMFNXJTIwNSUy Y291JTNkQ2VydGlmaWNhdGlvbiUyMEF1dGhvcml0aWVzJTJjb3UlM2RFQ0ElMmNvJTNkVS5TLiUy MEdvdmVybm1lbnQlMmNjJTNkVVM/Y0FDZXJ0aWZpY2F0ZTtiaW5hcnksY3Jvc3NDZXJ0aWZpY2F0 ZVBhaXI7YmluYXJ5MA4GA1UdDwEB/wQEAwIFIDAdBgNVHREEFjAUgRJjYXJ0ZXJAcW9zaWVudC5j b20wFwYDVR0gBBAwDjAMBgpghkgBZQMCAQwBMIHxBgNVHR8EgekwgeYwK6ApoCeGJWh0dHA6Ly9l Y2Eub3JjLmNvbS9DUkxzL09SQ0VDQVNXNS5jcmwwgbaggbOggbCGga1sZGFwOi8vZWNhLWRzLm9y Yy5jb206Mzg5L2NuJTNET1JDJTIwRUNBJTIwU1clMjA1JTJDJTIwb3UlM0RDZXJ0aWZpY2F0aW9u JTIwQXV0aG9yaXRpZXMlMkMlMjBvdSUzREVDQSUyQyUyMG8lM0RVLlMuJTIwR292ZXJubWVudCUy QyUyMGMlM0RVUz9jZXJ0aWZpY2F0ZVJldm9jYXRpb25MaXN0O2JpbmFyeTAbBgNVHQkEFDASMBAG CCsGAQUFBwkEMQQTAlVTMA0GCSqGSIb3DQEBBQUAA4IBAQAdNjcNYhcXvCHSn3F8XG9FdgIKJuc3 hjOsGmMTi/ChkSBi/511IdCqDwJEWnc8wH9mYJKYyiaDtCTyboYos9bIcgq2sHjIlAJK3DMfyBsi PJxg6LemVBojaPu4AAa8qi4BkLRkRPazsYQdGDioo8cXHRLWVvJIIVTPln3tPjUvD/o1OWYKIqUb p3HB7CtP3lIGOtklNL69Gc1om/fPPPxVeDQzUoYtQ5DDBMxTZz3L9TLNjD+7MAIFtgq5QobpLSNT Nbpo8VfwQZRU4yy0XwdtgESywzFyB0OVTO0upbRc8Vg+0VFw0W9lrx6Et7tF0y1fxEu9v5Mjicca KRmnxKFEMYIDIDCCAxwCAQEwdjBwMQswCQYDVQQGEwJVUzEYMBYGA1UEChMPVS5TLiBHb3Zlcm5t ZW50MQwwCgYDVQQLEwNFQ0ExIjAgBgNVBAsTGUNlcnRpZmljYXRpb24gQXV0aG9yaXRpZXMxFTAT BgNVBAMTDE9SQyBFQ0EgU1cgNQICL0owDQYJYIZIAWUDBAIBBQCgggF7MBgGCSqGSIb3DQEJAzEL BgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE4MTIxMzEzNTYzNFowLwYJKoZIhvcNAQkEMSIE IJ+nO9UzA0GQGMIVhB67KP9xxxX4Y2XKptT9zpD0I4xPMIGFBgkrBgEEAYI3EAQxeDB2MHAxCzAJ BgNVBAYTAlVTMRgwFgYDVQQKEw9VLlMuIEdvdmVybm1lbnQxDDAKBgNVBAsTA0VDQTEiMCAGA1UE CxMZQ2VydGlmaWNhdGlvbiBBdXRob3JpdGllczEVMBMGA1UEAxMMT1JDIEVDQSBTVyA1AgIvSzCB hwYLKoZIhvcNAQkQAgsxeKB2MHAxCzAJBgNVBAYTAlVTMRgwFgYDVQQKEw9VLlMuIEdvdmVybm1l bnQxDDAKBgNVBAsTA0VDQTEiMCAGA1UECxMZQ2VydGlmaWNhdGlvbiBBdXRob3JpdGllczEVMBMG A1UEAxMMT1JDIEVDQSBTVyA1AgIvSzANBgkqhkiG9w0BAQEFAASCAQBk9zMEOkDWATzFeisuSImJ exdaewbWlba19dhx/5PCJJyv7iCfLMjVio73wCXftZQYC05jPNrNzrCYHeR6KqeWkwrNpoGrYY2l 5jrMxJI8lR1R/acmyytALAkNa7YsXqV1Y5NzURcodeX2jjOrtKXmgrp30cZdXsyWlCFY9ZtXJEH7 v/hccooGVc5e6fdwMGDlBMhRCQerRe9NA/a5n2rwzT9hgx8eLq5duMxFtoP8BKk3kaeQE7FXIjZC CRDv/b1Pp7yvVPLNTi17vjhT8/goZdpc3Xlv2GBsauAtHx2vqpP8Eu2yy0KMIRlOZrlflSOBkOnu FRK1NqBj7MAaLHooAAAAAAAA --Apple-Mail=_97DB8316-5B44-43BC-B731-56C39C6B8354--