Re: writing to disk process

On Jul 19, 2006, at 11:36 AM, Torres, Javier wrote:

> Thanks for your answer Guy,
> What I am worried about is dropping data from the time I get the  
> packet
> in tshark to the time it actually writes to disk.

If a packet is read by TShark, *that* packet will not be dropped by  
TShark (unless you're using the -R flag and explicitly telling it to  
drop the packet).  It *will* be written to disk (assuming the file  
system doesn't fill up, of course).

Packets are dropped by the OS's packet capture mechanism *before*  
they're handed to the application - whether it's Wireshark or Tshark  
or dumpcap or tcpdump/WinDump or... - if the application isn't  
consuming them fast enough, so, between the time a packet is read by  
Tshark and the time it's written to disk, *later* packets could be  
dropped.

> Because of this I am
> trying to find a way to test and see if I received all the packets
> Tshark sees on the capture.  I had thought this app was looking at the
> interface so once it processed the information from the interface it
> would at that point write the data to disk.

The app (whether it's tcpdump/WinDump or Wireshark or Tshark or  
dumpcap or...) is calling libpcap/WinPcap, which supplies it with  
packets.  The app then processes the packet, where "processes"  could  
mean that it dissects the packet and prints the dissection to the  
standard output, or that it writes the packet to a file, or both of  
those.

> Since you are saying it is writing to disk at the same time it is
> looking at it,

I'm not sure how I'm saying that, because I'm not sure what "looking  
at it" means.  The application itself does *not* directly act on the  
interface.  The OS's driver for the interface is what fetches packets  
from the interface; it then hands them to parts of the OS kernel's  
networking networking code, which then supplies them to libpcap/ 
WinPcap, which then supplies them to the app.

> I was hoping to make sure the packets it is writing don't get  
> dropped in
> the time it takes them to write to disk since it is sensitive
> information we are gathering.

A packet that is being written to disk won't be dropped in the time  
it takes to write that packet to disk.

Packets that arrive *after* that packet could be dropped by the OS  
code in the time it takes to write the earlier packet to disk, if the  
app doesn't read those packets from the OS soon enough (the buffer in  
which the OS stores those packets has a finite size, and if it  
overflows, packets will be dropped, if packets aren't removed from  
the buffer soon enough by the app, the buffer will fill up, and if  
another packet arrives, it will overflow); those dropped packets will  
not be seen by the app.

As Ulf says, dumpcap does less processing on the packet, so it will  
be able to read packets faster, reducing the chances that they'll be  
dropped.