organizing large datasets that reference look up info from outside sources
mike tancsa <[email protected]> Thu, 19 Oct 2017 15:42:20 -0400
| Newsgroups | gmane.network.argus |
|---|---|
| Organization | Sentex Communications |
| Message-ID | <[email protected]> |
Hi folks, I have a lot of network flows that I need to answer questions about that are not easily done with a simple argus command and am looking for suggestions on how best to organize the data. I have 2 large classes of endpoints -- servers and support staff. The servers are further broken down by department. to complicate matters, the servers are all on dynamic IPs and some come and go a few times a day and will get different IP addrs as do the support staff. I get general questions like Did anyone access server x,y and z in the last 3 days via netops and when ? or Who has accessed all of the Accounting Department's servers in Toronto yesterday? I can easily tell by port (6502) if there was remote access to a server. But its the matching of "server" to sets of IPs by time as well as matching the user's IP address to the users x509 token name. Here are some of the challenges I have Did anyone access server x,y and z in the last 3 days ? I first need to figure out the IPs and their start and stop times for those IPs the servers were on so I can see that server 'x' was IP address 192.168.77.3 from Sep 1 3pm to 9pm and then IP address 10.2.3.4 from Sep 1 10pm to Sep 12 at 1pm and so on. I could have a few dozen IPs that the server would have in the time period I am interested in. This is from a mysql database of start and stop records. I can do a lot of manual steps to answer these questions, but manual steps take time and are prone to error. I havent had to do this very often in the past, but am seeing it more and more so I am looking for strategies to better prep the data ahead of time to make my life easier and answer these questions quicker. Any suggestions / pointers would be most welcome. I am thinking I want to look into using ralabel ? Perhaps as part of a daily script, * identify the IP(s) of the server for that day and their times. * Query out all their flows and label them with the server:department and drop them into new argus files based on the server:department Is there a better way to approach this ? ---Mike