Re: Feature: per-target port specification (with patch!)
Daniel Miller <[email protected]> Tue, 2 Apr 2019 13:30:36 -0500
| Newsgroups | gmane.comp.security.nmap.devel |
|---|---|
| Message-ID | <CABmvJnNBXurU-UhYhVB6VgWTStfnoaSeJ1F_9S3=XpX3ESBNiQ@mail.gmail.com> |
--===============7993033068166170367== Content-Type: multipart/alternative; boundary="000000000000ff473305859055e7" --000000000000ff473305859055e7 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Jan, Thanks for this contribution. We've had many requests for this type of feature in the past, but have elected not to include it for a variety of reasons. There is an open discussion on our issue tracker that lays out some of the challenges in correctly implementing such a feature: http://issues.nmap.org/1217 It looks like your patch has tried to handle some of these situations, for example the "Ports scanned" output for Grepable output (and maybe XML, but it didn't look complete at first glance). If we are to do an actual code review and include this new feature, we would have to look for a complete solution that can handle the following situations: * The "Not shown: X ports" output for Normal output. * Properly formed XML output, with changes to the DTD and a "xmloutputversion" number increase. * Combination of this feature with existing --top-ports/port-ratio and -p options * Combination of this feature with CIDR subnetting and IPv4 octet ranges * Use of this feature along with advanced features like -O --traceroute and -sV Have you done any measurement of scans before and after adding this feature to determine the actual impact on scan times and bandwidth? Do you have a bandwidth target for your scans that Nmap is exceeding right now, and by how much? What does a typical nmap command line look like, and what performance options have you already tried? I look forward to hearing more about this from you and our other devs and users. Dan On Tue, Apr 2, 2019 at 8:07 AM Jan Gocn=C3=ADk <[email protected]> wrote: > Hey, > > I would like to propose a feature enabling specifying ports for each > target separately. > > Rationale: > It often happens that we already have an nmap scan of 200 machines, and w= e > want to do a service scan on those same machines. Usually that forces us = to > scan the whole network for all the ports that appeared at least once. > That is a big waste of time and bandwidth. What we want to have is > essentially a rescan-like feature, that would rescan just ports that were > found to be open before. > > User experience: > Everywhere where you could specify a target (-iL file, command line) you > can supply a "target^ports". It works with all the nmap magic ranges, so > "192.168.1.1-255^22-60" works. The common ports (supplied with -p) are > scanned on all targets. > > Implementation details: > I tried to keep it so that if you don't use any "^" in the targets, the > code path should remain largely the same, so there should be no > regressions. However, I had to do some tuning in functions that expected > they can just get the number of probes by multiplying common ports by > targets. > There's a small issue, in that the results of the scan are not sorted > properly, as the target-specific ports get scanned last. > > Usage example: > =3D=3D=3Dpaste start=3D=3D=3D > $ nmap -v -Pn -n -p22 "165.227.141.119^80,443" "40.113.73.59^8080" > Starting Nmap 7.70SVN ( https://nmap.org ) at 2019-04-01 19:46 CEST > Initiating SYN Stealth Scan at 19:46 > Scanning 2 hosts [max 3 ports/host] > Discovered open port 22/tcp > Discovered open port 80/tcp > Discovered open port 443/tcp > Discovered open port 22/tcp > Completed SYN Stealth Scan at 19:46, 1.45s elapsed (1626388576 total port= s > max) > Nmap scan report for 165.227.141.119 > Host is up (0.0090s latency). > > PORT STATE SERVICE > 22/tcp open ssh > 80/tcp open http > 443/tcp open https > > Nmap scan report for 40.113.73.59 > Host is up (0.038s latency). > > PORT STATE SERVICE > 22/tcp open ssh > 8080/tcp filtered http-proxy > > Read data files from: /home/gocnik/nmap > Nmap done: 2 IP addresses (2 hosts up) scanned in 1.52 seconds > Raw packets sent: 6 (264B) | Rcvd: 4 (176B) > =3D=3D=3Dpaste end=3D=3D=3D > > If done the usual way: > $ nmap -v -Pn -n -p22,80,443,8080 165.227.141.119 40.113.73.59 > [...] > Raw packets sent: 10 (440B) | Rcvd: 6 (260B) > > > The patch is against svn trunk at this moment (revision 37608). > > > > Looking forward to all comments! > JaGoTu > > P.S.: Sorry if you recieve this e-mail twice, but the previous one > apparently got caught in a moderation queue or something, as it doesn't > show on seclists.org > _______________________________________________ > Sent through the dev mailing list > https://nmap.org/mailman/listinfo/dev > Archived at http://seclists.org/nmap-dev/ --000000000000ff473305859055e7 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div>Jan,</div><div><br></div><div>Thanks= for this contribution. We've had many requests for this type of featur= e in the past, but have elected not to include it for a variety of reasons.= There is an open discussion on our issue tracker that lays out some of the= challenges in correctly implementing such a feature: <a href=3D"http://iss= ues.nmap.org/1217">http://issues.nmap.org/1217</a></div><div><br></div><div= >It looks like your patch has tried to handle some of these situations, for= example the "Ports scanned" output for Grepable output (and mayb= e XML, but it didn't look complete at first glance). If we are to do an= actual code review and include this new feature, we would have to look for= a complete solution that can handle the following situations:</div><div><b= r></div><div>* The "Not shown: X ports" output for Normal output.= </div><div>* Properly formed XML output, with changes to the DTD and a &quo= t;xmloutputversion" number increase.</div><div>* Combination of this f= eature with existing --top-ports/port-ratio and -p options</div><div>* Comb= ination of this feature with CIDR subnetting and IPv4 octet ranges</div><di= v>* Use of this feature along with advanced features like -O --traceroute a= nd -sV</div><div><br></div><div>Have you done any measurement of scans befo= re and after adding this feature to determine the actual impact on scan tim= es and bandwidth? Do you have a bandwidth target for your scans that Nmap i= s exceeding right now, and by how much? What does a typical nmap command li= ne look like, and what performance options have you already tried?</div><di= v><br></div><div>I look forward to hearing more about this from you and our= other devs and users.</div><div><br></div><div>Dan<br></div></div></div><b= r><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Tue, = Apr 2, 2019 at 8:07 AM Jan Gocn=C3=ADk <<a href=3D"mailto:[email protected]= ">[email protected]</a>> wrote:<br></div><blockquote class=3D"gmail_quote" = style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);pa= dding-left:1ex"><span style=3D"font-size:10pt;font-family:sans-serif">Hey,<= /span> <br> <br><span style=3D"font-size:10pt;font-family:sans-serif">I would like to propose a feature enabling specifying ports for each target separately.</sp= an> <br> <br><span style=3D"font-size:10pt;font-family:sans-serif">Rationale:</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">It often happens that we already have an nmap scan of 200 machines, and we want to do a service scan on those same machines. Usually that forces us to scan the whole network for all the ports that appeared at least once.</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">That is a big waste of time and bandwidth. What we want to have is essentially a rescan-l= ike feature, that would rescan just ports that were found to be open before.</s= pan> <br> <br><span style=3D"font-size:10pt;font-family:sans-serif">User experience:<= /span> <br><span style=3D"font-size:10pt;font-family:sans-serif">Everywhere where you could specify a target (-iL file, command line) you can supply a "= target^ports". It works with all the nmap magic ranges, so "192.168.1.1-255^22-60&quo= t; works. The common ports (supplied with -p) are scanned on all targets.</spa= n> <br> <br><span style=3D"font-size:10pt;font-family:sans-serif">Implementation details:</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">I tried to keep it so that if you don't use any "^" in the targets, the code path should remain largely the same, so there should be no regressions. However, I had to do some tuning in functions that expected they can just get the number of probes by multiplying common ports by targets.</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">There's a sma= ll issue, in that the results of the scan are not sorted properly, as the target-specific ports get scanned last.</span> <br> <br><span style=3D"font-size:10pt;font-family:sans-serif">Usage example:</s= pan> <br><span style=3D"font-size:10pt;font-family:sans-serif">=3D=3D=3Dpaste st= art=3D=3D=3D</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">$ nmap -v -Pn -n -p22 "165.227.141.119^80,443" "40.113.73.59^8080"</s= pan> <br><span style=3D"font-size:10pt;font-family:sans-serif">Starting Nmap 7.70SVN ( </span><a href=3D"https://nmap.org/" target=3D"_blank"><span styl= e=3D"font-size:10pt;color:blue;font-family:sans-serif">https://nmap.org</sp= an></a><span style=3D"font-size:10pt;font-family:sans-serif"> ) at 2019-04-01 19:46 CEST</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">Initiating SYN Stealth Scan at 19:46</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">Scanning 2 hosts [max 3 ports/host]</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">Discovered open port 22/tcp</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">Discovered open port 80/tcp</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">Discovered open port 443/tcp</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">Discovered open port 22/tcp</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">Completed SYN Stealth Scan at 19:46, 1.45s elapsed (1626388576 total ports max)</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">Nmap scan report for 165.227.141.119</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">Host is up (0.009= 0s latency).</span> <br> <br><span style=3D"font-size:10pt;font-family:sans-serif">PORT =C2=A0 =C2= =A0STATE SERVICE</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">22/tcp =C2=A0open =C2=A0ssh</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">80/tcp =C2=A0open =C2=A0http</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">443/tcp open =C2= =A0https</span> <br> <br><span style=3D"font-size:10pt;font-family:sans-serif">Nmap scan report for 40.113.73.59</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">Host is up (0.038= s latency).</span> <br> <br><span style=3D"font-size:10pt;font-family:sans-serif">PORT =C2=A0 =C2= =A0 STATE =C2=A0 =C2=A0SERVICE</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">22/tcp =C2=A0 open =C2=A0 =C2=A0 ssh</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">8080/tcp filtered http-proxy</span> <br> <br><span style=3D"font-size:10pt;font-family:sans-serif">Read data files from: /home/gocnik/nmap</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">Nmap done: 2 IP addresses (2 hosts up) scanned in 1.52 seconds</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Raw packets sent: 6 (264B) | Rcvd: 4 (176B)</spa= n> <br><span style=3D"font-size:10pt;font-family:sans-serif">=3D=3D=3Dpaste en= d=3D=3D=3D</span> <br> <br><span style=3D"font-size:10pt;font-family:sans-serif">If done the usual way:</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">$ nmap -v -Pn -n -p22,80,443,8080 165.227.141.119 40.113.73.59</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">[...]</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">Raw packets sent: 10 (440B) | Rcvd: 6 (260B)</span> <br> <br> <br><span style=3D"font-size:10pt;font-family:sans-serif">The patch is agai= nst svn trunk at this moment (revision 37608).</span> <br> <br> <br> <br><span style=3D"font-size:10pt;font-family:sans-serif">Looking forward to all comments!</span> <br><span style=3D"font-size:10pt;font-family:sans-serif">JaGoTu</span> <br> <br><span style=3D"font-size:10pt;font-family:sans-serif">P.S.: Sorry if you recieve this e-mail twice, but the previous one apparently got caught in a moderation queue or something, as it doesn't show on <a href=3D"ht= tp://seclists.org" target=3D"_blank">seclists.org</a></span> <br>_______________________________________________<br> Sent through the dev mailing list<br> <a href=3D"https://nmap.org/mailman/listinfo/dev" rel=3D"noreferrer" target= =3D"_blank">https://nmap.org/mailman/listinfo/dev</a><br> Archived at <a href=3D"http://seclists.org/nmap-dev/" rel=3D"noreferrer" ta= rget=3D"_blank">http://seclists.org/nmap-dev/</a></blockquote></div> --000000000000ff473305859055e7-- --===============7993033068166170367== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Sent through the dev mailing list https://nmap.org/mailman/listinfo/dev Archived at http://seclists.org/nmap-dev/ --===============7993033068166170367==--