Re: Nessus scripts and Moore's Law

On Fri, 12 Nov 2004, Michel Arboi wrote:

> I guess that we could create sub-directories. Somebody suggested one
> directory for each family. Why not?
> We just have to change the install script.

That wouldn't help with CVSweb. :)

> Another trick: we could parse all the .nasl, and keep all the syntax
> tree in memory. But this might be too expensive (in MB).

The memory will stay shared among all nessusd processes unless you write
to it. You might even dump the result into a big file and map the file
readonly to make sure it will be shared (and this approach makes it
possible to choose between permanent mmap() plus extra fork() overhead and
indiviadual mmap()'s in children plus the overhead of minor page faults).

> You will get information about other services, so you might miss the
> untested service.

Yes but it is much easier to catch that all the reports for a particular 
service are missing than that some of the reports are missing.

> The service detection system tries to warn you anyway:
> e.g. find_service will say "an unknown service is running on this
> port, it is usually reserved for HTTP" and then find_service2 ou
> doublecheck_std_services will detect a web server. You will know that
> you have a problem, that you should increase the timeout or fix the
> network.

Fine. It works for services running on standard ports only but it 
covers the vast majority of cases. On the other hand, I find it quite 
confusing to get both that warning and some real reports simultaneously,
and I am tempted to ignore the warning as irrelevant noise. Joe Average 
Luser's temptation to ignore the warning would probably be much higher.
Perhaps such warnings should be promoted to "Nessus Alerts"?

But what's the point of repeated attempts to grab the banner in every
plugin that needs it? It might slow the test down quite noticeably (esp.  
if the problem does not disappear spontaneously), and I'll have to rerun
it anyway if the problem was caused by a transient/fixable communication
glitch.

I still think it makes more sense to try harder to get the banner at the 
same beginning of the test rather than to retry in every individual 
plugin.

Well, yes, there is one reason not to make pure kb-checking plugins:
the ability to run plugins in a standalone mode (esp. when the standard 
nasl is still unable to load and provide kb data).

> > MS Windows kill TCP connection with RST when the task dies (GPF etc.).
> 
> Excellent idea! We just have to enhance the API to get the last error
> code - I guess that nessusd will get ECONNRESET, no?

Probably. You can check this when you connect to a task running on Windows
and kill the task from the Task List.


On Fri, 12 Nov 2004, Michel Arboi wrote:

> We could imagine that every Nessus test tries very hard to get an
> answer from a previously identified service. This way, if there is a
> network problem, we will not miss anything.

It might be a good idea to write tests in such a way that a warning (or
alert...see above) is generated whenever the service fails to respond in
either of the expected (vulnerable or not vulnerable) ways. Tests 
expecting no output may do an additional http_is_dead()-like check; 
this is somewhat controversial because it adds more work but I think most 
of them are destructive/DoS test ergo they should check the service's 
liveness anyway.


On Fri, 12 Nov 2004, Renaud Deraison wrote:

> > *If* the VM really speeds things up. Does anybody know how I could
> 
> Not "speed things up". Lower the CPU usage.

Lowering CPU usage is pointless unless you want 1. to run other CPU 
intensive tasks on the same machine, or 2. to save some joules of 
electric energy, or 3. want to "speed things up" and CPU is the 
bottleneck. :)

> The problem of the original HTTP caching mecanism is that it would grow
> the KB, which would in turn make forking way too slow on Linux/FreeBSD
> systems (where the fork() time is proportional to the process size).

How much memory did it consume? Yes, you pay for every page whose entry
must be copied (and prepared for COW if it is writable) but there are
already hundreds of pages to be duplicated for the program itself, 
dynamic libraries etc.


On Sat, 13 Nov 2004, Renaud Deraison wrote:

> Just a side note on this : what takes time is the loading of the .nasl
> file from disk to memory. The way the interpretor is done today, a
> compiled .nasl file is actually bigger than a un-compiled one, so the
> CPU time gained by using a binary file is lost in system calls to read
> the file from disk.

Do you need more than 3 syscalls to read the whole file: open(), read(),
and close()?


On Sat, 13 Nov 2004, Michel Arboi wrote:

> If anybody finds a way to benchmark such a thing without writing a
> new interpretor, that would be great.

Put one "start timer" call at the beginning of the interpreter, one "stop
timer" plus "report timer value" at the end, and a pair of "stop" and
"start" around all I/O procedures. It will reveal how much time is spent
interpreting the code.


BTW: One obvious optimization: rewrite recv_line() to read input in bigger
chunks rather than sucking it by single characters. Esp. in non-SSL mode
where one character == two syscalls, and one of them is select() with a 
rather high overhead. This leads to another obvious optimization: poll() 
(if supported by the OS) is much more efficient than select() when only a 
small number of fd's (such as a single fd) is watched.

Yet another possible optimization: the hashing function in kb.c is
suspicious (repeated use of << shifts startings characters into oblivion
when it gets a long key) and might lead to uneven distribution of keys.


--Pavel Kankovsky aka Peak  [ Boycott Microsoft--http://www.vcnet.com/bms ]
"Resistance is futile. Open your source code and prepare for assimilation."

_______________________________________________
Nessus-devel mailing list
[email protected]
http://mail.nessus.org/mailman/listinfo/nessus-devel