Re: Proposed schema for the backend database for Nessus

Eric Arnoth wrote:
> On Tuesday 11 February 2003 05:04, Javier Fernandez-Sanguino wrote:
>>
>>The fact is, you've gotten this wrong IMHO.
> 
> Okay, your opinion is noted...care for a little debate?  :-)

Sure, let's go.

Let me first say that, reviewing my schema, I believe that the 
"Vulnerability" table is basicly misnamed. It should be 
"detectedvulnerability". The fact here is that I wanted to separate 
plugins that have been executed in a given run (which is quite high 
usually) and plugins return that a given system is vulnerable.

Also, note that I wouldn't put both information in the same place (i.e. 
table). There are two reasons for this:

1.- some plugins test more than one vulnerability. From a brief look:
http_w98_devname_dos.nasl (10930)
mediahouse_statistics_web_server.nasl (10748)
openssh_301.nasl (10802)
smb_nt_ms02-005.nasl (10861)
smb_nt_ms01-025.nasl (10668)

2.- Some other plugins give more than one security warning or note:
find_service2.nasl (11153)
find_service
http_methods.nasl (10498)
iis_nat.nasl (10759)
iplanet_dir_serv.nasl (10589)
mediahouse_statistics_web_server.nasl (10748)
ncl_items_2.nasl (10665)
novell_groupwise_webacc_information_disclosure.nasl (10789)
oracle_tnslsnr_version.nasl (10658)
php_nuke_admin_cp.nasl (10772)
phprocketaddin_traversal.nasl (10831)
showmount.nasl (10437)
web_traversal.nasl (10297)

This is even "worse" for plugins like find_services and hydra, they 
might output none, 1 or many different security warnings, notes or holes.

So the point here is the call to security_{hole|warning|hole} in the 
plugins. Is this information that I believe needs to be hold in a single 
table.

Noticed that this is not related to the "theoretic" vulnerabilities that 
  Eric pointed out, these are joined to the schema through external 
references (CVE or Bugtraq). These are vulnerabilities _specific_ to the 
  hosts tested.

>>Why are ExecutedPlugins and Vulnerabilities separated then? Because you
>>can execute a plugin against a host but don't detect any vulnerability
>>associated with it.
> 
> That's why a boolean column value called "FoundVulnerable" or some such thing 
> should be in the ExecutedPlugins table...because it's an attribute (the 
> result) of the action of running the plugin (which is what the 
> ExecutedPlugins table represents).
> 

The result is not only this attribute, it's also the information 
retrieved from it (take a look at security_notes or warnings). This 
information has to be stored in the database and is host-specific.


>>The "AdditionalInformation" stuff is a placeholder for information that
>>a Plugin might have retrieved from a host. Sample: banners from a telnet
>>server, open shares from a Netbios host. 
> 
> If you want to store data gathered during a plugin, just make it a column of 
> the ExecutedPlugins table.  It's a much more logical place for it, since that 
> table is the record of a moment in time - each row representing some given 
> state of the system we're scanning.  
> 
	Can't do, some plugins add more than one. The point is, all are need to 
address separately independently of which plugins executed them. I want 
to "fix" real vulnerabilities, and that's why they are separated.

> 
>>That's the main reasoning of having an "executedplugin -> vulnerability"
> 
> Unfortunately, this is putting the cart before the horse.  What we have here 
> is a hierarchical relationship between two entities and one state.  A 
> vulnerability is discovered.  Renaurd et al write a plugin against it.  You 
> and I run our Nessus scanners against our respective networks, executing the 
> plugin, where we get a postive or negative.  That's the hierarchical order of 
> these three things...to express it any other way is incorrect.

Following your analogy:

- A vulnerability is detected (CVE, Bugtraq... is updated)
- The plugin writers create a plugin, this plugin migh detect more than 
one of these vulnerabilities
- A plugin is run against a given host and returns security information. 
This is not just saying "you are vulnerable" but also saying "this is 
the information I found about you". And notice that he a given plugin 
can say "you are vulnerable to X" _and_ say "you are vulnerable to Y" or 
say "I found information on X" _and_ "I found information on Y"

Even if there is a single theoretic vulnerability there are multiple 
"detected" vulnerabilities. Take a given router host, if the 
snmp_communities plugin finds that I can access it through "cisco", 
"read" _and_ "write" these are 3 distinct issues in that router 
discovered by a single plugin.


>>Maybe I should have named it "Detected_Vulnerability" and not just
>>"Vulnerability". However, the point is: I want an easy way to determine
>>the exact # of vulnerabilties in my infraestructure.
> 
> To have a table that gets rows for every vulnerability on every host is 
> wasteful.  Let's make up an example (which you can find at 
> http://mywebpages.comcast.net/earnoth/example.tar! ;) where there are three 
> hosts who all have the same two plugins executed against them (let's just 
> arbitrarily pick "WS FTP overflows" (CAN-2001-1021) and "Canna Overflow" 
> (CVE-2000-0584).  Let's now pretend that all three are vulnerable to both.  

Incorrect. The "generic" information of the theoretic vulnerability is 
stored in the plugin database. The specific information to a host is 
detected in the table I called "vulnerability" (which should have been 
"detectedvulnerability").

> Now the Vulnerability table is going to be populated with no less than four 
> duplicate rows.  Now lets imagine this for a scan against 200 hosts where 
> there are common vulnerabilities between hosts (let's pick an average of 40 
> per host).  That scales out to about 7,960 (200*40 - 40) duplicate rows that 
> store redunant data.  This a terrible waste of space.  

Let's take a look at the real space of those rows:

- Vulnerability_ID and ExecutionID: an integer
- IsFalsepositive: a bit
- AdditionalInformation: a varchar(255), in your example is empty. There 
is no additional information, the proper information is in the 
Nessusplugin table.

So, against having all this in one table (as you proposed, in 
ExecutedPlugins) you only have an increase of one integer per 
vulnerability. That's 8kbytes. (being generous)

> 
>>I can then link
>>this to a ticket-based management system that provides me with a view
>>with which vulnerabilities are still "open" (i.e. have not been fixed by
>>the admins) and issue a ticket whenever I find a vulnerability.
> 
> I can understand this viewpoint entirely.  I work in a massive corporation, 
> and this kind of thing is the bane of our existance.  However, I think the 
> examples I've posted here show that the revisions I posted can achieve the 
> same thing.  It's simply a matter of how one builds one's additional table 
> structure around the foundation, however that foundation is built.  Moreover, 
> if the foundation is built around how the data being modeled relates to one 
> another, additional structure will acually work better, in the long-run.  


Please rethink my proposal, you will find it more convenient. Note that 
the ExecutedPlugins space grows larger than the (Detected)Vulnerability 
table space. That's because _every_time_ you execute a plugin regardless 
if it's successful or not, it's stored there. Whileas the 
(Detected)Vulnerability only increases when you find a vulnerability. 
There are ~1200 plugins. Hopefully, you will not get 1200 security 
notes/warnings/holes for each host (if you do, you are in trouble).


>>>=========================================================================
>>>======= Deleted PluginCVE table, added CVE column to Vulnerability table.
>>>-------------------------------------------------------------------------
>>>------- A CVE names a vulnerability, not a Plugin.  Plugins may reference
>>>vulnerabilities through the CVE number, but CVE's are not published
>>>against Nessus plugins.
>>>A CVE entry only accounts for a single vulnerability.  As such, each
>>>vulnerability listed in the Vulnerability table should only need one
>>>entry.>>
>>Not correct.
> 
> Really?  The CVE naming process documentation at 

	Yes, really. The plugins might test more than one vulnerability. That's 
why this table is necessary.


> I have yet to see a CVE that refers to more than one vuln...could you please 
> give an example where a single CVE references more than one discrete 
> vulnerability?   Even if that is the case, then simply breaking out the CVE 
> into its own table as it was originally done is a simple solution.

	I'm not talking about wether or not CVE refers to more than one 
vulnerability. I'm saying that _Plugins_ test more than one.


> Again, a theoretical vulnerability is one which has no *known* exploit against 
> it.  I understand the want to link a host with it's vulnerabilities (that's 
> the whole point of Nessus, of course!) but that doesn't change the fact that 
> a vulnerability is a discrete concept, or entity, and should be properly 
> represented in the database design.

However theoretical "vulnerabilities" is not what Nessus is about. 
That's what you would have if you loaded CVE/Bugtraq/CERT alongside your 
Nessus schema, not something that Nessus should provide.


>>As a matter of fact there are plugins which test more than a given
>>vulnerability. NASL won't allow this to be placed in the reports just
>>yet, but should do in the future.
> 
> Here again, could you please provide an example?  I admit I could be 
> completely wrong here (as with the CVE matter) but I have yet to find an 
> example.  I did check the plugin list on the web, but was unable to find 
> anything.


	See above, multiple examples. The most notorious are: find_service and 
hydra. There can be more than one security note for a given host. In 
find_service's cases it's not really a vulnerability, it's more like 
information exposure. In hydra's case there is not really  an specific 
vulnerability tied to a given product/service it's a generic one 
(default passwords). You want to store each and every one of these in 
separate rows.


> 
> It is good to note that the Executed->Plugin->Vuln structure parallels the 
> DetectedService->Services->Protocol  layout in the original design.  The two 
> sets of tables show very similar hierarchical relationships.  (A host has a 
> service listening on a given port over a given protocol, and a host has a 
> plugin result from the an execution of a plugin which was written to address 
> a vulnerability).

	The relationship I'm trying to mimic is:

	Host (X) -> Plugin (Y) -> DetectedVulnerability (Z)

i.e.: we have found host X vulnerable using Y and this is the additional 
information : Z


	Have I made my point? Are you convinced now? :-)

	Javi