Re: Proposed schema for the backend database for Nessus

Eric Arnoth wrote:
> I've been working on some modifications to Javier's proposal for a database 
> schema, and I thought I'd share with the group what I've done so far.  

That's great. See my comments below.

> > I've posted to a tarball  
> 
(...)
> 
> The sql statements (*pgsql, *mysql) were created with a script I wrote called 
> Dia2SQLpy.  The script takes the Dia UML diagrams and produces either a mysql 
> or postgresql create statement.  

Cool. I'll take this into account. I've worked with Dia before, could 
you please send this script to the Dia upstream developers so they 
include it in the contrib/ section of it?

> 
> Here's the list of changes I made, and my reasoning:
> ================================================================================
> Deleted ExecutionID column from Vulnerability table.
> Deleted fk relationship Vulnerability_ExecutedPlugins_ExecutionID.
> Created PluginID column in Vulnerability table.
> Created fk relationship Vulnerability_NessusPlugins_PluginID.
> --------------------------------------------------------------------------------
> A plugin will address a given vulnerability.  ExecutedPlugins relates plugins 
> to sessions.  Vulnerabilities are related to plugins regardless of scans 
> performed.

The fact is, you've gotten this wrong IMHO.
Vulnerabilities are associated with hosts. That is, the vulnerability 
table are vulnerabilities that have been _found_ while testing a host. 
Not "theoric" vulnerabilities.

Why are ExecutedPlugins and Vulnerabilities separated then? Because you 
can execute a plugin against a host but don't detect any vulnerability 
associated with it.

The "AdditionalInformation" stuff is a placeholder for information that 
a Plugin might have retrieved from a host. Sample: banners from a telnet 
server, open shares from a Netbios host. That's the main reasoning of 
having an

"executedplugin -> vulnerability"

Maybe I should have named it "Detected_Vulnerability" and not just 
"Vulnerability". However, the point is: I want an easy way to determine 
the exact # of vulnerabilties in my infraestructure. I can then link 
this to a ticket-based management system that provides me with a view 
with which vulnerabilities are still "open" (i.e. have not been fixed by 
the admins) and issue a ticket whenever I find a vulnerability.


> 
> ================================================================================
> Deleted PluginCVE table, added CVE column to Vulnerability table.
> --------------------------------------------------------------------------------
> A CVE names a vulnerability, not a Plugin.  Plugins may reference 
> vulnerabilities through the CVE number, but CVE's are not published against 
> Nessus plugins.
> A CVE entry only accounts for a single vulnerability.  As such, each 
> vulnerability listed in the Vulnerability table should only need one entry.

Not correct.

> 
> ================================================================================
> Deleted BID, added BugtraqID column to Vulnerability table.
> --------------------------------------------------------------------------------
> Same reasoning as CVE

Ditto. There are three wrong assumptions here:

- a vulnerability is a "theoric" one. Which is not, it's one that has 
been detected
- there is always one CVE entry for each Bugtraq entry
- there is always one CVE/Bugtraq entry for each plugin.

As a matter of fact there are plugins which test more than a given 
vulnerability. NASL won't allow this to be placed in the reports just 
yet, but should do in the future.

> 
> ================================================================================
> Deleted IsFalsePositive column from Vulnerability table.
> Added IsFalsePositive column to ExecutedPlugins table.
> --------------------------------------------------------------------------------
> In keeping with the model that differentiates a vulnerability from the 
> execution iof a plugin n a session, a false positive is an aspect of a scan, 
> not the vulnerability.
> 
	Since this assumption of vulnerability not associated with plugin 
execution is false (or at least not properly understood), I'm afraid 
this shouldn't be done either.


> ================================================================================
> Deleted DetectedService_NessusPlugin_PluginID fk relationship.
> Deleted Services_ExecutedPlugins_ServiceID fk relationship.
> Reversed Services_DetectedService_ServiceID fk relationship.
> Deleted ServiceID column from ExecutedPlugins table.
> Deleted PluginID column from DetecteService table.
> Created ServiceID column to NessusPlugin table.
> Created NessusPlugin_Services_ServiceID fk relationship.
> --------------------------------------------------------------------------------
> A plugin will only probe a vulnerability in a single service.  As such, 
> linking each NessusPlugin row will have an associated row in the Services 
> table, regardless of any scan run with the plugin (ExecutedPlugins).
> 
	That's not correct. There might be plugins which test more than one 
service. Simple example: a port scanner or the hydra brute force 
scanner. You cannot have a 1:1 relationship between plugins and 
services, that's just not correct.


> ================================================================================
> Created NessusVersion column in Session table.
> --------------------------------------------------------------------------------
> It might be useful to know what version of Nessus was used in a given scan
> 
	Yes. This one might be useful.


> ================================================================================
> Added FoundVulnerable column to ExecutedPlugins table.
> Deleted PluginID column from Vulnerability table.
> Added VulnerabilityID column to NessusPlugin table.
> Added Description column to Vulnerability table.
> Deleted Vulnerability_NessusPlugin_PluginID fk reference.
> Created NessusPlugin_Vulnerability_VulnerabilityID fk reference.
> --------------------------------------------------------------------------------
> Furthering the seperation of the Vulnerability, Plugin, and scan sessions 
> concepts.
> 
	Since vulnerabilities are not "generic" ones but specific ones. There 
is no way you can do these changes. Note that you could have this schema 
and add it whatever additional information you want to (for example 
CERT's/CVE's/Bugtraq's database) which would provide the detailed 
description information you are talking about.


> --------------------------------------------------------------------------------
> Other thoughts:
> 
> There may be an issue with the NessusPlugin table.  Due to the nature of all 
> the columns specified there, each one demands a large varchar() datatype.  
> However, having too large of a max row size can be problematic.  However, the 
> plugin is a discrete conceptual unit, so the columns all make sense to be in 
> that table.  

I'm not sure this is an issue. I have the whole plugin information in a 
table with this schema and I'm not seeing problems with this.

> 
> Should ExecutedPlugins have timestamps as it does now?  That is, does nessus 
> record when each plugin is stopped & started?  If not, then the StartTime & 
> EndTime should probably be removed from ExecutedPlugins.

Nessusd should (check Michael's code) introduce this information in the 
database:

- right after calling a plugin -> make an SQL insert
- right after finishing a plugin call -> make an SQL update

If the nessus server fails you can still see which plugins were not 
finished by checking which ones do not have an end time. This 
information is also useful to make estimates of how much a scan will 
take place, given the same condition. I.e. determine roughly how much on 
average a plugin against a given host will run.

However, it might make sense to add a column for the nessus server to 
add to the database when a given plugin was killed or did not end 
correctly (maybe a boolean?)


How about adding the .dia and .mysql, .pgsql files in the Nessus CVS 
source so we can keep track of what changes have been done to the schema?

Regards


Javi