Proposing a new instruction for rrdraph: DSN

--===============2136155648==
Content-Type: multipart/alternative; boundary=001a11c136ae9a9da504fccb25cc

--001a11c136ae9a9da504fccb25cc
Content-Type: text/plain; charset=UTF-8

Hello List,

I migrated my growing collection of RRD files to MySQL several months ago
and it really reduced the IO load on the data collecting host as well as
improved the speed of drawing graphs. A new problem I faced was that the
length of rrdgraph commands increased due to longer DEF statements - with
sql/ notation on average the length of a single DEF statement is about 250
characters. I have scripts that generate rrdgraph commands to draw complex
graphs with tens or sometimes hundreds of DEFs, CDEFs, GPRINTs for
formatted legends etc. and many times it happened that rrdgraph failed to
generate a picture, because the command was too long (the length limit on
my system is 130k characters). It would help to avoid this issue and
shorten the command length if it was possible to define a DSN (Data Source
Name) and reuse it in DEFs, ie.:

rrdtool graph ... \
DSN:dsn1=sql//mysql/host=
mysql1.example.com/port=3306/dbname=mydb/username=dbuser/password=dbpassword
\
... \
DEF:defA=dsn1/rrdminstepsize=60/TABLE_A/timestamp/value/RRDKeyID='"keyA'":avg:AVERAGE
\
DEF:defB=dsn1/rrdminstepsize=60/TABLE_B/timestamp/value/RRDKeyID='"keyB'":avg:AVERAGE
\
DEF:defC=dsn1/rrdminstepsize=60/TABLE_C/timestamp/value/RRDKeyID='"keyC'":avg:AVERAGE
\
...

It's debatable what elements should be included in DSN and what should be
left for DEF, however I would suggest that DSN only contained host, port,
dbname, dbuser and password, because one database can have multiple tables
with different step sizes, so the in DEFs it would be possible to use
different tables in the same DB referring to the same DSN, but giving
different step sizes and table names.

Another feature that I think would be very useful is reading DSN from
files: store DSN definition in a text file and then point to this file in
the command, ie.:

rrdtool graph ... \
DSN:dsn1=file:///path/to/file \
...

This way DB credentials would be better protected as there would be no risk
that they appear in the commands and later in application logs or elsewhere
in the system where other users could see them.

Please share your thoughts and comments on this feature.

--
Regards,
Maciek Kolbusz

--001a11c136ae9a9da504fccb25cc
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div><div><div>Hello List,<br><br></div>I migrated my=
 growing collection of RRD files to MySQL several months ago and it really =
reduced the IO load on the data collecting host as well as improved the spe=
ed of drawing graphs. A new problem I faced was that the length of rrdgraph=
 commands increased due to longer DEF statements - with sql/ notation on av=
erage the length of a single DEF statement is about 250 characters. I have =
scripts that generate rrdgraph commands to draw complex graphs with tens or=
 sometimes hundreds of DEFs, CDEFs, GPRINTs for formatted legends etc. and =
many times it happened that rrdgraph failed to generate a picture, because =
the command was too long (the length limit on my system is 130k characters)=
. It would help to avoid this issue and shorten the command length if it wa=
s possible to define a DSN (Data Source Name) and reuse it in DEFs, ie.:<br=
>
<br></div>rrdtool graph ... \<br></div>DSN:dsn1=3Dsql//mysql/host=3D<a href=
=3D"http://mysql1.example.com/port=3D3306/dbname=3Dmydb/username=3Ddbuser/p=
assword=3Ddbpassword">mysql1.example.com/port=3D3306/dbname=3Dmydb/username=
=3Ddbuser/password=3Ddbpassword</a> \<br>
... \<br></div>DEF:defA=3Ddsn1/rrdminstepsize=3D60/TABLE_A/timestamp/value/=
RRDKeyID=3D&#39;&quot;keyA&#39;&quot;:avg:AVERAGE \<br>DEF:defB=3Ddsn1/rrdm=
instepsize=3D60/TABLE_B/timestamp/value/RRDKeyID=3D&#39;&quot;keyB&#39;&quo=
t;:avg:AVERAGE \<br>
DEF:defC=3Ddsn1/rrdminstepsize=3D60/TABLE_C/timestamp/value/RRDKeyID=3D&#39=
;&quot;keyC&#39;&quot;:avg:AVERAGE \<br>...<br clear=3D"all"><div><div><div=
><div><div><div><br></div><div>It&#39;s debatable what elements should be i=
ncluded in DSN and what should be left for DEF, however I would suggest tha=
t DSN only contained host, port, dbname, dbuser and password, because one d=
atabase can have multiple tables with different step sizes, so the in DEFs =
it would be possible to use different tables in the same DB referring to th=
e same DSN, but giving different step sizes and table names.<br>
<br></div><div>Another feature that I think would be very useful is reading=
 DSN from files: store DSN definition in a text file and then point to this=
 file in the command, ie.:<br><br></div><div>rrdtool graph ... \<br></div>
<div>DSN:dsn1=3Dfile:///path/to/file \<br>...<br><br></div><div>This way DB=
 credentials would be better protected as there would be no risk that they =
appear in the commands and later in application logs or elsewhere in the sy=
stem where other users could see them.<br>
<br></div><div>Please share your thoughts and comments on this feature.<br>=
</div><div><br>--<br>Regards,<br>Maciek Kolbusz</div>
</div></div></div></div></div></div>

--001a11c136ae9a9da504fccb25cc--


--===============2136155648==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
rrd-developers mailing list
[email protected]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-developers

--===============2136155648==--