Re: limit mysql storage
Simone Mainardi <[email protected]> Tue, 12 Mar 2019 09:47:56 +0100
| Newsgroups | gmane.linux.ntop.general |
|---|---|
| Message-ID | <[email protected]> |
--===============1895385467642986449== Content-Type: multipart/alternative; boundary="Apple-Mail=_77B01470-5D58-43D6-9D86-B5829B587023" --Apple-Mail=_77B01470-5D58-43D6-9D86-B5829B587023 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Dan, > On 11 Mar 2019, at 19:48, Dan Craciun <[email protected]> wrote: >=20 > Thank you Simone. >=20 > I figured out how to get the space back.=20 >=20 > The question is why doesn't ntopng delete the old data?=20 > The "Duration in days of data retention" doesn't do anything? It does the delete - you won't be able to find records older than the = retention if you query the db - but deleting old records doesn't = automatically translate into mysql recycling unused space. Anyway, we have seen several shortcomings of MySQL when it comes to = store flows and decided to build a special-purpose database nIndex. I = would encourage you to try it out: = https://www.ntop.org/ntopng/say-hello-to-nindex-personal-big-data-system-f= or-network-flows/ Simone >=20 > Best regards, > Dan >=20 > On 11-Mar-19 16:35, Simone Mainardi wrote: >> "You can use OPTIMIZE TABLE to reclaim the unused space and to = defragment the data file. " >>=20 >> https://dev.mysql.com/doc/refman/8.0/en/optimize-table.html = <https://dev.mysql.com/doc/refman/8.0/en/optimize-table.html> >>=20 >> Simone >>=20 >>> On 8 Mar 2019, at 20:50, Dan Craciun <[email protected] = <mailto:[email protected]>> wrote: >>>=20 >>> Hi, >>>=20 >>> I've setup ntopng to dump flow data in a MySQL db, with 30 days = retention. >>> It's been over 45 days and the data is still there. >>>=20 >>> Furthermore, even if I manually delete the old data, the space = occupied >>> is not freed. It grows at a rate of 3GB/day. >>>=20 >>> What is the correct procedure of reusing free space when using MySQL >>> storage? >>>=20 >>> Best regards, >>> Dan Craciun >>> _______________________________________________ >>> Ntop mailing list >>> [email protected] <mailto:[email protected]> >>> http://listgateway.unipi.it/mailman/listinfo/ntop = <http://listgateway.unipi.it/mailman/listinfo/ntop> >>=20 >>=20 >>=20 >> _______________________________________________ >> Ntop mailing list >> [email protected] <mailto:[email protected]> >> http://listgateway.unipi.it/mailman/listinfo/ntop = <http://listgateway.unipi.it/mailman/listinfo/ntop> > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop --Apple-Mail=_77B01470-5D58-43D6-9D86-B5829B587023 Content-Transfer-Encoding: 7bit Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Dan,<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 11 Mar 2019, at 19:48, Dan Craciun <<a href="mailto:[email protected]" class="">[email protected]</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""> <meta http-equiv="Content-Type" content="text/html; charset=windows-1252" class=""> <div text="#000000" bgcolor="#FFFFFF" class=""> <div class="moz-cite-prefix">Thank you Simone.<br class=""> <br class=""> I figured out how to get the space back. <br class=""> <br class=""> The question is why doesn't ntopng delete the old data? <br class=""> The "Duration in days of data retention" doesn't do anything?<br class=""></div></div></div></blockquote><div><br class=""></div><div>It does the delete - you won't be able to find records older than the retention if you query the db - but deleting old records doesn't automatically translate into mysql recycling unused space.</div><div><br class=""></div><div>Anyway, we have seen several shortcomings of MySQL when it comes to store flows and decided to build a special-purpose database nIndex. I would encourage you to try it out: <a href="https://www.ntop.org/ntopng/say-hello-to-nindex-personal-big-data-system-for-network-flows/" class="">https://www.ntop.org/ntopng/say-hello-to-nindex-personal-big-data-system-for-network-flows/</a></div><div><br class=""></div>Simone</div><div>< br class=""><blockquote type="cite" class=""><div class=""><div text="#000000" bgcolor="#FFFFFF" class=""><div class="moz-cite-prefix"> <br class=""> Best regards,<br class=""> Dan<br class=""> <br class=""> On 11-Mar-19 16:35, Simone Mainardi wrote:<br class=""> </div> <blockquote type="cite" cite="mid:[email protected]" class="">"You can use OPTIMIZE TABLE to reclaim the unused space and to defragment the data file. " <div class=""><br class=""> </div> <div class=""><a href="https://dev.mysql.com/doc/refman/8.0/en/optimize-table.html" class="" moz-do-not-send="true">https://dev.mysql.com/doc/refman/8.0/en/optimize-table.html</a></div> <div class=""><br class=""> </div> <div class="">Simone<br class=""> <div class=""><br class=""> <blockquote type="cite" class=""> <div class="">On 8 Mar 2019, at 20:50, Dan Craciun <<a href="mailto:[email protected]" class="" moz-do-not-send="true">[email protected]</a>> wrote:</div> <br class="Apple-interchange-newline"> <div class=""> <div class="">Hi,<br class=""> <br class=""> I've setup ntopng to dump flow data in a MySQL db, with 30 days retention.<br class=""> It's been over 45 days and the data is still there.<br class=""> <br class=""> Furthermore, even if I manually delete the old data, the space occupied<br class=""> is not freed. It grows at a rate of 3GB/day.<br class=""> <br class=""> What is the correct procedure of reusing free space when using MySQL<br class=""> storage?<br class=""> <br class=""> Best regards,<br class=""> Dan Craciun<br class=""> _______________________________________________<br class=""> Ntop mailing list<br class=""> <a href="mailto:[email protected]" class="" moz-do-not-send="true">[email protected]</a><br class=""> <a class="moz-txt-link-freetext" href="http://listgateway.unipi.it/mailman/listinfo/ntop">http://listgateway.unipi.it/mailman/listinfo/ntop</a><br class=""> </div> </div> </blockquote> </div> <br class=""> </div> <br class=""> <fieldset class="mimeAttachmentHeader"></fieldset> <pre class="moz-quote-pre" wrap="">_______________________________________________ Ntop mailing list <a class="moz-txt-link-abbreviated" href="mailto:[email protected]">[email protected]</a> <a class="moz-txt-link-freetext" href="http://listgateway.unipi.it/mailman/listinfo/ntop">http://listgateway.unipi.it/mailman/listinfo/ntop</a></pre> </blockquote> <br class=""> </div> _______________________________________________<br class="">Ntop mailing list<br class=""><a href="mailto:[email protected]" class="">[email protected]</a><br class="">http://listgateway.unipi.it/mailman/listinfo/ntop</div></blockquote></div><br class=""></body></html> --Apple-Mail=_77B01470-5D58-43D6-9D86-B5829B587023-- --===============1895385467642986449== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop --===============1895385467642986449==--