Re: ra not reading a 1GB radium.out file
Monah Baki <[email protected]> Sun, 7 Oct 2018 14:51:28 -0400
| Newsgroups | gmane.network.argus |
|---|---|
| Message-ID | <CALP3=x9x4r5P=ZETKSr8FWdf0jxw2znKdR0p0+Ny=NiF_2u6oA@mail.gmail.com> |
--000000000000b0dfec0577a7fe48 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Never mind, used the Linux logrotate to get what I wanted Thanks Carter Monah On Sun, Oct 7, 2018 at 10:26 AM Monah Baki <[email protected]> wrote: > Hi Carter, > > I used the following command "rastream -r radium.out -M time 30m -w > %Y/%m/%d/radium.%Y.%m.%d.%H.%M.%S" per the example, but the radium.out fi= le > is still at 1Gb plus, due to disk space in /var/log, I need to 0 radium.o= ut > and have it start all over again > > Thanks > Monah > > On Sun, Oct 7, 2018 at 10:02 AM Carter Bullard <[email protected]> wrote= : > >> Hey Monah, >> You want to use rastream.1. Have it connect to radium to write out the >> files, don=E2=80=99t have radium do that. Think of radium as an argus r= ecord >> distribution node. Have rastream.1 split on time, not on size, as that >> organizes the data in a more natural scope. But if that is not possible= , >> rastream.1 can easily split based on file size, and after the file is >> closed, it will run a script against the file. Checkout the man page fo= r >> rastream. >> >> Carter >> [image: QoSient] <http://qosient.com> >> Carter Bullard <[email protected]> =E2=80=A2 CTO >> 150 E 57th Street, Suite 12D >> New York, New York 10022-2795 >> Phone +1.212.588.9133 =E2=80=A2 Mobile +1.917.497.9494 >> >> >> >> On Oct 7, 2018, at 9:57 AM, Monah Baki <[email protected]> wrote: >> >> Thanks Carter I was able to break it up. Question though, is there a fil= e >> somewhere where I can configure to say after every 10M of radium.out siz= e >> tar and start collecting logs from the start. >> >> >> Thanks >> Monah >> >> On Sun, Oct 7, 2018 at 9:13 AM Carter Bullard <[email protected]> wrote= : >> >>> Hey Monah, >>> Look at your system log and your system configuration. The log should >>> have some explanation, such as system file size exceeded. The size of t= he >>> file is close to 0x3D000000. This isn=E2=80=99t particularly magical t= o me, but >>> seems like a curious binary number, so its magical for your machine. >>> 32-bit Linux =E2=80=A6 Windows??? >>> >>> If there are hard limits on file size, rasplit.1 is designed to handle >>> this. I would suggest using rasplit.1 to create smaller files, say 5mi= n >>> files, so you can avoid this type of problem. >>> >>> Carter >>> [image: QoSient] <http://qosient.com/> >>> Carter Bullard <[email protected]> =E2=80=A2 CTO >>> 150 E 57th Street, Suite 12D >>> New York, New York 10022-2795 >>> Phone +1.212.588.9133 =E2=80=A2 Mobile +1.917.497.9494 >>> >>> >>> >>> On Oct 5, 2018, at 1:42 PM, Monah Baki <[email protected]> wrote: >>> >>> -rw-------. 1 root root 1023339245 Oct 5 13:40 radium.out >>> >>> >>> ra -r radium.out drops me to a shell but if I were to move the file and >>> have it start all over again, I get output. >>> >>> >>> Thanks >>> Monah >>> >>> >>> >> --000000000000b0dfec0577a7fe48 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Never mind, used the Linux logrotate to get what I wanted<= div><br></div><div><br></div><div>Thanks Carter</div><div><br></div><div>Mo= nah</div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Sun, Oct = 7, 2018 at 10:26 AM Monah Baki <<a href=3D"mailto:[email protected]">m= [email protected]</a>> wrote:<br></div><blockquote class=3D"gmail_quote= " style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><= div dir=3D"ltr"><div dir=3D"ltr">Hi Carter,<div><br></div><div>I used the f= ollowing command "rastream -r radium.out -M time 30m -w %Y/%m/%d/radiu= m.%Y.%m.%d.%H.%M.%S" per the example, but the radium.out file is still= at 1Gb plus, due to disk space in /var/log, I need to 0 radium.out and hav= e it start all over again</div><div><br></div><div>Thanks</div><div>Monah</= div></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Sun, Oct= 7, 2018 at 10:02 AM Carter Bullard <<a href=3D"mailto:[email protected]= m" target=3D"_blank">[email protected]</a>> wrote:<br></div><blockquote= class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc soli= d;padding-left:1ex"><div style=3D"word-wrap:break-word;line-break:after-whi= te-space">Hey Monah,<div>You want to use rastream.1. =C2=A0 Have it connect= to radium to write out the files, don=E2=80=99t have radium do that.=C2=A0= Think of radium as an argus record distribution node.=C2=A0 Have rastream.= 1 split on time, not on size, as that organizes the data in a more natural = scope.=C2=A0 But if that is not possible, rastream.1 can easily split based= on file size, and after the file is closed, it will run a script against t= he file.=C2=A0 Checkout the man page for rastream.</div><div><br><div>Carte= r<br><div> <table id=3D"m_572386257044315786m_-2124038879867933813sig" width=3D"360" c= ellspacing=3D"0" cellpadding=3D"0" style=3D"width:360px;margin:0;padding:0"= > <tbody><tr> <td width=3D"142" style=3D"width:142px;margin:0;padding:0"> <a href=3D"http://qosient.com" title=3D"QoSient" style=3D"border:none= ;text-decoration:none" target=3D"_blank"> <img alt=3D"QoSient" style= =3D"border:none;height:48px;font-family:'Gill Sans',sans-serif;font= -weight:100;font-size:48px;line-height:48px;color:#000000"> </a> </td> <td width=3D"20" style=3D"width:16px;min-width:16px;max-width:16px;marg= in:0;padding:0">=C2=A0</td> <td style=3D"margin:0;padding:0;border-collapse:collapse"> <table id=3D"m_572386257044315786m_-2124038879867933813sig2" cellspac= ing=3D"0" cellpadding=3D"0"> <tbody><tr> <td style=3D"font-family:'Gill Sans Light',sans-serif;whi= te-space:nowrap"> <a href=3D"mailto:[email protected]" style=3D"border:none;text= -decoration:none" target=3D"_blank"> <span style=3D"font-size:16px;line-height:16px;color:#0000b0= ">Carter Bullard</span> </a> <span style=3D"font-size:12px;line-height:14px;color:#000000">= =E2=80=A2</span> <span style=3D"font-size:12px;line-height:14px;color:#000000">C= TO</span> </td> </tr> <tr> <td style=3D"font-family:'Gill Sans Light',sans-serif;whi= te-space:nowrap"> <span style=3D"font-size:12px;line-height:14px;color:#000000">1= 50 E 57th Street, Suite 12D</span> </td> </tr> <tr> <td style=3D"font-family:'Gill Sans Light',sans-serif;whi= te-space:nowrap"> <span style=3D"font-size:12px;line-height:14px;color:#000000">N= ew York, New York 10022-2795</span> </td> </tr> <tr> <td style=3D"font-family:'Gill Sans Light',sans-serif;whi= te-space:nowrap"> <span style=3D"font-size:12px;line-height:12px;color:#000000">P= hone +1.212.588.9133 =E2=80=A2 Mobile +1.917.497.9494</span> </td> </tr> </tbody></table> </td> </tr><tr> </tr></tbody></table> <br> =C2=A0 </div> <div><br><blockquote type=3D"cite"><div>On Oct 7, 2018, at 9:57 AM, Monah B= aki <<a href=3D"mailto:[email protected]" target=3D"_blank">monahbaki@= gmail.com</a>> wrote:</div><br class=3D"m_572386257044315786m_-212403887= 9867933813Apple-interchange-newline"><div><div dir=3D"ltr">Thanks Carter I = was able to break it up. Question though, is there a file somewhere where I= can configure to say after every 10M of radium.out size tar and start coll= ecting logs from the start.<div><br></div><div><br></div><div>Thanks</div><= div>Monah</div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Sun= , Oct 7, 2018 at 9:13 AM Carter Bullard <<a href=3D"mailto:carter@qosien= t.com" target=3D"_blank">[email protected]</a>> wrote:<br></div><blockq= uote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc = solid;padding-left:1ex"><div style=3D"word-wrap:break-word;line-break:after= -white-space">Hey Monah,<div><div>Look at your system log and your system c= onfiguration.=C2=A0 The log should have some explanation, such as system fi= le size exceeded. The size of the file is close to 0x3D000000.=C2=A0 This i= sn=E2=80=99t particularly magical to me, but seems like a curious binary nu= mber, so its magical for your machine. =C2=A032-bit Linux =E2=80=A6 Windows= ???</div><div><br></div><div>If there are hard limits on file size, rasplit= .1 is designed to handle this.=C2=A0 I would suggest using rasplit.1 to cre= ate smaller files, say 5min files, so you can avoid this type of problem.</= div><div><br><div>Carter<br><div> <table id=3D"m_572386257044315786m_-2124038879867933813m_-36797640902532409= 20sig" width=3D"360" cellspacing=3D"0" cellpadding=3D"0" style=3D"width:360= px;margin:0;padding:0"> <tbody><tr> <td width=3D"142" style=3D"width:142px;margin:0;padding:0"> <a href=3D"http://qosient.com/" title=3D"QoSient" style=3D"border:non= e;text-decoration:none" target=3D"_blank"> <img alt=3D"QoSient" styl= e=3D"border:none;height:48px;font-family:"Gill Sans",sans-serif;f= ont-weight:100;font-size:48px;line-height:48px"> </a> </td> <td width=3D"20" style=3D"width:16px;min-width:16px;max-width:16px;marg= in:0;padding:0">=C2=A0</td> <td style=3D"margin:0;padding:0;border-collapse:collapse"> <table id=3D"m_572386257044315786m_-2124038879867933813m_-36797640902= 53240920sig2" cellspacing=3D"0" cellpadding=3D"0"> <tbody><tr> <td style=3D"font-family:'Gill Sans Light',sans-serif;whi= te-space:nowrap"> <a href=3D"mailto:[email protected]" style=3D"border:none;text= -decoration:none" target=3D"_blank"> <span style=3D"font-size:16px;line-height:16px;color:#0000b0= ">Carter Bullard</span> </a> <span style=3D"font-size:12px;line-height:14px">=E2=80=A2</span= > <span style=3D"font-size:12px;line-height:14px">CTO</span> </td> </tr> <tr> <td style=3D"font-family:'Gill Sans Light',sans-serif;whi= te-space:nowrap"> <span style=3D"font-size:12px;line-height:14px">150 E 57th Stre= et, Suite 12D</span> </td> </tr> <tr> <td style=3D"font-family:'Gill Sans Light',sans-serif;whi= te-space:nowrap"> <span style=3D"font-size:12px;line-height:14px">New York, New Y= ork 10022-2795</span> </td> </tr> <tr> <td style=3D"font-family:'Gill Sans Light',sans-serif;whi= te-space:nowrap"> <span style=3D"font-size:12px;line-height:12px">Phone +1.212.58= 8.9133 =E2=80=A2 Mobile +1.917.497.9494</span> </td> </tr> </tbody></table> </td> </tr><tr> </tr></tbody></table> <br> =C2=A0 </div> <div><br><blockquote type=3D"cite"><div>On Oct 5, 2018, at 1:42 PM, Monah B= aki <<a href=3D"mailto:[email protected]" target=3D"_blank">monahbaki@= gmail.com</a>> wrote:</div><br class=3D"m_572386257044315786m_-212403887= 9867933813m_-3679764090253240920Apple-interchange-newline"><div><div dir=3D= "ltr"><div dir=3D"ltr">-rw-------.=C2=A0 1 root=C2=A0 =C2=A0 root=C2=A0 =C2= =A0 1023339245 Oct=C2=A0 5 13:40 radium.out<br></div><div dir=3D"ltr"><br><= /div><div><br></div><div>ra -r radium.out drops me to a shell but if I were= to move the file and have it start all over again, I get output.</div><div= ><br></div><div><br></div><div>Thanks</div><div>Monah</div></div> </div></blockquote></div><br></div></div></div></div></blockquote></div> </div></blockquote></div><br></div></div></div></blockquote></div> </blockquote></div> --000000000000b0dfec0577a7fe48--