Re: PostGreSQL Replication and question on maintenance
Pavel Stehule <[email protected]> Wed, 20 Nov 2019 13:15:23 +0100
| Newsgroups | gmane.comp.db.postgresql.general,gmane.comp.db.postgresql.sql |
|---|---|
| Message-ID | <CAFj8pRABrsJeWDVJkOLvwTzxA6urdOo9Hteva7JUZf2TsV7Spg@mail.gmail.com> |
--000000000000cb898a0597c624e1 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi st 20. 11. 2019 v 13:12 odes=C3=ADlatel Soto Cuevas Manuel Alejandro < [email protected]> napsal: > Taking advantage of the instance, what software do you recommend to do > reverse engineering of postgresql? > please, open new thread. Your query is not related to subject. https://wiki.postgresql.org/wiki/Design_Tools Regards Pavel > Thank you > > Manuel > > El 16-11-2019, a la(s) 10:36, github kran <[email protected]> escribi= =C3=B3: > > =EF=BB=BF > > *ANEXO ENTEL **Este es un correo externo: **Verifique remitente, No haga > clic en v=C3=ADnculos. **Si recibes un correo malicioso av=C3=ADsanos a *= *[email protected] > <[email protected]>** . * > Any reply on this please ?. > > On Fri, Nov 15, 2019 at 9:10 AM github kran <[email protected]> wrote: > >> >> >> On Thu, Nov 14, 2019 at 11:42 PM Pavel Stehule <[email protected]> >> wrote: >> >>> these numbers looks crazy high - how much memory has your server - >>> more than 1TB? >>> >> >> The cluster got 244 GB of RAM and storage capacity it has is 64 TB. >> >>> >>> >>> p=C3=A1 15. 11. 2019 v 6:26 odes=C3=ADlatel github kran <githubkran@gma= il.com> >>> napsal: >>> >>>> >>>> Hello postGreSQL Community , >>>>> >>>>> >>>>> >>>>> Hope everyone is doing great !!. >>>>> >>>>> >>>>> *Background* >>>>> >>>>> We use PostgreSQL Version 10.6 version and heavily use PostgreSQL for >>>>> our day to day activities to write and read data. We have 2 clusters >>>>> running PostgreSQL engine , one cluster >>>>> >>>>> keeps data up to 60 days and another cluster retains data beyond 1 >>>>> year. The data is partitioned close to a week( ~evry 5 days a partiti= on) >>>>> and we have around 5 partitions per month per each table and we have = 2 >>>>> tables primarily so that will be 10 tables a week. So in the cluster-= 1 we >>>>> have around 20 partitions and in cluster-2 we have around 160 partit= ions ( >>>>> data from 2018). We also want to keep the data for up to 2 years in t= he >>>>> cluster-2 to serve the data needs of the customer and so far we reach= ed >>>>> upto 1 year of maintaining this data. >>>>> >>>>> >>>>> >>>>> *Current activity* >>>>> >>>>> We have a custom weekly migration DB script job that moves data from = 1 >>>>> cluster to another cluster what it does is the below things. >>>>> >>>>> 1) COPY command to copy the data from cluster-1 and split that data >>>>> into binary files >>>>> >>>>> 2) Writing the binary data into the cluster-2 table >>>>> >>>>> 3) Creating indexes after the data is copied. >>>>> >>>>> >>>>> >>>>> *Problem what we have right now. * >>>>> >>>>> When the migration activity runs(weekly) from past 2 times , we saw >>>>> the cluster read replica instance has restarted as it fallen behind t= he >>>>> master(writer instance). Everything >>>>> >>>>> after that worked seamlessly but we want to avoid the replica getting >>>>> restarted. To avoid from restart we started doing smaller binary file= s and >>>>> copy those files to the cluster-2 >>>>> >>>>> instead of writing 1 big file of 450 million records. We were >>>>> successful in the recent migration as the reader instance didn=E2=80= =99t restart >>>>> after we split 1 big file into multiple files to copy the data over b= ut did >>>>> restart after the indexes are created on the new table as it could be= write >>>>> intensive. >>>>> >>>>> >>>>> >>>>> *DB parameters set on migration job* >>>>> >>>>> work_mem set to 8 GB and maintenace_work_mem=3D32 GB. >>>>> >>>> >>> >>> >>> Indexes per table =3D 3 >>>>> >>>>> total indexes for 2 tables =3D 5 >>>>> >>>>> >>>>> >>>>> *DB size* >>>>> >>>>> Cluster-2 =3D 8.6 TB >>>>> >>>>> Cluster-1 =3D 3.6 TB >>>>> >>>>> Peak Table relational rows =3D 400 - 480 million rows >>>>> >>>>> Average table relational rows =3D 300 - 350 million rows. >>>>> >>>>> Per table size =3D 90 -95 GB , per table index size is about 45 GB >>>>> >>>>> >>>>> >>>>> *Questions* >>>>> >>>>> 1) Can we decrease the maintenace_work_mem to 16 GB and will it slow >>>>> down the writes to the cluster , with that the reader instance can sy= nc the >>>>> data slowly ?. >>>>> >>>>> 2) Based on the above use case what are your recommendations to keep >>>>> the data longer up to 2 years ? >>>>> >>>>> 3) What other recommendations you recommend ?. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Appreciate your replies. >>>>> >>>>> THanks >>>>> githubkran >>>>> >>>>>> > ------------------------------ > > Este correo es dirigido solo a las personas que aparecen como > destinatarios y > puede contener informaci=C3=B3n confidencial o privilegiada. Si usted rec= ibi=C3=B3 > este correo > por error, por favor notificar inmediatamente al emisor y elimine el > correo original. > Cualquier otro uso de este correo esta prohibido. > > > *************************************************************************= ******************** > > This message is for the designated recipient only and may contain > privileged or > confidential information. If you have received it in error, please notify > the sender > immediately and delete the original. Any other use of the email is > prohibited. > --000000000000cb898a0597c624e1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>Hi<br></div><br><div class=3D"gmail_quote"><div dir= =3D"ltr" class=3D"gmail_attr">st 20. 11. 2019 v=C2=A013:12 odes=C3=ADlatel = Soto Cuevas Manuel Alejandro <<a href=3D"mailto:[email protected]">MCSoto@= entel.cl</a>> napsal:<br></div><blockquote class=3D"gmail_quote" style= =3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding= -left:1ex"> <div dir=3D"auto"> <span style=3D"color:rgb(60,64,67);font-family:Roboto,HelveticaNeue,Arial,s= ans-serif;font-size:small;background-color:rgb(255,255,255)">Taking advanta= ge of the instance, what software do you recommend to do reverse engineering of postgresql?</span></div></blockquote><div><br></div><div>pl= ease, open new thread. Your query is not related to subject.</div><div><br>= </div><div> <a href=3D"https://wiki.postgresql.org/wiki/Design_Tools">https= ://wiki.postgresql.org/wiki/Design_Tools</a></div><div><br></div><div>Regar= ds</div><div><br></div><div>Pavel</div><div><br></div><blockquote class=3D"= gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(20= 4,204,204);padding-left:1ex"><div dir=3D"auto"> <div><font size=3D"2" face=3D"Roboto, HelveticaNeue, Arial, sans-serif" col= or=3D"#3c4043"><span style=3D"background-color:rgb(255,255,255)"><br> </span></font></div> <div><font size=3D"2" face=3D"Roboto, HelveticaNeue, Arial, sans-serif" col= or=3D"#3c4043"><span style=3D"background-color:rgb(255,255,255)">Thank you= =C2=A0</span></font></div> <div><font size=3D"2" face=3D"Roboto, HelveticaNeue, Arial, sans-serif" col= or=3D"#3c4043"><span style=3D"background-color:rgb(255,255,255)"><br> </span></font></div> <div><font size=3D"2" face=3D"Roboto, HelveticaNeue, Arial, sans-serif" col= or=3D"#3c4043"><span style=3D"background-color:rgb(255,255,255)">Manuel</sp= an></font> <div dir=3D"ltr"><br> <blockquote type=3D"cite">El 16-11-2019, a la(s) 10:36, github kran <<a = href=3D"mailto:[email protected]" target=3D"_blank">[email protected]= </a>> escribi=C3=B3:<br> <br> </blockquote> </div> <blockquote type=3D"cite"> <div dir=3D"ltr">=EF=BB=BF <p class=3D"MsoNormal"><b><span style=3D"font-size:10pt;font-family:"A= rial",sans-serif;color:yellow;background:blue none repeat scroll 0% 0%= "><font style=3D"vertical-align:inherit"><font style=3D"vertical-align:inhe= rit">ANEXO ENTEL </font></font></span></b><font style=3D"vertical-align:inherit"><b><span st= yle=3D"font-size:8.5pt;font-family:Verdana,sans-serif;color:rgb(197,90,17);= background-image:initial;background-position:initial;background-size:initia= l;background-repeat:initial;background-origin:initial;background-clip:initi= al"><font style=3D"vertical-align:inherit">Este es un correo externo: </font></span></b><b><span style=3D"font-size:10.5pt= ;font-family:Barlow;color:rgb(0,112,192);background-image:initial;backgroun= d-position:initial;background-size:initial;background-repeat:initial;backgr= ound-origin:initial;background-clip:initial"><font style=3D"vertical-align:= inherit">Verifique remitente, No haga clic en v=C3=ADnculos. </font></span></b><b><span style= =3D"font-size:10.5pt;font-family:Barlow;color:rgb(0,112,192);background-ima= ge:initial;background-position:initial;background-size:initial;background-r= epeat:initial;background-origin:initial;background-clip:initial"><font styl= e=3D"vertical-align:inherit">Si recibes un correo malicioso av=C3=ADsanos a=C2=A0</font></span></b></font>= <font style=3D"vertical-align:inherit"><b><span style=3D"font-size:10.5pt;f= ont-family:Barlow;color:rgb(197,90,17);background-image:initial;background-= position:initial;background-size:initial;background-repeat:initial;backgrou= nd-origin:initial;background-clip:initial;font-weight:normal"><a href=3D"ma= ilto:[email protected]" target=3D"_blank"><span style=3D"color:rgb(197,90,17)">= <font style=3D"vertical-align:inherit"><font style=3D"vertical-align:inheri= t"><font style=3D"vertical-align:inherit">[email protected]</font></font></font= ></span></a></span></b><b><span style=3D"font-size:10.5pt;font-family:Barlo= w;color:rgb(0,112,192);background-image:initial;background-position:initial= ;background-size:initial;background-repeat:initial;background-origin:initia= l;background-clip:initial"><font style=3D"vertical-align:inherit"><font sty= le=3D"vertical-align:inherit"><font style=3D"vertical-align:inherit">=C2=A0= .=C2=A0</font></font></font></span></b></font></p> <span style=3D"text-decoration:underline"><span style=3D"text-decoration:un= derline"></span></span> <div dir=3D"ltr">Any reply on this please ?.</div> <br> <div class=3D"gmail_quote"> <div dir=3D"ltr" class=3D"gmail_attr">On Fri, Nov 15, 2019 at 9:10 AM githu= b kran <<a href=3D"mailto:[email protected]" target=3D"_blank">github= [email protected]</a>> wrote:<br> </div> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div dir=3D"ltr"><br> </div> <br> <div class=3D"gmail_quote"> <div dir=3D"ltr" class=3D"gmail_attr">On Thu, Nov 14, 2019 at 11:42 PM Pave= l Stehule <<a href=3D"mailto:[email protected]" target=3D"_blank">= [email protected]</a>> wrote:<br> </div> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div dir=3D"ltr">=C2=A0 these numbers looks crazy high - how much memory ha= s your server - more than 1TB?=C2=A0</div> </div> </blockquote> <div><br> </div> <div>The cluster got 244 GB of RAM and storage capacity it has is 64 TB.=C2= =A0</div> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div dir=3D"ltr">=C2=A0<br> </div> <br> <div class=3D"gmail_quote"> <div dir=3D"ltr" class=3D"gmail_attr">p=C3=A1 15. 11. 2019 v=C2=A06:26 odes= =C3=ADlatel github kran <<a href=3D"mailto:[email protected]" target= =3D"_blank">[email protected]</a>> napsal:<br> </div> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"><br> <div class=3D"gmail_quote"> <div dir=3D"ltr"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div dir=3D"ltr"> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">Hello postGreSQL Community ,= </span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">=C2=A0</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">Hope everyone is doing great= !!.</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif"><br> </span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <b><u><span style=3D"font-family:Calibri,sans-serif">Background</span></u><= /b></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">We use PostgreSQL Version 10= .6 version and heavily use PostgreSQL for our day to day activities to writ= e and read data. We have 2 clusters running PostgreSQL engine , one cluster= </span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">keeps data up to 60 days and= another cluster retains data beyond 1 year. The data is partitioned close = to a week( ~evry 5 days a partition) and we have around 5 partitions per mo= nth per each table and we have 2 tables primarily so that will be 10 tables a week. So in the cluster-1 we have ar= ound=C2=A0 20 partitions and in cluster-2 we have around 160 partitions ( d= ata from 2018). We also want to keep the data for up to 2 years in the clus= ter-2 to serve the data needs of the customer and so far we reached upto 1 year of maintaining this data.</span= ></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">=C2=A0</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <b><u><span style=3D"font-family:Calibri,sans-serif">Current activity</span= ></u></b></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">We have a custom weekly migr= ation DB script job that moves data from 1 cluster to another cluster what = it does is the below things. </span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">1) COPY command to copy the = data from cluster-1 and split that data into binary files</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">2) Writing the binary data i= nto the cluster-2 table</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">3) Creating indexes after th= e data is copied. </span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <b><u><span style=3D"font-family:Calibri,sans-serif"><span style=3D"text-de= coration-line:none">=C2=A0</span></span></u></b></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <b><u><span style=3D"font-family:Calibri,sans-serif">Problem what we have r= ight now. </span></u></b></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">When the migration activity = runs(weekly) from past 2 times , we saw the cluster read replica instance h= as restarted as it fallen behind the master(writer instance). Everything</s= pan></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">after that worked seamlessly= but we want to avoid the replica getting restarted. To avoid from restart = we started doing smaller binary files and copy those files to the cluster-2 </span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">instead of writing 1 big fil= e of 450 million records. We were successful in the recent migration as the= reader instance didn=E2=80=99t restart after we split 1 big file into mult= iple files to copy the data over but did restart after the indexes are created on the new table as it could be write intens= ive. </span> </p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">=C2=A0</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <b><u><span style=3D"font-family:Calibri,sans-serif">DB parameters set on m= igration job</span></u></b></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">work_mem set to 8 GB=C2=A0 a= nd maintenace_work_mem=3D32 GB.</span></p> </div> </div> </blockquote> </div> </div> </div> </div> </blockquote> <div><br> </div> <div></div> <div><br> </div> <div><br> </div> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div class=3D"gmail_quote"> <div dir=3D"ltr"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div dir=3D"ltr"> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">Indexes per table =3D 3</spa= n></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">total indexes for 2 tables = =3D 5</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">=C2=A0</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <b><u><span style=3D"font-family:Calibri,sans-serif">DB size</span></u></b>= </p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">Cluster-2 =3D 8.6 TB</span><= /p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">Cluster-1 =3D 3.6 TB</span><= /p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">Peak Table relational rows = =3D 400 - 480 million rows</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">Average table relational row= s =3D 300 - 350 million rows.</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">Per table size =3D 90 -95 GB= , per table index size is about 45 GB</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">=C2=A0</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <b><u><span style=3D"font-family:Calibri,sans-serif">Questions</span></u></= b></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">1) Can we decrease the maint= enace_work_mem to 16 GB and will it slow down the writes to the cluster , w= ith that the reader instance can sync the data slowly ?.</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">2) Based on the above use ca= se what are your recommendations to keep the data longer up to 2 years ? </span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">3) What other recommendation= s you recommend ?.</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">=C2=A0</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">=C2=A0</span></p> <p class=3D"MsoNormal" style=3D"margin:4.8pt 0in 2.4pt;line-height:110%;fon= t-size:10pt;font-family:Verdana,sans-serif"> <span style=3D"font-family:Calibri,sans-serif">Appreciate your replies.</sp= an></p> </div> <br> <div class=3D"gmail_quote"> <div class=3D"gmail_attr">THanks</div> <div class=3D"gmail_attr">githubkran</div> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div dir=3D"ltr"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div dir=3D"ltr"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> </blockquote> </div> </div> </blockquote> </div> </div> </blockquote> </div> </div> </blockquote> </div> </div> </blockquote> </div> </blockquote> </div> </div> </blockquote> </div> </div> </blockquote> </div> </blockquote> </div> </div> </div> </blockquote> </div> </blockquote> </div> </div> </div> </blockquote> </div> </blockquote> </div> </div> </blockquote> </div> </blockquote> </div> </div> </blockquote> </div> </div> </div> </div> </blockquote> </div> </div> </blockquote> </div> </div> </blockquote> </div> </div> </blockquote> </div> <br> <hr> <font size=3D"1" face=3D"Arial" color=3D"Gray"><br> Este correo es dirigido solo a las personas que aparecen como destinatarios= y<br> puede contener informaci=C3=B3n confidencial o privilegiada. Si usted recib= i=C3=B3 este correo<br> por error, por favor notificar inmediatamente al emisor y elimine el correo= original.<br> Cualquier otro uso de este correo esta prohibido.<br> <br> ***************************************************************************= ******************<br> <br> This message is for the designated recipient only and may contain privilege= d or<br> confidential information. If you have received it in error, please notify t= he sender<br> immediately and delete the original. Any other use of the email is prohibit= ed.<br> </font> </div> </blockquote></div></div> --000000000000cb898a0597c624e1--