Re: Database growth

Hi Mehrdad, all,

Am 19.11.2024 um 10:11 schrieb Eric Bollengier via Bacula-users:
> Hello Mehrdad,
> 
> On 11/19/24 09:30, Mehrdad Ravanbod wrote:
...
>> I am still concerned about the databse and how much it will grow on a 
>> yearly basism and the storage needed for the database tables, i am 
>> planning on using PostgreSQL since it is said that it handles large 
>> tables better than MySQL. Also the retention period specidied is very 
>> long, 10 years to be exact, so purging the datbase is not an option

10 years in IT seems to imply that any prediction will just be a wild 
guess. But we're used to that, right? ;-)

>> Anyone who has some knowledge on this and some parameters to think 
>> about when dealing with such requirements?? The Disk i am using on the 
>> server is 1 TB with a second disk option of 1 TB
> 
> I would suggest to use LVM on your server, doing so, you can add new 
> disks when
> needed.

Also, 10 years life time means there will be some hardware replacements, 
too. But keeping things flexible is indeed always a good choice, and for 
storage, LVM is one way. If you use ZFS or btrfs, there are different 
solutions, and ZFS is known to be robust, whereas btrfs is probably not 
at the same level. But ignore my unfounded assumptions here, I really 
only know software RAID plus LVM plus XFS :-)

> 1TB can hold a lot of records, I think we are around 150-200 
> bytes per
> record

150 to 200 bytes is rather on the low end if you also store job reports 
in the catalog.

In my office/lab:
bacula=# select pg_size_pretty(pg_database_size('bacula')/(select 
count(*) from File));
  pg_size_pretty
----------------
  341 bytes
(1 row)
and I know the size measurement for PostgreSQl is always tricky, but on 
the other hand, you also need the disk for all the collected slack and 
temporary data:
bacula=# select pg_size_pretty(pg_database_size('bacula'));
  pg_size_pretty
----------------
  19 GB
(1 row)
contrasted with
postgres@vanyar:~$ du -sh /var/lib/postgresql/
34G     /var/lib/postgresql/
and thus
$ echo $(( $(du -B1 -s /var/lib/postgresql/|cut -f 1) / 58976511 ))
604

Of course I can not claim my environment to be more normal than anybody 
elses, but at least now you have more data points to start estimating.

> As it's hard to predict, it's always good to have a flexible 
> system.

And that is the most important piece of advice, I think!

Cheers,

Arno

-- 
Arno Lehmann

IT-Service Lehmann
Sandstr. 6, 49080 Osnabrück


_______________________________________________
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users