Pg 17 Incremental backup support for Barman Cloud

[vladimirfx]

Please consider the implementation of incremental backups for the Cloud variant.

Motivation:

Now Barman Cloud (in contract to Barman) is the weakest backup option on the 'cloud' market because it lacks incremental backup support. Moreover, it kills any chance of deduplication on the storage side because of 100Gb tar blobs that are always unique instead of Pg data files themselves. As a result, Barman Cloud archives are up to 10 times bigger than PgBackrest or WAL-G for a 1-1.5Tb database with moderate update activity. It's a practical observation based on running a CNPG-based cluster, so incremental backups are a game changer for us (now is a stopper for us to fully adoption of CNPG on cluster > 1Tb).

Implementing support for Pg 17 incremental backups does not require a complete change of Braman Cloud architecture but mostly metadata and logic enhancements. In that way, tar archives can't be deduplicated but will be much smaller than now (of course for incremental backups only).

[martinmarques]

I don't think that the current implementation of incremental backups in PG17 will work well with CNPG. The backup part is simple and straight forward to implement. But the restore is a lot more complex, and would require changes in CNPG or the use of a barman operator that would take care of the combining.

We have been thinking of this, but we haven't landed on an idea with consensus