DEVOPS-1770 - db_zip2cloud - RC 1.0

[bio-boris]

Motivation

This pull request aims to enhance the db_zip2cloud application's efficiency and versatility:

• The updates are designed to facilitate different types of data restoration:
  • Fast recent restores directly from a local dump directory.
  • Longer-term restores from the local 7zip files.
  • Disaster recovery restores from remote S3 storage via rclone
  • Local retention can be set by variable
• The script serves as a universal tool, capable of zipping and uploading backups from any tool.
• This enhances its adaptability across diverse environments.
• Last but not least, it provides changes needed to work with

Changes

Dockerfile

The Dockerfile has been updated to optimize the build process and reduce the final image size. A multi-stage build process based on Alpine provides a smaller and more secure Docker image.

Application

The zip2cloud application has undergone several changes to improve its functionality.
The new script runs through a number of steps:

1. Cleanup: The cleanup process has been refined to ensure old zip files and backup dumps are removed based on the retention policies set in the environment variables.
2. Zip: The process of creating .7z archives of dump directories has been optimized for better performance.
3. Checksum: The checksum retrieval process has been improved for better accuracy and efficiency when comparing local zip files against remote MD5 checksums.
4. Create Upload List: The process of verifying and updating the list of files to upload has been enhanced to handle filename conflicts more effectively.
5. Upload: The upload process now uses the rclone command for more reliable and efficient uploads to the remote storage.

Conclusion

We've made some key improvements to the db_zip2cloud application. The goal is to make our backup and restore process faster and more reliable, regardless of the data source.

[kkellerlbl]

LGTM. I didn't scrutinize the code, since Jason went over it in detail with me last week, and we know the backups are running.