Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache / prebuild hvp_libraries.xml instead of building it from scratch for each backup #534

Open
tsbartel opened this issue Jan 11, 2024 · 0 comments

Comments

@tsbartel
Copy link

As other issues have reported, backups with libraries tend to get big and - which is worse - extremely slow. The reason for this is the backup step backup_hvp_libraries_structure_step (see code).

The file hvp_libraries.xml can reach sizes of 30-40 MB and take up to 40 minutes to build depending on the number of libraries installed. There may very well be sites where these numbers are exceeded.

Suggestion: Create a folder hvp in the moodledata directory, where a nightly cronjob builds the file or rebuilds it if the libraries have been updated. Ideally, build a course-specific file that only contains libraries (and dependencies) actually used within the course!

Reasoning:

  • Requirements for course backups can differ within the same instance. Some, like automatic backups, are mainly used for availability and restores in case of user error - these do not need to contain libraries, as they are usually restored on the same site. Others are generated by teachers themselves to share on other sites (as OER content or when moving to another institution).
  • When importing to another instance, it is generally not desired to import all the hvp libraries as these usually contain lots of legacy libraries and also libraries which will not be used on the new instance.

Pros:

  • Backup time greatly reduced. On our production environment, automated backups with libraries would take about 53 days longer with the current approach. We are currently able to run automated backups once a week with a few days to spare.
  • Greater flexibility when reusing content on other sites. Currently the preferred method is to either do a complete course backup with libraries and just deal with the extra unwanted libraries or do a backup without libraries and re-upload the mod_hvp content as .h5p files manually where it fails.

Cons:

  • If hvp_libraries.xml is created via Cron instead of on demand, some backups may still have to wait for a current version to be created in case it is outdated. Otherwise they would be created with outdated libraries, which is not desirable.
  • In case of course-specific files (i.e. hvp_libraries_course_{courseID}.xml), lots of duplicate data would be stored on moodledata, potentially bloating storage space.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant