WC2-817: Optimise memory usage of celery task

[butofleury]

What problem is this PR solving? Explain here in one sentence.

Related JIRA tickets : WC2-817

Self proofreading checklist

• Did I use eslint and ruff formatters?
• Is my code clear enough and well documented?
• Are my typescript files well typed?
• New translations have been added or updated if new strings have been introduced in the frontend
• My migrations file are included
• Are there enough tests?
• Documentation has been included (for new feature)

Doc

Tell us where the doc can be found (docs folder, wiki, in the code...).

Changes

• Added a filter on last updated_at based to the latest date when ETL task run successfully.
• Generating analytics data only for entities based their latest updated date.

How to test

From a local instance with wfp coda database having South Sudan and/or Ethiopia and/or Nigeria data run ETL script to generate data and Celery configured, try to identify latest success task for:

• Update the date_created for latest task for each task_name 'plugins.wfp.tasks.etl_ethiopia', 'plugins.wfp.tasks.etl_ng', 'plugins.wfp.tasks.etl_ssd'

• South Sudan: select id,task_name,status,date_created from django_celery_results_taskresult where status='SUCCESS' and task_name='plugins.wfp.tasks.etl_ssd' order by id desc limit 1

• Nigeria: select id,task_name,status,date_created from django_celery_results_taskresult where status='SUCCESS' and task_name='plugins.wfp.tasks.etl_ng' order by id desc limit 1

• Ethiopia: select id,task_name,status,date_created from django_celery_results_taskresult where status='SUCCESS' and task_name='plugins.wfp.tasks.etl_ethiopia' order by id desc limit 1

Try to update the date_created for each of the 3 tasks.

Then run the script to ETL for all data : docker compose run iaso manage etl_ssd && docker compose run iaso manage etl_eth && docker compose run iaso manage etl_ng

It will generate analytics data for entities with the updated at = date_created.

Print screen / video

Upload here print screens or videos showing the changes.

Notes

Things that the reviewers should know:

• known bugs that are out of the scope of the PR
• other trade-offs that were made
• does the PR depends on a PR in bluesquare-components?
• should the PR be merged into another PR?

Follow the Conventional Commits specification

The merge message of a pull request must follow the Conventional Commits specification.

This convention helps to automatically generate release notes.

Use lowercase for consistency.

Example:

fix: empty instance pop up

Refs: IA-3665

Note that the Jira reference is preceded by a line break.

Both the line break and the Jira reference are entered in the Add an optional extended description… field.

[madewulf]

@butofleury I think this PR should also include a way to re-run the ETL on the whole data so that we can re-run it easily if an error is identified in the computation

[quang-le]

see @madewulf 's comment