Skip to content

Conversation

@bmartel
Copy link
Contributor

@bmartel bmartel commented Dec 11, 2025

This pull request refactors how background jobs are started for file exports and storage synchronization by consolidating the logic into the start_job_async_or_sync utility. This change ensures that context (like user and organization IDs) is consistently captured and restored, simplifies the code by removing repetitive Redis connection checks, and improves job tracking. The main updates are in the export and storage sync flows, making them more robust and easier to maintain.

Refactoring and simplification of background job execution:

  • Replaced direct usage of django_rq and manual Redis connection checks with the start_job_async_or_sync utility in label_studio/data_export/mixins.py and label_studio/io_storages/base_models.py, ensuring consistent context propagation and reducing code duplication. [1] [2] [3] [4] [5] [6]

Job tracking and logging improvements:

  • Updated job tracking logic to store and log job IDs only when an asynchronous job is actually created, improving reliability and observability of background tasks. [1] [2]

Background job deduplication:

  • Adjusted the logic for checking duplicate jobs to ensure it only runs when Redis is connected, preventing unnecessary checks in synchronous mode.

Context handling enhancements:

  • Added comments and ensured that user context (user_id, organization_id, request_id) is always captured and restored for background jobs, which is required for correct export and storage sync processing. [1] [2] [3]

@bmartel bmartel requested review from a team, matt-bernstein and mcanu December 11, 2025 15:13
@netlify
Copy link

netlify bot commented Dec 11, 2025

Deploy Preview for label-studio-docs-new-theme canceled.

Name Link
🔨 Latest commit c2b38c9
🔍 Latest deploy log https://app.netlify.com/projects/label-studio-docs-new-theme/deploys/693b2a925f1c6e000877ce5c

@github-actions github-actions bot added the fix label Dec 11, 2025
@netlify
Copy link

netlify bot commented Dec 11, 2025

Deploy Preview for label-studio-playground canceled.

Name Link
🔨 Latest commit c2b38c9
🔍 Latest deploy log https://app.netlify.com/projects/label-studio-playground/deploys/693b2a92f9e35d0008822c63

@netlify
Copy link

netlify bot commented Dec 11, 2025

Deploy Preview for heartex-docs ready!

Name Link
🔨 Latest commit c2b38c9
🔍 Latest deploy log https://app.netlify.com/projects/heartex-docs/deploys/693b2a92246b7b000836f8ec
😎 Deploy Preview https://deploy-preview-8990--heartex-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@netlify
Copy link

netlify bot commented Dec 11, 2025

Deploy Preview for label-studio-storybook canceled.

Name Link
🔨 Latest commit c2b38c9
🔍 Latest deploy log https://app.netlify.com/projects/label-studio-storybook/deploys/693b2a9253d49700075de93f

Comment on lines +118 to +120
# Debug: Log what we're capturing
logger.info(f'_capture_context: captured context_data={context_data}')

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Debug: Log what we're capturing
logger.info(f'_capture_context: captured context_data={context_data}')

Comment on lines +101 to +105

logger.info(
f'BaseExportDataSerializer: task_id={task.id}, state_in_ret={ret.get("state")}, '
f'user={user}, user_id={getattr(user, "id", None)}, flag_568={flag_568}, flag_710={flag_710}'
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
logger.info(
f'BaseExportDataSerializer: task_id={task.id}, state_in_ret={ret.get("state")}, '
f'user={user}, user_id={getattr(user, "id", None)}, flag_568={flag_568}, flag_710={flag_710}'
)

Comment on lines +79 to +80
logger = logging.getLogger(__name__)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
logger = logging.getLogger(__name__)

Comment on lines +74 to +75
import logging

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
import logging

Copy link
Contributor

@yyassi-heartex yyassi-heartex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, just clearing out some logging that looks to be there for debugging

@yyassi-heartex
Copy link
Contributor

yyassi-heartex commented Dec 12, 2025

/git merge

Workflow run
Successfully merged: Already up to date.

@bmartel bmartel closed this Dec 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants