Skip to content

Conversation

kalleep
Copy link
Contributor

@kalleep kalleep commented Oct 14, 2025

PR Description

loki.source.file have a main loop that is responsible to read and forward log entries from tailer tasks. When we get an update to add new tasks we store new tasks and add a signal to this main loop, this is done to keep calls to Update cheap.

The problem is that we will not update running tasks if we are blocked on sending an entry. This pr splits up the consumer work and task scheduling into two different go routines. With this change we no longer need to start a drain job while we update tasks.

Which issue(s) this PR fixes

Notes to the Reviewer

PR Checklist

  • CHANGELOG.md updated
  • Documentation added
  • Tests updated
  • Config converters updated

@kalleep kalleep added the publish-dev:linux builds and deploys an image to grafana/alloy-dev container repository label Oct 14, 2025
We also make sure to collect all targets and perform the export every
time component is updated with new discovery data
@kalleep kalleep added publish-dev:linux builds and deploys an image to grafana/alloy-dev container repository and removed publish-dev:linux builds and deploys an image to grafana/alloy-dev container repository labels Oct 16, 2025
@kalleep kalleep requested a review from Copilot October 16, 2025 12:58
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR optimizes the loki.source.file component by splitting the main loop into two separate goroutines: one for consuming and forwarding log entries, and another for managing task updates. This architectural change eliminates blocking issues where task updates were delayed when the system was busy sending entries, removing the need for a drain job during updates.

Key changes:

  • Separated log entry consumption and task scheduling into independent goroutines using a WaitGroup
  • Removed the complex drain goroutine mechanism that was previously needed during task updates
  • Enhanced local.file_match component with immediate update triggers and change detection to reduce unnecessary exports

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
internal/component/loki/source/file/file.go Refactored main loop into two goroutines (consumer and scheduler), removed drain mechanism, improved error handling for task updates
internal/component/local/file_match/file.go Added trigger channel for immediate updates, implemented target comparison to avoid redundant exports, changed RWMutex to Mutex

@kalleep kalleep added publish-dev:linux builds and deploys an image to grafana/alloy-dev container repository and removed publish-dev:linux builds and deploys an image to grafana/alloy-dev container repository labels Oct 16, 2025
@kalleep kalleep added publish-dev:linux builds and deploys an image to grafana/alloy-dev container repository and removed publish-dev:linux builds and deploys an image to grafana/alloy-dev container repository labels Oct 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

publish-dev:linux builds and deploys an image to grafana/alloy-dev container repository

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant