Skip to content

Latest commit

 

History

History
270 lines (199 loc) · 15.5 KB

README.md

File metadata and controls

270 lines (199 loc) · 15.5 KB

GitHub Actions Receiver

Status
Stability alpha: traces, logs, metrics
Distributions [grafana-ci-otel-collector]
Issues Open issues Closed issues
Code Owners @Elfo404

[grafana-ci-otel-collector]:

The GitHub Actions Receiver processes GitHub Actions webhook events to observe workflows and jobs. It handles workflow_job and workflow_run event payloads, transforming them into trace and log telemetry.

If the receiver is configured in a trace pipeline, each completed GitHub Action workflow or job, along with its steps, are converted into trace spans, allowing the observation of workflow execution times, success, and failure rates.

If a token is provided and the receiver is configured in a logs pipeline, the receiver fetches logs from the GitHub API. If the receiver is configured also in a traces pipeline, logs will contain the traceID and spanId of the relevant span. This provides a complete view of the workflow execution, including logs from each step.

If a secret is configured (recommended), it validates the payload ensuring data integrity before processing.

The receiver supports linking spans to previous runs for workflow_run events, enhancing traceability across workflow attempts. This feature utilises deterministic Trace IDs generated based on the run ID and run attempt. When a workflow_run event contains a PreviousAttemptURL, and the run attempt is greater than 1, the receiver automatically links the current run's root span to the previous run's Trace ID, providing a direct connection between sequential workflow attempts.

Configuration

The following settings are required:

  • endpoint (no default): The endpoint where you may point your webhook to emit events to

The following settings are optional:

  • path (default: '/events'): Path where the receiver instance will accept events
  • secret: GitHub webhook hash signature
  • gh_api: GitHub API configuration details used to retireve workflows logs (can be omitted if no log pipeline is configured for the receiver)
    • base_url: GitHub Enterprise base URL. Can be omitted if using GitHub.com
    • upload_url: GitHub Enterprise upload URL. Can be omitted if using GitHub.com
    • auth: GitHub API authentication details
      • token: GitHub personal access token. Must be empty if any of the following is set
      • app_id GitHub App ID
      • installation_id GitHub App Installation ID
      • private_key_path Path to the GitHub App private key file

Example:

receivers:
  githubactions:
    endpoint: localhost:19418
    path: /events
    secret: It's a Secret to Everybody
    gh_api:
      auth:
        app_id: 123
        installation_id: 456
        private_key_path: /path/to/key.pem

The full list of settings exposed for this receiver are documented here with a detailed sample configuration here

GitHub API Authentication

The receiver requires GitHub API authentication to fetch logs for workflows. The authentication method can be either a Fine-grained personal access token (Beta) or a GitHub App. The receiver supports both methods, but only one can be used at a time.

GitHub apps are recommended as they have higher rate limits and are more secure. To use a GitHub App, you need to create a GitHub App and install it on your repository.

Fine-grained Personal Access Token

Warning

Personal Access Tokens are not recommended. They have lower rate limits and are less secure than GitHub Apps. Use a GitHub App for production deployments.

To create a Fine-graned personal access token (Beta):

  1. On GitHub.com, navigate to Settings -> Developer settings -> Personal access tokens -> Fine-grained tokens
  2. Click Generate new token.
  3. Select a name and an expiration date for the token.
  4. Under Repository access, select either All repositories or Only select repositories and select the repositories you want to access.
  5. Under Permissions -> Repository permissions, set Actions to Read-only.
  6. Click Generate token.

Use the generated token as the ghauth.token in the receiver configuration.

Advanced Configuration

Several helper files are leveraged to provide additional capabilities automatically:

Service Name Generation

The GitHub Actions Receiver dynamically generates the service.name attribute, which is crucial for identifying the source of telemetry data in observability platforms. This name helps in categorising and filtering traces originating from different GitHub repositories or actions workflows.

Default Service Name

By default, the service name is derived from the GitHub repository's full name (e.g., org/repo-name) provided in the payload. This name is then formatted to be all lowercase and to have slashes (/) and underscores (_) replaced with dashes (-), ensuring a clean, URL-friendly identifier.

Customisation Options

While the default naming convention should suffice for most use cases, there might be scenarios where a more descriptive or specific service name is required. For such cases, the receiver's configuration provides options for customization:

  • Custom Service Name: A completely custom service.name that overrides the default naming convention. This is useful when you want to assign a specific, recognisable name to the telemetry data from a particular GitHub repository or workflow.

  • Service Name Prefix and Suffix: These allow for the addition of prefixes or suffixes to the automatically generated service name, offering a balance between automatic naming and customization.

Here's an example snippet from a configuration file (config.yaml) demonstrating how to set these options:

receivers:
  githubactionsreceiver:
    custom_service_name: 'github-com-myorg' # Completely overrides the default service name
    service_name_prefix: 'foo-' # Prepended to the default service name (ignored if custom_service_name is set)
    service_name_suffix: '-bar' # Appended to the default service name (ignored if custom_service_name is set)

GitHub webhooks

Webhooks provide a way for notifications to be delivered to an external web server whenever certain events occur on GitHub.

Subscribing to webhooks

You can either create a webhook for a specific repository or subscribe to events via a GitHub App. The latter is recommended if you are configuring the receiver to use a GitHub App for authentication.

Creating a respositoty webhook

You can create a webhook to subscribe to events that occur in a specific repository. You must be a repository owner or have admin access in the repository to create webhooks in that repository.

  1. On GitHub.com, navigate to the main page of the repository.
  2. Under your repository name, click Settings. If you cannot see the "Settings" tab, select the dropdown menu, then click Settings.
  3. In the left sidebar, click Webhooks.
  4. Click Add webhook.
  5. Under "Payload URL", type the URL where you'd like to receive payloads.
  6. Select the Content type drop-down menu, and click a data format to receive the webhook payload in.
    • Select application/json, which will deliver the JSON payload directly as the body of the POST request.
  7. Optionally (recomended), under "Secret", type a string to use as a secret key. You should choose a random string of text with high entropy. You can use the webhook secret to limit incoming requests to only those originating from GitHub.
  8. Under "Which events would you like to trigger this webhook?"
    • Select Let me select individual events.
    • Uncheck Pushes, which is selected by default.
    • Select Workflow jobs.
    • Select Workflow runs.
  9. To make the webhook active immediately after adding the configuration, select Active.
  10. Click Add webhook.

You can use the GitHub web interface or the REST API to create a repository webhook. For more information about using the REST API to create a repository webhook, see Repository Webhooks.

Alternatively you can use a configuration management tool such as terraform with the github_repository_webhook resource, to configure webhooks.

Subscribing via a GitHub App

If you configure the receiver to use a GitHub App for authentication, you can subscribe to events via the GitHub App. This allows you to receive events from all repositories that have installed the GitHub App.

Creating a GitHub App
  1. On GitHub.com, navigate to Settings -> Developer Settings -> GitHub Apps
  2. Click New GitHub App.
  3. Choose a name for your app, and enter an Homepage URL.
  4. Uncheck Expire user authorization tokens, which is checked by default.
  5. Under Webhook, make sure Active is checked.
  6. Under Webhook URL, type the URL where you'd like to receive payloads.
  7. Optionally (recomended), under Webhook secret (optional), type a string to use as a secret key. You should choose a random string of text with high entropy. You can use the webhook secret to limit incoming requests to only those originating from GitHub.
  8. Under Permissions -> Repository permissions, set Actions to Read-only.
  9. Under Subscribe to events, select Workflow job and Workflow run.
  10. Select where the github can be installed.
  11. Click Create GitHub App.
  12. Once the app is created, you will be taken to the app's settings page. Take note of the App ID shown at the top of the page.
  13. Near the bottom of the page, click Generate a private key and save the private key file.
Installing the GitHub App

After creating the GitHub App, you can install it on your repository.

  1. On GitHub.com, navigate to Settings -> Developer Settings -> GitHub Apps and click Edit next to the GitHub App you created.
  2. On the left sidebar, click Install App.
  3. Click Install next to the account you want to install the GitHub App on.
  4. Select on which repositories you want to install the GitHub App on.
  5. Click Install.
  6. Navigate back to the GitHub App settings page.
  7. On the left sidebar, click Advanced.
  8. Under Recent deliveries, expand a delivery and take note of the Installation ID as shown in the payload.
Configure the Receiver to use the GitHub App

When configuring the receiver, you will need to provide the App ID and Installation ID for the GitHub App. You will also need to provide the path to the private key file.

example:

receivers:
  githubactions:
    # [...] other settings
    gh_api:
      auth:
        app_id: 1
        installation_id: 234
        private_key_path: /path/to/key.pem

Important

Each time you install the GitHub App on a repository, a new installation ID is generated. When configuring the receiver, you will need to provide the installation ID for the repository you want to receive events from. You can create multiple instances of the receiver, each with a different installation ID, to receive events from multiple installations.

receivers:
  githubactions/one:
    # [...] other settings
    gh_api:
      auth:
        app_id: 1
        installation_id: 234
        private_key_path: /path/to/key.pem
  githubactions/two:
    # [...] other settings
    gh_api:
      auth:
        app_id: 1
        installation_id: 789
        private_key_path: /path/to/key.pem

Deterministic IDs

The GitHub Actions Receiver generates deterministic IDs to ensure traceability and consistency across emitted spans. Here’s how the IDs are generated:

  • Trace ID: Generated based on the run ID and run attempt, with a 't' appended to ensure uniqueness across workflow runs and to distinguish it as a trace ID.
  • Parent Span ID: Derived from the workflow job ID and run attempt, with an 's' appended to distinguish it as a span ID and allow association of all steps under a job.
  • Span ID: Specifically generated for each step within a job, using the job ID, run attempt, step name, and an optional step number, to ensure each step within a job can be uniquely identified.

These IDs allow for the correlation of telemetry data within the observability platform, enabling users to link their own spans to those emitted by the receiver.

Generating IDs

Below are example functions in a couple of langues for generating each ID, replicating the logic used by the receiver:

Generating IDs in bash

generate_trace_id() {
  local run_id=$1
  local run_attempt=$2
  echo -n "${run_id}${run_attempt}t" | openssl dgst -sha256 | sed 's/^.* //' | cut -c-32
}

generate_job_span_id() {
  local run_id=$1
  local run_attempt=$2
  local job_name=$3
  echo -n "${run_id}${run_attempt}${job_name}" | openssl dgst -sha256 | sed 's/^.* //' | cut -c-16
}

generate_step_span_id() {
  local run_id=$1
  local run_attempt=$2
  local job_name=$3
  local step_name=$4
  input="${run_id}${run_attempt}${job_name}${step_name}"
  echo -n "$input" | openssl dgst -sha256 | sed 's/^.* //' | cut -c-16
}

# https://docs.github.com/en/actions/learn-github-actions/variables
trace_id=$(generate_trace_id ${GITHUB_RUN_ID} ${GITHUB_RUN_ATTEMPT})
job_span_id=$(generate_job_span_id ${GITHUB_RUN_ID} ${GITHUB_RUN_ATTEMPT} ${GITHUB_JOB})
step_span_id=$(generate_step_span_id ${GITHUB_RUN_ID} ${GITHUB_RUN_ATTEMPT} ${GITHUB_JOB} "your-step-name")

echo "Trace ID: ${trace_id}"
echo "Job Span ID: ${parent_span_id}"
echo "Step Span ID: ${span_id:0:16}"

Generating IDs in Other Languages

You can adapt the logic shown in the bash examples to any programming language that supports SHA-256 hashing. The key is to ensure the input string format matches between your code and the receiver for consistent ID generation.