Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

279 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

marathon-match-api-v6

NestJS service for managing marathon match scorer configuration, compiling tester JARs, consuming submission events from Kafka, and launching ECS scoring tasks.

Development runtime

Use Node.js 26.5.1 and pnpm 11.15.1. Run nvm use in this project before running pnpm commands. The production image uses Alpine's dynamically linked Node.js package and patched system OpenSSL, and runs as the unprivileged app user (UID 10001). Both Docker stages use the same Alpine release. The runtime accepts Alpine's available Node.js package within the build's Node.js major version, allowing minor, patch, and package revision updates without depending on older packages remaining available. The final image excludes npm and pnpm. The image declares an app-owned /work volume for compilation workspaces and Maven's runtime cache. For ECS tasks with readonlyRootFilesystem: true, mount a writable task volume at /work; the path must match the Dockerfile's VOLUME so ECS initializes its contents and ownership from the image. A root-owned volume with mode 0755 prevents the app user from creating compilation files. The image defaults to COMPILATION_TMP_DIR=/work/mm-compile and MAVEN_OPTS=-Dmaven.repo.local=/work/.m2/repository. If COMPILE_MAVEN_OPTS overrides Maven options, include a repository path on the writable volume. The image also creates an app-owned /work/tmp directory for deployments that set TMPDIR, TMP, and TEMP to /work/tmp. Point these variables only at directories that exist in the image. The startup prisma migrate deploy resolves the temp directory and exits with ENOENT if it is missing, which stops the task before the API starts.

Service base path

All HTTP endpoints are exposed under:

/v6/marathon-match

Swagger UI:

/v6/marathon-match/api-docs

Documentation

Configuration values

The service is configured via environment variables.

Core service and database

Variable Required Default Used for
PORT No 3000 HTTP listen port
NODE_ENV No (unset) Token-expiration behavior and Prisma logging behavior
CORS_ALLOWED_ORIGIN No Built-in localhost/topcoder regex list Exact CORS matching for a full browser Origin value
DATABASE_URL Yes None Prisma + pg-boss Postgres connection
POSTGRES_SCHEMA No public Prisma schema name used in runtime logging/connection context
MM_SERVICE_PRISMA_TIMEOUT No 10000 Prisma transaction timeout (ms)

JWT auth and M2M auth

Variable Required Default Used for
AUTH_SECRET Yes (for JWT validation) None tc-core JWT authenticator secret
VALID_ISSUERS No Topcoder/Auth0 issuer JSON array string Accepted JWT issuers
AUTHORIZATION_SESSION_VALIDATION_ENABLED No true Enables Identity API active-session validation for user JWTs
AUTHORIZATION_VALIDATION_URL No Derived from https://api.topcoder(-dev).com token issuers Full Identity API authorizations/1 URL used to reject logged-out user tokens
AUTHORIZATION_VALIDATION_TIMEOUT_MS No 3000 Timeout for active-session validation requests
AUTH0_ISSUER No https://topcoder-dev.auth0.com/ Legacy JWT config field
TOKEN_AUDIENCE No https://m2m.topcoder-dev.com/ Legacy JWT config field
AUTH0_URL No http://localhost:4000/oauth/token M2M token endpoint
AUTH0_DOMAIN No topcoder-dev.auth0.com M2M config metadata
AUTH0_AUDIENCE No https://m2m.topcoder-dev.com/ M2M audience
AUTH0_PROXY_SERVER_URL No (unset) Optional M2M proxy
AUTH0_CLIENT_ID Yes (for outbound API calls) None M2M client ID
AUTH0_CLIENT_SECRET Yes (for outbound API calls) None M2M client secret

Kafka consumer/producer

Variable Required Default Used for
DISABLE_KAFKA No false Fully disable Kafka connection/consumption
KAFKA_URL No localhost:9092 Broker list (comma-separated). If unset, KAFKA_BROKERS is also accepted
KAFKA_BROKERS No (fallback only) Alternative broker list env key (review-api compatibility)
KAFKA_CLIENT_ID No tc-marathon-match-api Kafka client ID
KAFKA_GROUP_ID No tc-marathon-match-consumer-group Consumer group ID
KAFKA_SSL_ENABLED No false Enable TLS
KAFKA_SASL_MECHANISM No plain SASL mechanism (plain, scram-sha-256, scram-sha-512)
KAFKA_SASL_USERNAME No (unset) SASL username (enables SASL when set)
KAFKA_SASL_PASSWORD No empty string SASL password
KAFKA_CONNECTION_TIMEOUT No 10000 Kafka connect timeout (ms)
KAFKA_REQUEST_TIMEOUT No 30000 Client-side deadline for an in-flight Kafka request (ms)
KAFKA_BROKER_TIMEOUT No 5000 Timeout sent to broker APIs that support one (ms)
KAFKA_SESSION_TIMEOUT No 60000 Consumer group session timeout (ms)
KAFKA_HEARTBEAT_INTERVAL No 3000 Consumer group heartbeat interval (ms)
KAFKA_MAXBYTES / KAFKA_MAX_BYTES No 10485760 Consumer aggregate Fetch limit (10 MiB)
KAFKA_MIN_BYTES No Kafka client default Consumer fetch minimum bytes
KAFKA_MAX_WAIT_TIME No 5000 Maximum time the broker may hold an idle Fetch request (ms)
KAFKA_RETRY_ATTEMPTS No 5 Client reconnection retry count
KAFKA_INITIAL_RETRY_TIME No 100 Initial retry delay (ms)
KAFKA_MAX_RETRY_TIME No 30000 Max exponential retry delay (ms)
KAFKA_DLQ_ENABLED No false Enable DLQ publishing after retry exhaustion
KAFKA_DLQ_TOPIC_SUFFIX No .dlq DLQ topic suffix
KAFKA_DLQ_MAX_RETRIES No 3 Per-message retries before DLQ

Kafka access uses @platformatic/kafka 2.8.0. KAFKA_REQUEST_TIMEOUT must exceed both KAFKA_BROKER_TIMEOUT and KAFKA_MAX_WAIT_TIME, and KAFKA_SESSION_TIMEOUT must exceed the sum of KAFKA_HEARTBEAT_INTERVAL and KAFKA_REQUEST_TIMEOUT. Terminal consumer, producer, stream, and offset-commit errors move Kafka health to reconnecting and rebuild both clients. Reconnect delays use bounded jitter; exhausted attempts leave Kafka health as failed with the last error reason. Initial startup and every reconnect resume from the consumer group's committed offsets, falling back to the latest offset only when the group has no committed position. If an offset commit fails, the active processing loop stops before a later record from that partition can advance the committed position, and recovery rebuilds the stream.

Scoring completion email notifications

When both EXAMPLE and PROVISIONAL review summations are complete for a submission, the service sends one email event through Bus API topic external.action.email. A failed PROVISIONAL result also triggers this email without waiting for an example result. Terminal FAILED progress callbacks trigger the notification check even when progress is below 100% and the runner exits without a final result callback; cancelled runs do not count as completed results. Submission emails use SENDGRID_TEMPLATE_ID_SCORING_COMPLETE for pass and SENDGRID_TEMPLATE_ID_SCORING_FAILED for fail. If the selected template is unset, the service logs a warning and skips sending without reserving a notification; failures never fall back to the success template.

When SYSTEM scoring is complete for every latest member submission in the challenge, the service sends separate system-results emails with each member's final placement. All template payloads include scoringStatus as pass or fail, and local notification markers prevent duplicate sends for duplicate scorer callbacks. The email challengeName is resolved from challenge-api-v6, not from the local Marathon Match scorer config name. The service resolves member identity from submission-api-v6 using the submission ID; if no handle is present, it uses the submission userId or memberId to call member-api-v6 GET /members?userId={userId} before publishing the email event.

Variable Required Default Used for
SENDGRID_TEMPLATE_ID_SCORING_COMPLETE Yes (for success emails) None SendGrid template ID for successful example/provisional scoring
SENDGRID_TEMPLATE_ID_SCORING_FAILED Yes (for failure emails) None SendGrid template ID for failed example/provisional scoring
SENDGRID_TEMPLATE_ID_SYSTEM_TEST_RESULTS Yes (to send emails) None SendGrid template ID for system test result emails
MEMBER_API_URL No https://api.topcoder-dev.com/v6 member-api-v6 base URL used to fetch the competitor email by handle
BUS_EVENTS_URL No (derived) Full Bus API event endpoint URL; overrides the base URL variables
BUS_API_URL / BUSAPI_URL No https://api.topcoder-dev.com/v5 Bus API base URL used to publish external.action.email events
TC_EMAIL_FROM_EMAIL / EMAIL_FROM No no-reply@topcoder.com Sender and reply-to address for the email payload

The failure template receives the same data fields as the success template: memberHandle, submissionId, challengeName, challengeId, challengeUrl, challengeURL, scoringStatus (fail), and aggregateProvisionalScore (-1 for runner failures). Set SENDGRID_TEMPLATE_ID_SCORING_FAILED to the new SendGrid dynamic template ID to enable these emails.

Marathon scoring integration

Variable Required Default Used for
CHALLENGE_API_URL No https://api.topcoder-dev.com Challenge API lookup for current active phase
SUBMISSION_API_URL No ${CHALLENGE_API_URL}/v6 Submission API base URL used for submission lookups/downloads; prod should use https://api.topcoder.com/v6
RESOURCES_API_URL No ${CHALLENGE_API_URL}/v6/resources Resource API lookup for challenge-specific copilot access to manual reruns
DISABLE_PG_BOSS No false Disable pg-boss queue/worker and run tester compilation inline
DEFAULT_REVIEW_SCORECARD_ID Yes (for UI defaults) None Default review scorecard returned by GET /challenge/defaults
DEFAULT_TEST_TIMEOUT_MS No 90000 Default test timeout returned by GET /challenge/defaults
DEFAULT_COMPILE_TIMEOUT_MS No 120000 Default compile timeout returned by GET /challenge/defaults
DEFAULT_SYSTEM_TEST_TIMEOUT_MS No 86400000 Default total SYSTEM scoring timeout returned by GET /challenge/defaults
DEFAULT_MEMORY_LIMIT_MB No 2048 Default ECS scorer task memory limit (MiB) returned by GET /challenge/defaults
DEFAULT_TASK_DEFINITION_NAME No empty string Default ECS task definition family returned by GET /challenge/defaults
DEFAULT_TASK_DEFINITION_VERSION No empty string Default ECS task definition revision returned by GET /challenge/defaults
COMPILE_TIMEOUT_MS No 120000 Maven tester compilation timeout
COMPILE_JAVA_MAX_HEAP_MB No 384 Max JVM heap (MB) enforced for tester compilation Maven process when -Xmx is not already provided
COMPILE_MAVEN_OPTS No (auto-derived) Compile-worker specific MAVEN_OPTS; if unset, falls back to MAVEN_OPTS and auto-appends -Xmx cap
MVN_BINARY No mvn Maven executable for tester compilation
BOILERPLATE_DIR No <repo>/ecs-runner/boilerplate Java boilerplate project copied for compilation
COMPILATION_TMP_DIR No Image: /work/mm-compile; otherwise TMPDIR, os.tmpdir(), <repo>/tmp Writable, executable temp root used for compile workspaces; owned by the application user
PG_BOSS_COMPILE_TEAM_SIZE No 1 Number of pg-boss compile workers processing jobs in parallel
PG_BOSS_COMPILE_TEAM_CONCURRENCY No 1 Per-worker concurrency for compile jobs

When all compilation temp directories fail, compilationError includes each attempted path and its error. Check the configured path first: EACCES indicates insufficient permissions, while EROFS indicates a read-only filesystem. The last fallback (/app/tmp in the image) can report ENOENT even when an earlier candidate failed because the writable volume was owned by the wrong user.

ECS launch configuration

Variable Required Default Used for
AWS_REGION No us-east-1 AWS SDK ECS client region
ECS_CLUSTER Yes (for scoring) None ECS cluster for RunTask
ECS_SUBNETS Yes (for scoring) None Comma-separated subnets for awsvpc task networking
ECS_SECURITY_GROUPS Yes (for scoring) None Comma-separated security groups for awsvpc networking
ECS_CONTAINER_NAME Yes (for scoring) None Container override target in task definition
ECS_SCORER_MAX_CONCURRENT_TASKS No 20 Maximum pending/running Marathon Match scorer tasks this API will allow before retry back-pressure
MARATHON_MATCH_API_URL Yes (for scoring) None Base URL passed to ECS runner
REVIEW_API_URL Yes (for scoring) None Review API base URL used by NestJS scoring callback processor
REVIEW_TYPE_ID Yes (for scoring) None Review type ID passed to ECS runner callback payload
DEBUG_LOG_ACCESS_TOKEN No false Pass-through to ECS runner for access-token debug logging (redacted token + decoded JWT header/payload)

launchScorerTask(...) already disables public IP assignment for scorer tasks. It also enforces the pending/running scorer task cap before calling RunTask, skips duplicate active launches for the same challenge/submission/phase, and stops older active tasks for the same challenge/member when a newer submission arrives. Use dedicated scorer security groups with least-privilege egress for the trusted bootstrap/callback traffic that remains on the parent runner process.

SYSTEM review dispatch uses a durable pg-boss queue when ECS_SCORER_MAX_CONCURRENT_TASKS is reached, so Review-phase submissions are retried until scorer capacity frees up instead of being dropped by the phase-open request. Configure that queue with:

Variable Required Default Used for
SYSTEM_SCORE_DISPATCH_RETRY_DELAY_SECONDS No 300 Fixed delay between pg-boss retries when deferred SYSTEM dispatch cannot launch yet
SYSTEM_SCORE_DISPATCH_RETRY_LIMIT No 10000 Retry count for deferred SYSTEM dispatch jobs
SYSTEM_SCORE_DISPATCH_WORKER_CONCURRENCY No 1 Number of deferred SYSTEM dispatch jobs the API worker attempts concurrently

ECS runner task environment (injected at launch)

These are required by ecs-runner and are passed in container overrides when a task is launched:

  • TESTER_CONFIG_ID (contains the challenge ID used by /challenge/:challengeId endpoints)
  • SUBMISSION_ID
  • ACCESS_TOKEN
  • MARATHON_MATCH_API_URL
  • REVIEW_TYPE_ID
  • TEST_PHASE
  • PHASE_CONFIG_TYPE
  • PHASE_START_SEED (decimal 64-bit integer string)
  • PHASE_NUMBER_OF_TESTS
  • REVIEW_ID (optional; used for SYSTEM review completion callbacks)

Optional debug vars (set on API service env to be forwarded to runner):

  • DEBUG_LOG_ACCESS_TOKEN

Submission isolation inside the ECS runner

The ECS task still needs trusted outbound access to fetch challenge config, download submission artifacts, upload artifacts, and post the scoring callback. Untrusted tester/submission execution is therefore split from that bootstrap logic inside the container:

  • The container starts as the non-root runner-parent user (uid 10000). Do not override the ECS task-definition user; the entrypoint rejects other UIDs. Narrow root:runner mode 4750 setuid helpers fork and immediately drop tester commands to runner (uid 10001) and submitted solution commands to scorer (uid 10002), while their supervisors return to the invoking non-root UID with only timeout-signalling capability. The scorer account is not in group runner and cannot invoke either privileged helper directly.
  • The trusted parent runner holds ACCESS_TOKEN, performs network calls, and never loads untrusted submission code directly.
  • The parent launches a separate child JVM through mm-runner-isolate with a scrubbed environment, so submission processes do not inherit the bearer token or other runner env vars.
  • Generic submitted solution commands run through mm-scorer-isolate as the separate non-root scorer user. Downloaded tester JARs and serialized scorer config are parent-owned, runner-group-readable mode 0440, so submitted code cannot read or modify them; the config is deleted before tester code executes.
  • Standard generic-runner seed execution asks mm-scorer-isolate to reset scorer-owned writable state before and after each test case. The cleanup helper accepts no path argument, scans only fixed writable roots such as /tmp, /var/tmp, /dev/shm, and the scorer home, and only removes entries owned by the scorer UID. Descriptor-relative no-follow operations prevent a scorer-controlled symlink swap from redirecting the privileged traversal.
  • Generic submitted solution commands also run under a filesystem allowlist that permits runtime/toolchain reads, /proc/self/maps for glibc/Mono stack introspection, and scorer temp writes but does not permit reading infrastructure paths such as /etc/hostname, /etc/resolv.conf, /proc/self/cgroup, /proc/self/mounts, or proc network tables.
  • Native wrappers block io_uring and creation of non-AF_UNIX sockets for submitted solution processes and their fork/exec children, so submissions cannot open live outbound network connections.
  • Standard Topcoder Marathon testers run through the generic runner flow, which creates the callback score payload from trusted runner code. Custom tester runTester(...) result maps remain supported for advanced cases.

Exit code 137 (OOM) mitigation

137 usually means the process was killed by the container runtime due to memory pressure. For development environments with tight memory limits, use these settings first:

COMPILE_JAVA_MAX_HEAP_MB=256
COMPILE_MAVEN_OPTS="-Xms128m -Xmx256m"
PG_BOSS_COMPILE_TEAM_SIZE=1
PG_BOSS_COMPILE_TEAM_CONCURRENCY=1

If you are not actively consuming submission events or async compile queues in a dev smoke test, also disable background workers:

DISABLE_KAFKA=true
DISABLE_PG_BOSS=true

Endpoints and auth

All secured endpoints require Authorization: Bearer <token>.

Auth model in code:

  • User JWT with role administrator passes role checks.
  • User JWT with role copilot also passes the scorer/tester setup routes used by platform-ui.
  • M2M JWT passes with required scope.
  • all:marathon-match and all:marathon-match-tester are expanded to their CRUD scopes.

Public endpoints

Method Path Auth Notes
GET /v6/marathon-match/health None DB and Kafka consumer health check
GET /v6/marathon-match/api-docs None Swagger docs route

Tester endpoints

Method Path Required role/scope
POST /v6/marathon-match/testers administrator OR copilot OR create:marathon-match-tester
GET /v6/marathon-match/testers administrator OR copilot OR read:marathon-match-tester
GET /v6/marathon-match/testers/:id administrator OR copilot OR read:marathon-match-tester
PUT /v6/marathon-match/testers/:id administrator OR copilot OR update:marathon-match-tester
DELETE /v6/marathon-match/testers/:id administrator OR delete:marathon-match-tester

POST /v6/marathon-match/testers creates only the first record in a tester family and rejects names that already exist. GET /v6/marathon-match/testers returns tester summary rows only. GET /v6/marathon-match/testers/:id returns tester details with sourceCode. PUT /v6/marathon-match/testers/:id creates a new tester version while preserving older versions for lookup and selection. Detail and version-create responses omit jarFile by default; add ?includeJarFile=true only when you explicitly need the compiled jar payload.

Marathon match config endpoints

Method Path Required role/scope
POST /v6/marathon-match/challenge/:challengeId administrator OR copilot OR create:marathon-match
GET /v6/marathon-match/challenge administrator OR read:marathon-match
GET /v6/marathon-match/challenge/defaults administrator OR copilot OR read:marathon-match
GET /v6/marathon-match/challenge/:challengeId administrator OR copilot OR read:marathon-match
GET /v6/marathon-match/challenge/:challengeId/tester-jar administrator OR read:marathon-match
PUT /v6/marathon-match/challenge/:challengeId administrator OR copilot OR update:marathon-match
DELETE /v6/marathon-match/challenge/:challengeId administrator OR delete:marathon-match

Submission runner log endpoint

Method Path Required role/scope
GET /v6/marathon-match/submissions/:submissionId/runner-logs administrator OR copilot OR Manager OR read:marathon-match

Internal scoring callback endpoint

Method Path Required role/scope
POST /v6/marathon-match/internal/scoring-progress administrator OR update:marathon-match
POST /v6/marathon-match/internal/scoring-results administrator OR update:marathon-match
POST /v6/marathon-match/internal/system-score administrator OR update:marathon-match

POST /v6/marathon-match/internal/scoring-results rejects callbacks whose challengeId does not map to an existing Marathon Match config. POST /v6/marathon-match/internal/scoring-progress stores runner progress in review summation metadata as testProcess (provisional or system), testProgress (0 to 1), and testStatus (IN PROGRESS, SUCCESS, or FAILED). Completed scoring can still report SUCCESS when individual testcases timed out or crashed; those counts stay in testProgressDetails.failedTests.

How to set up a challenge for marathon match scoring

For the full operator guide, see Marathon Match Setup. The steps below remain as a quick reference.

1. Create a tester

Create a tester (POST /testers) with:

  • name
  • version
  • className (fully-qualified Java Marathon tester class)
  • sourceCode

POST /testers is only for a brand-new tester name. If that tester family already exists, use PUT /testers/:id and a higher version instead.

Compilation is async through pg-boss. The create/version-create endpoint returns before compilation finishes and includes the tester record you should poll for compilation status.

2. Wait for compilation success

Poll GET /testers/:id until:

  • compilationStatus = SUCCESS

If you need the compiled jar in the response, call GET /testers/:id?includeJarFile=true after compilation succeeds.

If compilationStatus = FAILED, publish a higher tester version via PUT /testers/:id.

3. Create challenge config

Create config on the challenge id (POST /challenge/:challengeId) and include at minimum:

  • testerId (from step 1)
  • reviewScorecardId
  • relativeScoringEnabled (true to normalize scores against the latest submission per member; defaults to true)
  • scoreDirection (MAXIMIZE when higher raw testcase scores are better, MINIMIZE when lower scores are better)
  • submissionApiUrl
  • taskDefinitionName
  • taskDefinitionVersion
  • active (true to enable scoring)
  • phase mappings (example, provisional, system) with:
    • phaseId (from challenge-api phases[].phaseId, not the challenge-phase row id)
    • startSeed as a decimal string for full PostgreSQL BIGINT/Java long range support
    • numberOfTests

Config identity semantics:

  • id is an internal nano-id for the marathon match config record.
  • challengeId stores the challenge identifier and is used for /challenge/:challengeId CRUD endpoints.
  • POST /challenge/:challengeId validates that the challenge exists in challenge-api and that reviewScorecardId resolves in review-api before persisting. If a config already exists for the challenge, the API returns 409 Conflict.
  • phaseConfig.phaseId stores the canonical challenge phase definition id from challenge-api phases[].phaseId. Create/update requests also accept the challenge-phase row id and normalize it before persistence for backwards compatibility.
  • reviewScorecardId can be either the current review-api scorecard id or a legacy id; scoring callback processing resolves it to the canonical scorecard id before posting review summations.
  • When relativeScoringEnabled = true, review scores are recomputed from per-test raw scores against the best score currently held by the latest submission from each member, so final review summation scores stay within 0..100 and can change as new submissions arrive. The raw callback result is stored only as a pending placeholder until the recompute worker writes the normalized score. Recompute uses submission-api to choose the latest submission per member, hydrates phase review summation metadata directly from Review API, finalizes pending placeholders it can see, and retries instead of writing a normalized score if no latest-submission baseline with usable raw testScores is available.

Important runtime behavior:

  • Incoming submission events are only processed when config is active = true.
  • The handler resolves currently open challenge phases from challenge-api (phases[].isOpen = true) and matches stored phase config rows by the canonical challenge phaseId. Legacy stored challenge-phase row ids are also recognized for backwards compatibility.
  • If no matching phase config exists, the submission is skipped.
  • If multiple phase configs match the currently open challenge phases, the handler launches one scorer task per match. This allows EXAMPLE and PROVISIONAL scoring to run from the same Submission phase when both configs point at that phase.

4. Ensure scorer infrastructure is configured

Before live scoring, verify:

  • Kafka topic marathonmatch.submission.received is receiving events.
  • Service can fetch M2M token (AUTH0_CLIENT_ID/AUTH0_CLIENT_SECRET).
  • ECS env vars are set (ECS_CLUSTER, ECS_SUBNETS, ECS_SECURITY_GROUPS, ECS_CONTAINER_NAME, MARATHON_MATCH_API_URL, REVIEW_API_URL, REVIEW_TYPE_ID).
  • Task definition referenced by taskDefinitionName:taskDefinitionVersion exists and contains the configured container name.

5. Optional verification calls

  • GET /challenge/:challengeId to verify stored config and phase mappings.
  • GET /challenge/:challengeId/tester-jar to verify compiled jar retrieval.

ECS runner image (ECR)

The scorer task launched by EcsService must reference an ECR image built from:

  • ecs-runner/Dockerfile

Recommended publish flow:

AWS_REGION=us-east-1 ECR_REPOSITORY=mm-ecs-runner ./ecs-runner/scripts/build-and-push-ecr.sh

Detailed runner image and tagging guidance:

  • ecs-runner/README.md

Submission scoring flow (Kafka to score)

sequenceDiagram
  autonumber
  participant K as Kafka Topic<br/>marathonmatch.submission.received
  participant C as KafkaConsumerService
  participant H as MarathonMatchSubmissionHandler
  participant DB as Postgres (Prisma)
  participant CA as challenge-api-v6
  participant ECS as EcsService
  participant F as AWS ECS Fargate Task
  participant MM as marathon-match-api-v6
  participant SA as Submission API
  participant SRS as ScoringResultService
  participant RA as review-api-v6

  K->>C: Submission event envelope ({topic, payload{submissionId, challengeId, ...}})
  C->>H: handle(payload)
  H->>DB: Load marathonMatchConfig + tester + phaseConfigs

  alt Config missing or inactive
    H-->>C: Skip message
  else Config active
    H->>CA: GET /v6/challenges/:challengeId (M2M token)
    CA-->>H: current/open phases

    alt No open phase or no mapped phase config
      H-->>C: Skip message
    else One or more phase configs are mapped
      alt Tester compilationStatus != SUCCESS
        H-->>C: Throw (retry/DLQ path)
      else Tester ready
        loop Once per matching phase config
          H->>ECS: launchScorerTask(configId, submissionId, phaseConfig)
          ECS->>F: RunTask with env overrides
          F->>MM: GET /challenge/:id
          F->>MM: GET /challenge/:id/tester-jar
          F->>MM: GET /testers/:testerId
          F->>SA: Download submission artifacts
          F->>F: Load tester JAR + generic MarathonController execution
          F->>MM: POST /internal/scoring-results
          MM->>SRS: processScoringResult(payload)
          SRS->>RA: POST/PUT review summations
          Note over SRS,RA: When relative scoring is enabled and testScores metadata is present,\nScoringResultService persists the raw result and queues latest-submission recomputation.
          F-->>ECS: Task exits
        end
        H-->>C: Success
      end
    end
  end

  C->>C: Commit offset on success/skip
  Note over C: On failures retries with exponential backoff. If DLQ is enabled and retries are exhausted, publish to topic.dlq and commit.
Loading

SYSTEM review scoring uses the same scorer pipeline after autopilot dispatches:

  • POST /v6/marathon-match/internal/system-score
  • ECS scorer task execution
  • POST /v6/marathon-match/internal/scoring-progress
  • POST /v6/marathon-match/internal/scoring-results

Review phase scoring flow

See Review Phase Scoring for the end-to-end review-phase sequence from review creation through challenge completion.

Shared authentication and bus client (v6 refresh)

This branch uses tc-core-library-js 4.0.0-rc.1. Use its named createJwtVerifier / createJwtAuthenticator, createM2MClient, and createBusApiClient exports for shared authentication and bus API publishing. Service-specific roles, scopes, sessions and ownership checks remain in this service. The former bus wrapper dependency and local JWT cryptography must not be reintroduced.

RSA verification requires an exact configured issuer and audience (JWT_AUDIENCE for client tokens, AUTH0_AUDIENCE for machine/API tokens, or this service's explicit issuer-policy settings). HMAC secrets must be at least 32 bytes; no default secret is supplied. Expiry is enforced. Machine identity comes from the shared verifier, not the presence of scopes. Supply real development credentials through the service's existing environment configuration before deployment.

Bus publishers use createBusApiClient(config).postEvent(event) with the existing event envelope and optional key. Configure BUSAPI_URL for v6 and the existing OAuth credentials/token provider. The client uses bounded HTTPS requests and does not retry POSTs automatically. Application retry/outbox behavior remains local.

See the core migration guide for the API mapping, compatibility requirements, and rollout scope.

Alpine Node runtime packages

Container stages that install Node through APK constrain nodejs-current to the major version in NODE_VERSION, rather than an exact Alpine package revision. Alpine replaces older patch packages in its repositories; the compatible pin allows current security patches while a build-time check rejects a different Node major. The runtime keeps Alpine's dynamically linked Node and system libraries. Existing non-root users and application entrypoints are preserved. Build stages using a versioned official Node image retain their configured version, and pnpm build arguments are explicitly declared in the consuming stage.

Shared JWT deployment configuration

Development deployments load /config/common/global-appvar alongside the service's application parameters. JWT verification uses the same four settings as the other v6 services: VALID_ISSUERS (JSON array or comma-separated exact issuers), AUTH_SECRET (the existing shared HS256 signing key, at least 32 UTF-8 bytes), JWT_AUDIENCE (the browser application's audience), and AUTH0_AUDIENCE (the machine/API audience). RSA keys are fetched over HTTPS from the trusted issuer's /.well-known/jwks.json. Audience settings must reach the shared verifier; OAuth client IDs/secrets used for outbound requests are independent of JWT audiences.

The development browser audience is BXWXUWnilVUPdN01t2Se29Tw2ZYNGZvH, and the machine audience is https://m2m.topcoder-dev.com/. Use the existing Parameter Store secret rather than creating a per-service key. Reconcile any service-specific overrides of these four keys with the global values before deployment. Parameters are consumed on deployment/task startup; changing Parameter Store alone does not replace running tasks. Existing role, scope, ownership and active-session checks continue after JWT verification.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages