Skip to content

Commit 3c35f86

Browse files
Ben ThomasCopilot
authored andcommitted
fix(hub): address Learnings review feedback
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent 5c12c85 commit 3c35f86

51 files changed

Lines changed: 1277 additions & 198 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/adr/0001-learning-extension-architecture.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
Status: proposed on October 6, 2026, and revised on October 10, 2026 to follow the requirements the Replica maintainers agreed on. Needs review by the Replica maintainers.
44

5-
The learning package, providers, CLI, and demos still work. Hub descriptions below record an earlier proposal, not what the [new Hub](../../hub/README.md) can do today. It does not collect comments, run learning jobs, store learning data, sign people in, or connect Copilot. Adding those features requires separate product and architecture decisions.
5+
The learning package, providers, CLI, and demos implement this architecture. The [new Hub](../../hub/README.md) now stores review conversations, captures complete repository snapshots, connects Copilot, runs explicit learning jobs, records decisions, recalls accepted guidance, and redacts deleted comment text. Daily change-driven discovery remains separate work.
66

77
## Context
88

@@ -96,9 +96,9 @@ Each import records whether every list page and reply page was present. The impo
9696

9797
### Integration with Hub
9898

99-
The earlier proposal had Hub run `replica-learning` for every step, so the package's rules applied throughout. It kept one store per repository outside any checkout. A background job imported review comments, removed deleted text, labeled new or edited comments, and proposed learnings. Administrators reviewed proposals, and the plugin read accepted learnings. The [new Hub](../../hub/README.md) does not run these steps.
99+
Hub runs `replica-learning` for every step, so the package's rules apply throughout. It keeps one store per repository outside any checkout. An explicit background job imports review comments, removes deleted text, labels new or edited comments, and proposes learnings. Administrators review proposals, and recall returns accepted learnings for agents.
100100

101-
The package can label through the [Copilot SDK](https://github.com/github/copilot-sdk). Its session has no tools, refuses every permission request, and never sees who wrote a comment. Copilot chooses the model unless the store's configuration names one, and each run records the usage Copilot reports. The new Hub does not set up or authorize this connection.
101+
The package can label through the [Copilot SDK](https://github.com/github/copilot-sdk). Its session has no tools, refuses every permission request, treats comment text as untrusted data rather than instructions, and never sees who wrote a comment. Copilot chooses the model unless the store's configuration names one, and each run records the usage Copilot reports. Hub supplies only a currently usable configured credential.
102102

103103
The command still runs in a checkout, with data under `.replica/learning/`, for contributors and for other hosts. This complements #230: its discovery output can become labels, and its judge can become a consolidator.
104104

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
model LearningDecisionOperation {
2+
// ID columns
3+
id String @id @default(uuid()) @db.Uuid
4+
repositoryId String @map("repository_id") @db.Uuid
5+
actorMembershipId String @map("actor_membership_id") @db.Uuid
6+
7+
// Relations
8+
repository Repository @relation(fields: [repositoryId], references: [id], onDelete: Cascade)
9+
actorMembership Membership @relation(fields: [actorMembershipId], references: [id], onDelete: Cascade)
10+
11+
// Attribute columns
12+
learningId String @map("learning_id") @db.VarChar(255)
13+
decision String @db.VarChar(20)
14+
15+
// Timestamp columns
16+
createdAt DateTime @default(now()) @map("created_at") @db.Timestamptz(3)
17+
updatedAt DateTime @default(now()) @updatedAt @map("updated_at") @db.Timestamptz(3)
18+
19+
@@index([repositoryId, createdAt])
20+
@@map("learning_decision_operations")
21+
}

‎hub/schema/learning-run.prisma‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ model LearningRun {
44
repositoryId String @map("repository_id") @db.Uuid
55
requestedById String? @map("requested_by_id") @db.Uuid
66
requestedByMembershipId String? @map("requested_by_membership_id") @db.Uuid
7-
number Int @default(autoincrement())
7+
number Int @unique @default(autoincrement())
88
99
// Relations
1010
repository Repository @relation(fields: [repositoryId], references: [id], onDelete: Cascade)

‎hub/schema/membership.prisma‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ model Membership {
1010
sensitiveAuthorizations SensitiveAuthorization[]
1111
copilotConnections CopilotConnection[]
1212
learningRuns LearningRun[]
13+
learningDecisionOperations LearningDecisionOperation[]
1314
1415
// Attribute columns
1516
lifecycle MembershipLifecycle @default(ACTIVE)

‎hub/schema/migrations/20261113190000_learning_run_foundation/migration.sql‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,31 @@ CREATE INDEX "review_comments_pending_learning_redaction_idx"
88
ON "review_comments"("deleted_at", "id")
99
WHERE "deleted_at" IS NOT NULL AND "learning_redacted_at" IS NULL;
1010

11+
CREATE TABLE "learning_decision_operations" (
12+
"id" UUID NOT NULL,
13+
"repository_id" UUID NOT NULL,
14+
"actor_membership_id" UUID NOT NULL,
15+
"learning_id" VARCHAR(255) NOT NULL,
16+
"decision" VARCHAR(20) NOT NULL,
17+
"created_at" TIMESTAMPTZ(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
18+
"updated_at" TIMESTAMPTZ(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
19+
20+
CONSTRAINT "learning_decision_operations_pkey" PRIMARY KEY ("id"),
21+
CONSTRAINT "learning_decision_operations_decision_check"
22+
CHECK ("decision" IN ('accept', 'reject', 'retire'))
23+
);
24+
25+
CREATE INDEX "learning_decision_operations_repository_id_created_at_idx"
26+
ON "learning_decision_operations"("repository_id", "created_at");
27+
28+
ALTER TABLE "learning_decision_operations"
29+
ADD CONSTRAINT "learning_decision_operations_repository_id_fkey"
30+
FOREIGN KEY ("repository_id") REFERENCES "repositories"("id") ON DELETE CASCADE ON UPDATE CASCADE;
31+
32+
ALTER TABLE "learning_decision_operations"
33+
ADD CONSTRAINT "learning_decision_operations_actor_membership_id_fkey"
34+
FOREIGN KEY ("actor_membership_id") REFERENCES "memberships"("id") ON DELETE CASCADE ON UPDATE CASCADE;
35+
1136
CREATE TABLE "learning_runs" (
1237
"id" UUID NOT NULL,
1338
"repository_id" UUID NOT NULL,

‎hub/schema/repository.prisma‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ model Repository {
44
55
// Relations
66
learningRuns LearningRun[]
7+
learningDecisionOperations LearningDecisionOperation[]
78
prototypes Prototype[]
89
reviewThreads ReviewThread[]
910

‎hub/server/domains/learning/create-hub-review-collection/index.test.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,8 @@ describe("createHubReviewCollection()", () => {
105105
})
106106
expect(thread.resolutionHistory).toEqual([
107107
{
108+
id: "66666666-6666-4666-8666-666666666666",
109+
sequence: 1,
108110
state: "resolved",
109111
at: "2026-11-13T10:04:00.000Z",
110112
actor: {
@@ -121,6 +123,8 @@ describe("createHubReviewCollection()", () => {
121123
const laterCapture = snapshot()
122124
laterCapture.snapshotId = "88888888-8888-4888-8888-888888888888"
123125
laterCapture.capturedAt = new Date("2026-11-13T10:06:00Z")
126+
laterCapture.threads[0]!.updatedAt = new Date("2026-11-13T10:06:00Z")
127+
laterCapture.threads[0]!.root.updatedAt = new Date("2026-11-13T10:06:00Z")
124128
expect(createHubReviewCollection(laterCapture, "https://hub.example.com").digest).toBe(
125129
result.digest,
126130
)

‎hub/server/domains/learning/create-hub-review-collection/index.ts‎

Lines changed: 40 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,43 @@ function message(
4040
}
4141
}
4242

43+
function digestMessage(value: ReviewMessage) {
44+
return {
45+
id: value.id,
46+
rootId: value.rootId,
47+
body: value.body,
48+
author: value.author,
49+
createdAt: value.createdAt.toISOString(),
50+
editedAt: value.editedAt?.toISOString() ?? null,
51+
deletedAt: value.deletedAt?.toISOString() ?? null,
52+
}
53+
}
54+
55+
function reviewState(snapshot: RepositoryReviewSnapshot) {
56+
return {
57+
repository: snapshot.repository,
58+
threads: snapshot.threads.map((thread) => ({
59+
id: thread.id,
60+
repositoryId: thread.repositoryId,
61+
prototype: thread.prototype,
62+
buildNumber: thread.buildNumber,
63+
anchor: thread.anchor,
64+
activityVersion: thread.activityVersion,
65+
resolvedAt: thread.resolvedAt?.toISOString() ?? null,
66+
resolvedBy: thread.resolvedBy,
67+
root: digestMessage(thread.root),
68+
replies: thread.replies.map(digestMessage),
69+
resolutionHistory: thread.resolutionHistory.map((event) => ({
70+
id: event.id,
71+
sequence: event.sequence,
72+
state: event.state,
73+
actor: event.actor,
74+
occurredAt: event.occurredAt.toISOString(),
75+
})),
76+
})),
77+
}
78+
}
79+
4380
export function createHubReviewCollection(snapshot: RepositoryReviewSnapshot, hubOrigin: string) {
4481
const target = {
4582
hubOrigin,
@@ -60,6 +97,8 @@ export function createHubReviewCollection(snapshot: RepositoryReviewSnapshot, hu
6097
resolvedAt: thread.resolvedAt?.toISOString() ?? null,
6198
resolvedBy: thread.resolvedBy ? actor(thread.resolvedBy) : null,
6299
resolutionHistory: thread.resolutionHistory.map((event) => ({
100+
id: event.id,
101+
sequence: event.sequence,
63102
state: event.state === "resolved" ? "resolved" : "unresolved",
64103
at: event.occurredAt.toISOString(),
65104
actor: actor(event.actor),
@@ -90,12 +129,7 @@ export function createHubReviewCollection(snapshot: RepositoryReviewSnapshot, hu
90129
return {
91130
collection,
92131
digest: `sha256:${createHash("sha256")
93-
.update(
94-
JSON.stringify({
95-
repository: snapshot.repository,
96-
threads: snapshot.threads,
97-
}),
98-
)
132+
.update(JSON.stringify(reviewState(snapshot)))
99133
.digest("hex")}` as const,
100134
}
101135
}

‎hub/server/domains/learning/create-learning-job-processors/index.ts‎

Lines changed: 95 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,22 @@
11
import type { Logger } from "pino"
22

33
import { getCopilotConnectionStatus } from "../../copilot-connection/get-copilot-connection-status/index.ts"
4+
import { resolveLocalSignInEnvironment } from "../../authentication/resolve-local-sign-in-environment/index.ts"
45
import { createReviewCommentLearningSource } from "../create-review-comment-learning-source/index.ts"
56
import { learningRunJob } from "../jobs/learning-run.ts"
67
import { recoverLearningRunsJob } from "../jobs/recover-learning-runs.ts"
78
import { redactReviewCommentJob } from "../jobs/redact-review-comment.ts"
89
import { prepareLearningStore } from "../prepare-learning-store/index.ts"
910
import { processLearningRun } from "../process-learning-run/index.ts"
1011
import { processReviewCommentRedaction } from "../process-review-comment-redaction/index.ts"
12+
import { reconcileLearningDecisionOperation } from "../reconcile-learning-decision-operation/index.ts"
1113
import { readLearningConfiguration } from "../read-learning-configuration/index.ts"
1214
import { createJobProcessor } from "../../../integrations/bullmq/create-job-processor/index.ts"
1315
import { createJobQueue } from "../../../integrations/bullmq/create-job-queue/index.ts"
1416
import { enqueueRecoverableJob } from "../../../integrations/bullmq/enqueue-recoverable-job/index.ts"
1517
import { runLearningCommand } from "../../../integrations/learning/run-learning-command/index.ts"
18+
import { LearningCommandError } from "../../../integrations/learning/run-learning-command/errors.ts"
19+
import { CopilotCredentialError } from "../../../integrations/github/create-installation-access-token/errors.ts"
1620
import { withLearningExecutionLock } from "../../../integrations/postgres/with-learning-execution-lock/index.ts"
1721
import { getCopilotConnectionState } from "../../../repositories/copilot-connection-states/get-copilot-connection-state/index.ts"
1822
import { advanceCopilotConnectionState } from "../../../repositories/copilot-connection-states/advance-copilot-connection-state/index.ts"
@@ -23,6 +27,8 @@ import { claimLearningRun } from "../../../repositories/learning-runs/claim-lear
2327
import { finishLearningRun } from "../../../repositories/learning-runs/finish-learning-run/index.ts"
2428
import { getLearningRun } from "../../../repositories/learning-runs/get-learning-run/index.ts"
2529
import { listQueuedLearningRuns } from "../../../repositories/learning-runs/list-queued-learning-runs/index.ts"
30+
import { deleteLearningDecisionOperation } from "../../../repositories/learning-decision-operations/delete-learning-decision-operation/index.ts"
31+
import { listLearningDecisionOperations } from "../../../repositories/learning-decision-operations/list-learning-decision-operations/index.ts"
2632
import { updateLearningRunProgress } from "../../../repositories/learning-runs/update-learning-run-progress/index.ts"
2733
import { isEffectiveAdministrator } from "../../../repositories/memberships/is-effective-administrator/index.ts"
2834
import { listDeletedReviewComments } from "../../../repositories/review-comments/list-deleted-review-comments/index.ts"
@@ -34,9 +40,10 @@ export function createLearningJobProcessors(environment: NodeJS.ProcessEnv, logg
3440
let dependenciesPromise: ReturnType<typeof loadDependencies> | undefined
3541
function loadDependencies() {
3642
return (async () => {
37-
const configuration = await readLearningConfiguration(environment)
38-
const databaseUrl = environment.DATABASE_URL
39-
const hubOrigin = environment.HUB_APPLICATION_ORIGIN
43+
const resolvedEnvironment = resolveLocalSignInEnvironment(environment)
44+
const configuration = await readLearningConfiguration(resolvedEnvironment)
45+
const databaseUrl = resolvedEnvironment.DATABASE_URL
46+
const hubOrigin = resolvedEnvironment.HUB_APPLICATION_ORIGIN
4047
if (!databaseUrl || !hubOrigin) {
4148
throw new Error("DATABASE_URL and HUB_APPLICATION_ORIGIN are required for learning jobs.")
4249
}
@@ -52,6 +59,14 @@ export function createLearningJobProcessors(environment: NodeJS.ProcessEnv, logg
5259

5360
const run = createJobProcessor(learningRunJob, async ({ runId }) => {
5461
const { configuration, databaseUrl, prisma, source } = await dependencies()
62+
let credentialSnapshot:
63+
| {
64+
connectionId: string
65+
generation: bigint
66+
credentialDigest: string
67+
credentialProvider: string
68+
}
69+
| undefined
5570
await processLearningRun(runId, {
5671
claim: (id, startedAt) => claimLearningRun(prisma, id, startedAt),
5772
capture: (repositoryId) => source.captureCompleteSnapshot({ repositoryId }),
@@ -68,23 +83,25 @@ export function createLearningJobProcessors(environment: NodeJS.ProcessEnv, logg
6883
staticDirectories: configuration.staticDirectories,
6984
}),
7085
refuseCredential: async () => {
71-
const digest = configuration.copilotCredentialProvider.configurationDigest
72-
if (!digest) return
86+
if (!credentialSnapshot) return
87+
const snapshot = credentialSnapshot
7388
await prisma.$transaction(async (transaction) => {
7489
const state = await lockCopilotConnectionState(transaction)
7590
const connection = await getCopilotConnection(transaction)
7691
if (
7792
!state ||
7893
!connection ||
79-
connection.credentialProvider !== configuration.copilotCredentialProvider.key ||
80-
connection.credentialDigest !== digest
94+
state.generation !== snapshot.generation ||
95+
connection.id !== snapshot.connectionId ||
96+
connection.credentialProvider !== snapshot.credentialProvider ||
97+
connection.credentialDigest !== snapshot.credentialDigest
8198
) {
8299
return
83100
}
84101
if (
85102
!(await recordCopilotCredentialRefusal(transaction, {
86-
credentialDigest: digest,
87-
credentialProvider: configuration.copilotCredentialProvider.key,
103+
credentialDigest: snapshot.credentialDigest,
104+
credentialProvider: snapshot.credentialProvider,
88105
refusedAt: new Date(),
89106
}))
90107
) {
@@ -94,7 +111,7 @@ export function createLearningJobProcessors(environment: NodeJS.ProcessEnv, logg
94111
await createSecurityAudit(transaction, {
95112
actorMembershipId: connection.connectedByMembershipId,
96113
connectionId: connection.id,
97-
credentialProvider: configuration.copilotCredentialProvider.key,
114+
credentialProvider: snapshot.credentialProvider,
98115
id: randomUUID(),
99116
phase: "denied",
100117
reason: "credential_refused",
@@ -135,17 +152,32 @@ export function createLearningJobProcessors(environment: NodeJS.ProcessEnv, logg
135152
refusedAt: state.refusedAt,
136153
},
137154
)
138-
return status.connected
139-
? await configuration.copilotCredentialProvider.getCredential()
140-
: null
155+
if (!status.connected || !connection) return null
156+
credentialSnapshot = {
157+
connectionId: connection.id,
158+
generation: state.generation,
159+
credentialDigest: connection.credentialDigest,
160+
credentialProvider: connection.credentialProvider,
161+
}
162+
try {
163+
return await configuration.copilotCredentialProvider.getCredential()
164+
} catch (error) {
165+
if (
166+
error instanceof CopilotCredentialError &&
167+
(error.code === "installation_revoked" || error.code === "permission_denied")
168+
) {
169+
throw new LearningCommandError(error.code, error.message, { cause: error })
170+
}
171+
throw error
172+
}
141173
},
142174
updateProgress: (id, input) => updateLearningRunProgress(prisma, id, input),
143175
withLock: (operation) => withLearningExecutionLock({ databaseUrl, run: operation }),
144176
})
145177
})
146178

147179
const recover = createJobProcessor(recoverLearningRunsJob, async () => {
148-
const { databaseUrl, prisma } = await dependencies()
180+
const { configuration, databaseUrl, prisma } = await dependencies()
149181
const queue = createJobQueue({ name: "on-demand", databaseUrl, logger })
150182
try {
151183
for (const queued of await listQueuedLearningRuns(prisma)) {
@@ -168,6 +200,55 @@ export function createLearningJobProcessors(environment: NodeJS.ProcessEnv, logg
168200
`learning-redact-${comment.thread.repositoryId}-${comment.publicId}`,
169201
)
170202
}
203+
for (const operation of await listLearningDecisionOperations(prisma)) {
204+
try {
205+
await reconcileLearningDecisionOperation(operation, {
206+
finish: (input) =>
207+
prisma.$transaction(async (transaction) => {
208+
const claimed = await deleteLearningDecisionOperation(
209+
transaction,
210+
input.operationId,
211+
)
212+
if (claimed.count !== 1) return
213+
await createSecurityAudit(transaction, {
214+
id: input.auditId,
215+
type: "learning_decision",
216+
actorMembershipId: operation.actorMembershipId,
217+
decision: operation.decision as "accept" | "reject" | "retire",
218+
learningId: operation.learningId,
219+
operationId: operation.id,
220+
outcome: input.outcome,
221+
...(input.reason ? { reason: input.reason } : {}),
222+
repositoryId: operation.repositoryId,
223+
})
224+
}),
225+
prepare: (repositoryId) =>
226+
prepareLearningStore({
227+
applicationRoot: process.cwd(),
228+
repositoryKey: repositoryId,
229+
repositoryScope: `github.com/repositories/${repositoryId}`,
230+
root: configuration.root,
231+
staticDirectories: configuration.staticDirectories,
232+
}),
233+
runCommand: (arguments_, options) =>
234+
runLearningCommand(arguments_, {
235+
cliPath: configuration.cliPath,
236+
nodePath: configuration.nodePath,
237+
...options,
238+
}),
239+
withLock: (operation_) => withLearningExecutionLock({ databaseUrl, run: operation_ }),
240+
})
241+
} catch (error) {
242+
logger.warn(
243+
{
244+
errorType: error instanceof Error ? error.name : "UnknownError",
245+
operationId: operation.id,
246+
repositoryId: operation.repositoryId,
247+
},
248+
"Pending learning decision reconciliation failed",
249+
)
250+
}
251+
}
171252
} finally {
172253
await queue.close()
173254
}

0 commit comments

Comments
 (0)