Skip to content

Remove remaining usage of the pull_request_build table #2363

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Kobzol merged 5 commits into from
Jan 12, 2026
Merged

Remove remaining usage of the pull_request_build table #2363

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b37140d
Do not check the old queue in the new queue anymore
Kobzol Jan 6, 2026
17e2e89
Stop using `pull_request_build` to implement `pr_of`
Kobzol Jan 6, 2026
162eb6a
Stop using `pull_request_build` to implement `parent_of`
Kobzol Jan 6, 2026
a82c299
Remove legacy tables from DB export/import
Kobzol Jan 6, 2026
c3f151d
Remove mention of manually deleting data from the DB
Kobzol Jan 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 1 addition & 26 deletions database/manual-modifications.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
This document contains useful queries and commands that should be performed manually in exceptional
situations.

## Remove data for a stable artifact from the DB
## Remove data for an artifact from the DB
This is important for situations where there is some compilation error for a stable benchmark,
with a stable release of the compiler. While this should be rare, it happens sometimes e.g. because
of future incompatibility lints turning into errors.
Expand All @@ -16,28 +16,3 @@ You can do that either using the following command:
$ cargo run --bin collector purge_artifact <artifact-name>
# $ cargo run --bin collector purge_artifact 1.70.0 # Remove stable artifact 1.70.0
```

Or using SQL queries. There are `ON DELETE CASCADE` clauses for `aid` (artifact ID) on tables that
reference it, so it should be enough to just delete the artifact from the `artifact` table.
The set of queries below show an example of removing the measured data and errors for Rust `1.69`
and `1.70`:
```sql
DELETE FROM artifact
WHERE name IN ('1.69.0', '1.70.0') AND
type = 'release';
```
After executing this query, the server should automatically re-benchmark these artifacts again.

## Remove data for a master/try artifact from the DB
This is similar to removing a stable artifact, however it also has to be removed from the
`pull_request_build` table.

```sql
DELETE FROM artifact
WHERE name = "<sha>";

DELETE FROM pull_request_build
WHERE bors_sha = "<sha>";
```
After that, the server has to be restarted, or you have to send a GET request to its `/perf/onpush`
endpoint.
95 changes: 0 additions & 95 deletions database/src/bin/postgres-to-sqlite.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,33 +77,6 @@ impl Table for Artifact {
}
}

struct ArtifactCollectionDuration;

impl Table for ArtifactCollectionDuration {
fn name(&self) -> &'static str {
"artifact_collection_duration"
}

fn postgres_select_statement(&self, since_weeks_ago: Option<u32>) -> String {
let s = "select aid, date_recorded, duration from ".to_string() + self.name();
with_filter_clause_maybe(s, ARTIFACT_JOIN_AND_WHERE, since_weeks_ago)
}

fn sqlite_insert_statement(&self) -> &'static str {
"insert into artifact_collection_duration (aid, date_recorded, duration) VALUES (?, ?, ?)"
}

fn sqlite_execute_insert(&self, statement: &mut rusqlite::Statement, row: tokio_postgres::Row) {
statement
.execute(params![
row.get::<_, i32>(0),
row.get::<_, DateTime<Utc>>(1).timestamp(),
row.get::<_, i32>(2)
])
.unwrap();
}
}

struct Benchmark;

impl Table for Benchmark {
Expand Down Expand Up @@ -153,36 +126,6 @@ impl Table for Collection {
}
}

struct CollectorProgress;

impl Table for CollectorProgress {
fn name(&self) -> &'static str {
"collector_progress"
}

fn postgres_select_statement(&self, since_weeks_ago: Option<u32>) -> String {
let s = "select aid, step, start_time, end_time from ".to_string() + self.name();
with_filter_clause_maybe(s, ARTIFACT_JOIN_AND_WHERE, since_weeks_ago)
}

fn sqlite_insert_statement(&self) -> &'static str {
"insert into collector_progress (aid, step, start, end) VALUES (?, ?, ?, ?)"
}

fn sqlite_execute_insert(&self, statement: &mut rusqlite::Statement, row: tokio_postgres::Row) {
statement
.execute(params![
row.get::<_, i32>(0),
row.get::<_, &str>(1),
row.get::<_, Option<DateTime<Utc>>>(2)
.map(|d| d.timestamp()),
row.get::<_, Option<DateTime<Utc>>>(3)
.map(|d| d.timestamp()),
])
.unwrap();
}
}

struct Error;

impl Table for Error {
Expand Down Expand Up @@ -269,41 +212,6 @@ impl Table for PstatSeries {
}
}

struct PullRequestBuild;

impl Table for PullRequestBuild {
fn name(&self) -> &'static str {
"pull_request_build"
}

fn postgres_select_statement(&self, _since_weeks_ago: Option<u32>) -> String {
"select bors_sha, pr, parent_sha, complete, requested, include, exclude, runs, backends from "
.to_string()
+ self.name()
}

fn sqlite_insert_statement(&self) -> &'static str {
"insert into pull_request_build (bors_sha, pr, parent_sha, complete, requested, include, exclude, runs, backends) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)"
}

fn sqlite_execute_insert(&self, statement: &mut rusqlite::Statement, row: tokio_postgres::Row) {
statement
.execute(params![
row.get::<_, Option<&str>>(0),
row.get::<_, i32>(1),
row.get::<_, Option<&str>>(2),
row.get::<_, Option<bool>>(3),
row.get::<_, Option<DateTime<Utc>>>(4)
.map(|d| d.timestamp()),
row.get::<_, Option<&str>>(5),
row.get::<_, Option<&str>>(6),
row.get::<_, Option<i32>>(7),
row.get::<_, Option<&str>>(8),
])
.unwrap();
}
}

struct RawSelfProfile;

impl Table for RawSelfProfile {
Expand Down Expand Up @@ -450,14 +358,11 @@ async fn main() -> anyhow::Result<()> {
// Order matters to the extent necessary to satisfy foreign key constraints.
let tables: &[&dyn Table] = &[
&Artifact,
&ArtifactCollectionDuration,
&Benchmark,
&Collection,
&CollectorProgress,
&Error,
&PstatSeries,
&Pstat,
&PullRequestBuild,
&RawSelfProfile,
&RustcCompilation,
&RuntimePstatSeries,
Expand Down
144 changes: 0 additions & 144 deletions database/src/bin/sqlite-to-postgres.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,45 +81,6 @@ impl Table for Artifact {
}
}

struct ArtifactCollectionDuration;

#[derive(Serialize)]
struct ArtifactCollectionDurationRow {
aid: i32,
date_recorded: DateTime<Utc>,
duration: i32,
}

impl Table for ArtifactCollectionDuration {
fn name() -> &'static str {
"artifact_collection_duration"
}

fn sqlite_attributes() -> &'static str {
"aid, date_recorded, duration"
}

fn postgres_attributes() -> &'static str {
"aid, date_recorded, duration"
}

fn postgres_generated_id_attribute() -> Option<&'static str> {
None
}

fn write_postgres_csv_row<W: Write>(writer: &mut csv::Writer<W>, row: &rusqlite::Row) {
let date_recorded: i64 = row.get(1).unwrap();

writer
.serialize(ArtifactCollectionDurationRow {
aid: row.get(0).unwrap(),
date_recorded: Utc.timestamp_opt(date_recorded, 0).unwrap(),
duration: row.get(2).unwrap(),
})
.unwrap();
}
}

struct Benchmark;

#[derive(Serialize)]
Expand Down Expand Up @@ -193,50 +154,6 @@ impl Table for Collection {
}
}

struct CollectorProgress;

#[derive(Serialize)]
struct CollectorProgressRow<'a> {
aid: i32,
step: &'a str,
start_time: Nullable<DateTime<Utc>>,
end_time: Nullable<DateTime<Utc>>,
}

impl Table for CollectorProgress {
fn name() -> &'static str {
"collector_progress"
}

fn sqlite_attributes() -> &'static str {
"aid, step, start, end"
}

fn postgres_attributes() -> &'static str {
"aid, step, start_time, end_time"
}

fn postgres_generated_id_attribute() -> Option<&'static str> {
None
}

fn write_postgres_csv_row<W: Write>(writer: &mut csv::Writer<W>, row: &rusqlite::Row) {
let start: Option<i64> = row.get(2).unwrap();
let end: Option<i64> = row.get(3).unwrap();
let start_time = Nullable(start.map(|seconds| Utc.timestamp_opt(seconds, 0).unwrap()));
let end_time = Nullable(end.map(|seconds| Utc.timestamp_opt(seconds, 0).unwrap()));

writer
.serialize(CollectorProgressRow {
aid: row.get(0).unwrap(),
step: row.get_ref(1).unwrap().as_str().unwrap(),
start_time,
end_time,
})
.unwrap();
}
}

struct Error;

#[derive(Serialize)]
Expand Down Expand Up @@ -360,64 +277,6 @@ impl Table for PstatSeries {
}
}

struct PullRequestBuild;

#[derive(Serialize)]
struct PullRequestBuildRow<'a> {
bors_sha: Nullable<&'a str>,
pr: i32,
parent_sha: Nullable<&'a str>,
complete: Nullable<bool>,
requested: Nullable<DateTime<Utc>>,
include: Nullable<&'a str>,
exclude: Nullable<&'a str>,
runs: Nullable<i32>,
commit_date: Nullable<DateTime<Utc>>,
backends: Nullable<&'a str>,
}

impl Table for PullRequestBuild {
fn name() -> &'static str {
"pull_request_build"
}

fn sqlite_attributes() -> &'static str {
"bors_sha, pr, parent_sha, complete, requested, include, exclude, runs, commit_date, backends"
}

fn postgres_attributes() -> &'static str {
"bors_sha, pr, parent_sha, complete, requested, include, exclude, runs, commit_date, backends"
}

fn postgres_generated_id_attribute() -> Option<&'static str> {
None
}

fn write_postgres_csv_row<W: Write>(writer: &mut csv::Writer<W>, row: &rusqlite::Row) {
let requested: Option<i64> = row.get(4).unwrap();
let commit_date: Option<i64> = row.get(8).unwrap();

writer
.serialize(PullRequestBuildRow {
bors_sha: row.get_ref(0).unwrap().try_into().unwrap(),
pr: row.get(1).unwrap(),
parent_sha: row.get_ref(2).unwrap().try_into().unwrap(),
complete: row.get(3).unwrap(),
requested: Nullable(
requested.map(|seconds| Utc.timestamp_opt(seconds, 0).unwrap()),
),
include: row.get_ref(5).unwrap().try_into().unwrap(),
exclude: row.get_ref(6).unwrap().try_into().unwrap(),
runs: row.get(7).unwrap(),
commit_date: Nullable(
commit_date.map(|seconds| Utc.timestamp_opt(seconds, 0).unwrap()),
),
backends: row.get_ref(9).unwrap().try_into().unwrap(),
})
.unwrap();
}
}

struct RawSelfProfile;

#[derive(Serialize)]
Expand Down Expand Up @@ -731,14 +590,11 @@ async fn main() -> anyhow::Result<()> {
disable_table_triggers(&postgres_tx, &tables).await;
// Order matters to the extent necessary to satisfy foreign key constraints.
copy::<Artifact>(&sqlite_tx, &postgres_tx).await;
copy::<ArtifactCollectionDuration>(&sqlite_tx, &postgres_tx).await;
copy::<Benchmark>(&sqlite_tx, &postgres_tx).await;
copy::<Collection>(&sqlite_tx, &postgres_tx).await;
copy::<CollectorProgress>(&sqlite_tx, &postgres_tx).await;
copy::<Error>(&sqlite_tx, &postgres_tx).await;
copy::<PstatSeries>(&sqlite_tx, &postgres_tx).await;
copy::<Pstat>(&sqlite_tx, &postgres_tx).await;
copy::<PullRequestBuild>(&sqlite_tx, &postgres_tx).await;
copy::<RawSelfProfile>(&sqlite_tx, &postgres_tx).await;
copy::<RustcCompilation>(&sqlite_tx, &postgres_tx).await;
copy::<RuntimePstatSeries>(&sqlite_tx, &postgres_tx).await;
Expand Down
9 changes: 2 additions & 7 deletions database/src/pool.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,15 +150,10 @@ pub trait Connection: Send + Sync {

async fn collector_remove_step(&self, aid: ArtifactIdNumber, step: &str);

// TODO: the following two functions do not work anymore and should not be used!
/// Returns the sha of the parent commit, if available.
///
/// (Currently only works for try commits)
/// Returns the SHA of the parent of the given SHA commit, if available.
async fn parent_of(&self, sha: &str) -> Option<String>;

/// Returns the PR of the parent commit, if available.
///
/// (Currently only works for try commits)
/// Returns the PR associated with an artifact with the given SHA, if available.
async fn pr_of(&self, sha: &str) -> Option<u32>;

/// Returns the collection ids corresponding to the query. Usually just one.
Expand Down
7 changes: 2 additions & 5 deletions database/src/pool/postgres.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1249,7 +1249,7 @@ where
async fn parent_of(&self, sha: &str) -> Option<String> {
self.conn()
.query_opt(
"select parent_sha from pull_request_build where bors_sha = $1",
"SELECT parent_sha FROM benchmark_request WHERE tag = $1",
&[&sha],
)
.await
Expand All @@ -1258,10 +1258,7 @@ where
}
async fn pr_of(&self, sha: &str) -> Option<u32> {
self.conn()
.query_opt(
"select pr from pull_request_build where bors_sha = $1",
&[&sha],
)
.query_opt("SELECT pr FROM benchmark_request WHERE tag = $1", &[&sha])
.await
.unwrap()
.map(|r| r.get::<_, i32>(0) as u32)
Expand Down
Loading
Loading