ENH: Add templates.tsv and cohorts.tsv for template/cohort entity metadata #2287

yarikoptic · 2025-12-18T23:48:37Z

Introduce templates.tsv and cohorts.tsv files consistent with existing BIDS entity-specific TSV files (participants.tsv, sessions.tsv).

templates.tsv: placed at derivative dataset root, describes tpl- entities
cohorts.tsv: placed within tpl-/ directory, describes cohort- entities

This enables consistent documentation of template and cohort entities in derivative datasets. Part of BEP038 atlas metadata improvements.

Schema changes:

Add template_id and cohort_id columns to columns.yaml
Add templates and cohorts suffixes to suffixes.yaml
Add Templates and Cohorts rules to common_derivatives.yaml
Add templates and cohorts file rules to tables.yaml
Update atlas.md with templates.tsv and cohorts.tsv documentation

Related to:

Part of the larger

Replace atlas-<label>_description.json with atlases.tsv pattern and also add templates.tsv and cohorts.tsv #2286

for independent consideration. Attn @bids-standard/bep038

CPernet · 2025-12-19T07:25:47Z

I have to say it annoys me to see this after a lot of work to get there .. but damn it works, and it is consistent with other parts of BIDS .. nice +1 👍 I cannot see any reason not to adopt that change @oesteban what do you think?

oesteban · 2025-12-19T07:46:24Z

I have to say it annoys me to see this after a lot of work to get there .. but damn it works, and it is consistent with other parts of BIDS .. nice +1 👍 I cannot see any reason not to adopt that change @oesteban what do you think?

I understand Yarik's points. Also, his proposal is "idempotent" (in the sense that it doesn't create anything that cannot be completed/modified later down the line, as opposed to our _description.json file that once introduced is hard to get rid of).

That said, Yarik's proposal opens a small hole for fields that require long descriptions, such as a custom license (that cannot be expressed with an identifier). Current BEP038 doesn't solve that problem (which is above its scope btw), but having JSON fields to stick that metadata is more reasonable than having them stuffed into the TSV column.

yarikoptic · 2025-12-20T02:11:35Z

such as a custom license (that cannot be expressed with an identifier). Current BEP038 doesn't solve that problem (which is above its scope btw), but having JSON fields to stick that metadata is more reasonable than having them stuffed into the TSV column.

isn't it a 80/20 rule concern? could likely be "custom: see LICENSE file" for those likely <20% cases?

yarikoptic · 2025-12-20T02:14:49Z

note that this one alone doesn't solve _descriptions.json for which there is a follow up

Replace atlas-<label>_description.json with atlases.tsv pattern and also add templates.tsv and cohorts.tsv #2286

codecov · 2025-12-20T02:32:52Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.81%. Comparing base (074ae6f) to head (2119a13).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2287   +/-   ##
=======================================
  Coverage   82.81%   82.81%           
=======================================
  Files          22       22           
  Lines        1693     1693           
=======================================
  Hits         1402     1402           
  Misses        291      291

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

oesteban · 2025-12-20T07:16:08Z

such as a custom license (that cannot be expressed with an identifier). Current BEP038 doesn't solve that problem (which is above its scope btw), but having JSON fields to stick that metadata is more reasonable than having them stuffed into the TSV column.

isn't it a 80/20 rule concern? could likely be "custom: see LICENSE file" for those likely <20% cases?

It does apply, 80% of the licenses are custom. Universities used to weigh in and request nonstandard licenses be set.

yarikoptic · 2026-01-07T16:25:03Z

80% of the licenses are custom.

do we actually have some stats on that?

Universities used to weigh in and request nonstandard licenses be set.

anyways, licence texts better live in files, not json or TSV, we have docs/ so could be docs/licenses/ and pointed there. Moreover, likely all atlases from the same institution/group would have the same (even if custom) license. For a similar reason Debian now ships /usr/share/common-licenses and just refers to their texts in each package copyright file instead of duplicating them all in each package.
FWIW I even see us (me) proposing such a generic placement (docs/licenses/) for license texts and suggested wording for the top level LICENSE file in such a case to refer to detailed/individual licenses under there.

oesteban · 2026-01-08T07:26:10Z

FWIW I even see us (me) proposing such a generic placement (docs/licenses/) for license texts and suggested wording for the top level LICENSE file in such a case to refer to detailed/individual licenses under there.

That sounds very reasonable to me 👍

…adata Introduce templates.tsv and cohorts.tsv files consistent with existing BIDS entity-specific TSV files (participants.tsv, sessions.tsv). - templates.tsv: placed at derivative dataset root, describes tpl-<label> entities - cohorts.tsv: placed within tpl-<label>/ directory, describes cohort-<label> entities This enables consistent documentation of template and cohort entities in derivative datasets. Part of BEP038 atlas metadata improvements. Schema changes: - Add template_id and cohort_id columns to columns.yaml - Add templates and cohorts suffixes to suffixes.yaml - Add Templates and Cohorts rules to common_derivatives.yaml - Add templates and cohorts file rules to tables.yaml - Update atlas.md with templates.tsv and cohorts.tsv documentation Related to: - #2285 - #2283 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>

src/schema/rules/tabular_data/derivatives/common_derivatives.yaml

effigies

Overall this seems reasonable. We should update at least one example to use these features.

yarikoptic · 2026-01-08T14:22:48Z

FWIW I even see us (me) proposing such a generic placement (docs/licenses/) for license texts and suggested wording for the top level LICENSE file in such a case to refer to detailed/individual licenses under there.

That sounds very reasonable to me 👍

on that aspect -- let's agree to use a standard form (I added above docs/licenses/ there now), please review/follow up there:

Adopt REUSE formalization for licenses specification #2015

but overall point for here is that docs/ is there and not structured ATM and thus "open" to absorb any such relevant files without exploding top level files (IMHO should stay compact for the best overview for humans and machines...).

oesteban · 2026-01-08T19:04:58Z

do we actually have some stats on that?

It is my subjective perception that licenses of atlases/templates follow the 80/20 rule (only 20 use standard licenses, and often with no-derivs restrictions, which is nuts for that kind of resource).

oesteban

This is great, thanks!

oesteban · 2026-01-08T19:06:31Z

src/derivatives/atlas.md

+`templates.tsv` example:
+
+```tsv
+template_id	description


Suggested change

template_id description

template_id long_name

I would strongly suggest RRID here as a second column before "long_name"

+1 for RRID, as always columns are optional but we know that when we put things in example they are more adopted than when we do not - might also consider DOI

Right now description is REQUIRED by the table definition, so either that needs to stay in addition to long_name or it needs to be demoted from REQUIRED. long_name would need to be defined in objects.columns and added to the table definition.

oesteban · 2026-01-08T19:07:57Z

src/derivatives/atlas.md

+`tpl-MNIPediatricAsym_cohorts.tsv` example:
+
+```tsv
+cohort_id	description


This is great 👍

src/schema/objects/columns.yaml

Open discussion points

effigies

Ongoing discussion. Cleared my approval until these are wrapped up.

effigies · 2026-01-09T20:35:04Z

src/schema/rules/tabular_data/derivatives/common_derivatives.yaml

+    - template_id
+    - description__entities


Suggested change

- template_id

- description__entities

- template_id

description being free-text, I would expect many people to choose to put it last for better alignment. I would drop the initial column requirement.

effigies · 2026-01-09T20:37:22Z

src/schema/rules/tabular_data/derivatives/common_derivatives.yaml

+    description__entities:
+      level: required
+      description_addendum: |
+        The corresponding label column is `template_id`.


We can make rrid show up in the table and also validate that it matches the RRID schema:

Suggested change

description__entities:

level: required

description_addendum: |

The corresponding label column is `template_id`.

description__entities:

level: required

description_addendum: |

The corresponding label column is `template_id`.

rrid: optional

This also requires adding rrid to objects.columns.

cc @oesteban @CPernet

effigies · 2026-01-09T20:37:54Z

src/schema/rules/tabular_data/derivatives/common_derivatives.yaml

+    - extension == ".tsv"
+  initial_columns:
+    - cohort_id
+    - description__entities


Again, I think this is overly prescriptive with column ordering.

Suggested change

- description__entities

effigies · 2026-01-09T20:42:26Z

src/derivatives/atlas.md

+`templates.tsv` example:
+
+```tsv
+template_id	description


Right now description is REQUIRED by the table definition, so either that needs to stay in addition to long_name or it needs to be demoted from REQUIRED. long_name would need to be defined in objects.columns and added to the table definition.

effigies · 2026-01-09T20:47:44Z

src/schema/objects/suffixes.yaml

+  description: |
+    A TSV file describing labels found for the `cohort` entity within a template.
+
+    This file MUST be located within a `tpl-<label>/` directory.


This would need an explicit check to validate. Any TSV or JSON file can show up at the root without, even if it doesn't make much sense to do it.

effigies · 2026-01-09T20:49:19Z

src/schema/objects/suffixes.yaml

+  value: templates
+  display_name: Template Entity Definitions
+  description: |
+    A TSV file describing labels found for the `tpl` entity in a Derivatives dataset.


This is a weird way of talking about it. I would say participants.tsv describes the participants, not the participant labels. Similarly with templates. Cohorts feels more reasonable to say labels.

yarikoptic requested a review from effigies December 18, 2025 23:48

yarikoptic added the BEP label Dec 18, 2025

yarikoptic requested a review from erdalkaraca as a code owner December 18, 2025 23:48

yarikoptic mentioned this pull request Dec 18, 2025

Replace atlas-<label>_description.json with atlases.tsv pattern and also add templates.tsv and cohorts.tsv #2286

Open

yarikoptic force-pushed the bep038-cohorts_tsv branch from e15c610 to 5e1ac9b Compare December 20, 2025 02:28

effigies changed the base branch from bep038 to master January 8, 2026 13:30

effigies requested a review from DimitriPapadopoulos as a code owner January 8, 2026 13:30

effigies force-pushed the bep038-cohorts_tsv branch from 5e1ac9b to 9e967fb Compare January 8, 2026 13:31

effigies reviewed Jan 8, 2026

View reviewed changes

src/schema/rules/tabular_data/derivatives/common_derivatives.yaml Outdated Show resolved Hide resolved

effigies previously approved these changes Jan 8, 2026

View reviewed changes

yarikoptic mentioned this pull request Jan 8, 2026

Adopt REUSE formalization for licenses specification #2015

Open

3 tasks

oesteban approved these changes Jan 8, 2026

View reviewed changes

Update src/schema/rules/tabular_data/derivatives/common_derivatives.yaml

2119a13

effigies reviewed Jan 9, 2026

View reviewed changes

ENH: Add templates.tsv and cohorts.tsv for template/cohort entity metadata #2287

Are you sure you want to change the base?

ENH: Add templates.tsv and cohorts.tsv for template/cohort entity metadata #2287

Conversation

yarikoptic commented Dec 18, 2025

Uh oh!

CPernet commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oesteban commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yarikoptic commented Dec 20, 2025

Uh oh!

yarikoptic commented Dec 20, 2025

Uh oh!

codecov bot commented Dec 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

oesteban commented Dec 20, 2025

Uh oh!

yarikoptic commented Jan 7, 2026

Uh oh!

oesteban commented Jan 8, 2026

Uh oh!

Uh oh!

effigies left a comment

Choose a reason for hiding this comment

Uh oh!

yarikoptic commented Jan 8, 2026

Uh oh!

oesteban commented Jan 8, 2026

Uh oh!

oesteban left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CPernet Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

effigies left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

CPernet commented Dec 19, 2025 •

edited

Loading

oesteban commented Dec 19, 2025 •

edited

Loading

codecov bot commented Dec 20, 2025 •

edited

Loading

CPernet Jan 9, 2026 •

edited

Loading