Summary
NeuroWorkflow currently lets users build workflows out of nodes that compute on, transform, and load/save data — but there is no node for pulling data directly from external reference databases such as the Allen Brain Atlas, NeuroMorpho, or PubMed. The library already contains adapter code for several of these databases, but it is not exposed as nodes, so it can't be used from the workflow editor. This issue proposes a new family of Database nodes that make these external data sources available as first-class, drag-and-drop nodes.
Background / Motivation
A typical workflow today starts from data the user already has on disk (NIfTI, CSV, connectome files) or downloads from a fixed source (e.g. BMCR via S3). Bringing in data from a public neuroscience database means leaving the tool, fetching/exporting the data by hand, and re-importing it as a file. This is slow, hard to reproduce, and invisible inside the workflow graph.
At the same time, the library already ships adapters for several external databases (under neuroworkflow.utils.database_adapters):
- Allen Brain — Allen Brain Atlas
- NeuroMorpho — neuron morphologies (neuromorpho.org)
- PubMed — NCBI literature
- NeuroML-DB — computational model database
- Generic — custom databases via REST / GraphQL / SDK
- Local RAG — local retrieval source
These adapters work, but they live as utility classes only. There is no node that surfaces them in the editor, so users can't drop a "query the Allen Brain Atlas" step into a workflow.
Proposal
Introduce a Database node family. A Database node connects to an external reference database, runs a query/fetch, and outputs the retrieved data into the workflow so that downstream nodes can use it.
How it would feel to a user:
- Drag a Database node onto the canvas (e.g. "Allen Brain query", "PubMed search").
- Configure its parameters (search terms, IDs, filters, etc.).
- The node outputs the fetched data (records / files / tables) on its output ports.
- Existing nodes consume that output exactly as they would any other node's output.
The key idea: Database nodes are where a workflow's data comes from — the entry points that read from external reference databases — as opposed to the existing nodes that work on data the workflow already holds.
Example use cases
- Pull neuron morphologies from NeuroMorpho by species / brain region and feed them into a network-building node.
- Query the Allen Brain Atlas for connectivity or gene-expression data and use it as input to a simulation.
- Search PubMed for references related to a model and attach them as metadata / context.
- Use the Generic adapter to connect a lab's own REST / GraphQL database without writing a new node from scratch.
Scope / non-goals
- This proposal is about reading from external reference databases.
- Persisting workflow results into a relational database (caching / storing outputs) is a separate concern and is not part of this issue.
Open questions (for discussion)
- Which databases should the first version cover? (Suggestion: start with the adapters we already have.)
- How should query parameters and credentials / API keys be handled in the node UI?
- What output port type(s) best represent fetched data — records vs. files vs. tables?
- One node per database, or a single configurable "Database query" node with a source selector?
Possible implementation direction (rough, non-binding)
- Wrap each existing adapter in
neuroworkflow.utils.database_adapters as a node, following the standard node-definition pattern.
- Add a
database node category alongside the existing ones so these nodes group together in the palette.
Summary
NeuroWorkflow currently lets users build workflows out of nodes that compute on, transform, and load/save data — but there is no node for pulling data directly from external reference databases such as the Allen Brain Atlas, NeuroMorpho, or PubMed. The library already contains adapter code for several of these databases, but it is not exposed as nodes, so it can't be used from the workflow editor. This issue proposes a new family of Database nodes that make these external data sources available as first-class, drag-and-drop nodes.
Background / Motivation
A typical workflow today starts from data the user already has on disk (NIfTI, CSV, connectome files) or downloads from a fixed source (e.g. BMCR via S3). Bringing in data from a public neuroscience database means leaving the tool, fetching/exporting the data by hand, and re-importing it as a file. This is slow, hard to reproduce, and invisible inside the workflow graph.
At the same time, the library already ships adapters for several external databases (under
neuroworkflow.utils.database_adapters):These adapters work, but they live as utility classes only. There is no node that surfaces them in the editor, so users can't drop a "query the Allen Brain Atlas" step into a workflow.
Proposal
Introduce a Database node family. A Database node connects to an external reference database, runs a query/fetch, and outputs the retrieved data into the workflow so that downstream nodes can use it.
How it would feel to a user:
The key idea: Database nodes are where a workflow's data comes from — the entry points that read from external reference databases — as opposed to the existing nodes that work on data the workflow already holds.
Example use cases
Scope / non-goals
Open questions (for discussion)
Possible implementation direction (rough, non-binding)
neuroworkflow.utils.database_adaptersas a node, following the standard node-definition pattern.databasenode category alongside the existing ones so these nodes group together in the palette.