Skip to content

Yuta-K19418/Refedle

Repository files navigation

Refedle

Refedle is a TUI-driven data transformation tool for CSV and JSON files, built with .NET 10 and Terminal.Gui v2. It lets you explore a file interactively, apply column-level transformations, and replay them as a recipe against large files from the command line.

Download

Prebuilt binaries (no .NET SDK required) are published on the Releases page for:

OS Architecture
Windows x64
macOS Apple Silicon (arm64)
Linux x64
Linux arm64

Intel Macs are not covered by a prebuilt binary (Apple discontinued Intel support as of macOS 26 Tahoe, and GitHub Actions retired its osx-x64 runner). Build from source instead with dotnet publish src/App/Refedle.App.csproj -r osx-x64 -c Release.

Download the archive matching your platform, extract it, and run the refedle (or refedle.exe on Windows) binary directly:

./refedle [--file <path>] [--recipe <path.yaml>]

The binaries are unsigned, so the OS may block them on first launch:

  • macOS: Gatekeeper quarantines downloaded files. Run xattr -d com.apple.quarantine refedle before launching, or allow it via System Settings → Privacy & Security.
  • Windows: SmartScreen may warn about an unrecognized app. Click "More info" → "Run anyway".

In the examples below, dotnet run --project src/App -- can be replaced with ./refedle (or refedle.exe on Windows) when using a downloaded binary instead of building from source.

To build and run from source instead, see TUI Usage below.

Supported Formats

Format TUI (Tree) TUI (Table) CLI batch (--cli)
CSV (.csv)
JSON Lines (.jsonl)
JSON Array (.json) via drill-down only planned
JSON Object (.json) via drill-down only planned

Any file extension other than those listed above results in a NotSupportedException.

CSV (.csv) — TUI Table view. No Tree view, since CSV rows have no nested structure to drill into.

JSON Lines (.jsonl) — TUI Tree and Table view (toggle with t), plus full-file aggregation drill-down (see TUI Usage).

JSON Array (.json) — TUI Tree view, with Table view available only via full-file aggregation drill-down (see TUI Usage). Not yet supported in CLI batch mode.

JSON Object (.json) — TUI Tree view, with Table view available only via single-node drill-down (see TUI Usage). Not yet supported in CLI batch mode.

TUI Usage

dotnet run --project src/App -- [--file <path>] [--recipe <path.yaml>]

Key bindings:

Key Action
o Open file
s Save recipe
t Toggle Tree/Table view (JSON Lines only)
x Action menu (Column/Row Actions, Drill-down)
c Clear action stack
Backspace Back from drill-down
? Help
q Quit (confirms if there are unsaved actions)

Column/Row Actions

Available from the action menu (x):

View Column/Row Actions
CSV (Table)
JSON Lines (Table)
Table from drill-down (any format) planned

Rename — renames a column.

Before:

nm age
Alice 30

After (renamed nm to name):

name age
Alice 30

Delete — removes a column from the dataset.

Before:

name age
Alice 30

After (deleted age):

name
Alice

Cast — converts a column's values to a different type (text, whole number, floating point, etc.).

Before:

age
"30"

After (cast age from text to whole number):

age
30

Filter — keeps only the rows where a column matches a condition (equals, not-equals, greater/less than, etc.). Multiple filters combine with AND.

Before:

age
30
20

After (filtered age > 25):

age
30

Fill — overwrites every value in a column with a fixed value; useful for anonymization, masking, or bulk initialization.

Before:

email
alice@example.com
bob@example.com
carol@example.com

After (filled with "REDACTED"):

email
REDACTED
REDACTED
REDACTED

Format Timestamp — reformats a Timestamp column's string values into a different date/time format.

Before:

created_at
2024-01-15T09:30:00Z
2024-03-02T14:05:00Z
2024-06-21T08:45:00Z

After (formatted as yyyy-MM-dd):

created_at
2024-01-15
2024-03-02
2024-06-21

Drill-down

Also available from the action menu (x), when the current view is in Tree mode. Two modes exist:

View Drill-down type
JSON Lines (Tree) Full-aggregation
JSON Array (Tree) Full-aggregation
JSON Object (Tree) Single-node

Single-node drill-down — turns the selected node itself into a table (JSON Object only — there's a single record to explore). The selected node must be a non-empty array of objects; selecting an object, a scalar (a plain value — not an object or array), or an array with non-object elements fails with an error.

Example — a JSON Object file:

{
  "user": "alice",
  "orders": [
    { "id": 1, "item": "Book", "price": 12.5 },
    { "id": 2, "item": "Pen", "price": 1.2 }
  ]
}

Path: orders

Drilling down produces:

id item price
1 Book 12.5
2 Pen 1.2

Full-file aggregation drill-down — scans the entire file and aggregates the selected path across every record into a table (JSON Lines/Array).

Behavior by selected node type

The shape of the resulting table depends on what the selected path resolves to in each record:

  • Object — one row per record, using the object's keys as columns.
  • Array — one row per element; object elements become row columns, primitive elements become a single value column (always typed as Text). Selecting a specific array element (e.g. tags[0]) produces the same result as selecting the array itself — the whole array is always expanded.
  • Scalar (a plain value — not an object or array) — one row with a single column (named after the path's last key), always typed as Text regardless of the actual value.

Example — a JSON Lines file (one record per line):

{"user": "alice", "cart": {"orders": [{"id": 1, "item": "Book"}]}}
{"user": "bob", "cart": {"orders": [{"id": 2, "item": "Pen"}, {"id": 3, "item": "Mug"}]}}

Path: cart > orders

Drilling down scans every line and aggregates all matching arrays into one table:

id item
1 Book
2 Pen
3 Mug

Recipes

Pressing s saves the current action stack as a .yaml recipe, named after the source file.

Example — people.csv:

nm age email
Alice 30 alice@example.com
Bob 20 bob@example.com

After renaming nmname, filling email with "REDACTED", and filtering age > 25:

name age email
Alice 30 REDACTED

Pressing s at this point produces people.yaml:

name: "people"
lastModified: "2026-07-26T12:34:56.0000000+00:00"
actions:
  - type: rename
    oldName: "nm"
    newName: "name"
  - type: fill
    columnName: "email"
    value: "REDACTED"
  - type: filter
    columnName: "age"
    operator: GreaterThan
    value: "25"

Recipes can then be replayed against other files via CLI Batch Usage, without opening the UI.

CLI Batch Usage

Input \ Output CSV JSON Lines JSON Array JSON Object
CSV planned planned
JSON Lines in development in development planned planned
JSON Array planned planned planned planned
JSON Object planned planned planned planned
dotnet run --project src/App -- --cli --input <input> --recipe <recipe.yaml> --output <output>

Format dispatch (reader → transform → writer) is resolved at compile time via a source generator (src/Generators/FormatDispatcherGenerator.cs), not reflection.

Project Structure

src/
  App/         TUI (Terminal.Gui v2) and CLI entry point (Program.cs, Cli/)
  Engine/      File I/O (mmap-backed), schema scanning, filtering, actions, recipe (de)serialization
  Generators/  Roslyn incremental source generator for format-agnostic dispatch
tests/
  Refedle.Tests/
docs/          Design documents

Implementation Notes

  • File reads use System.IO.MemoryMappedFiles with ArrayPool<byte> buffer reuse; CSV parsing is backed by the Sep library. There is no SIMD/vectorized scanning code in the engine at this time.
  • Recipe YAML is a hand-written, AOT-safe reader/writer (no YamlDotNet or reflection-based serialization).
  • Error handling favors a Result/Result<T> return type over exceptions on expected failure paths.
  • Both App and Engine are configured for Native AOT (PublishAot=true / IsAotCompatible=true) with TreatWarningsAsErrors enabled.

Requirements

Build & Test

dotnet build
dotnet test

Acknowledgements

Built with Terminal.Gui and Sep, both MIT licensed. See THIRD-PARTY-NOTICES.txt for full license texts.

License

MIT

About

Interactively shape CSV/JSON data in a terminal UI, then replay the transformation as a CLI batch recipe.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages