Skip to content

MIGRATION.md

Latest commit

 

History

History
455 lines (351 loc) · 20 KB

MIGRATION.md

File metadata and controls

455 lines (351 loc) · 20 KB
Raw

Migration Guide

This guide covers moving existing client state into TorrentNG. The migration goal is the import side of universal compatibility: preserve the torrent metadata, paths, labels/categories/tags, counters, file priorities, and resume state that other clients have accumulated wherever the source format makes that recoverable.

TorrentNG also has a reverse export library path in rt_migrate::export for anti-lock-in workflows. It projects TorrentNG-client DB rows, persisted .torrent blobs, and TorrentNG-client fastresume state back into qBittorrent/Deluge libtorrent, Transmission, rTorrent, uTorrent/BitTorrent Classic, BiglyBT/Vuze, or generic .torrent + manifest layouts. Export plans report whether each torrent is recheck-free, complete-only, metadata-only, or torrent-only for the chosen target.

Current migration coverage includes rTorrent, qBittorrent, Transmission, Deluge, uTorrent/BitTorrent Classic, BiglyBT/Vuze, Tixati, and generic .torrent directories, with exact status tracked in CLIENT_COMPATIBILITY_MATRICES.md. For the broader TorrentNG client overview and client-swap workflow, see ENGINE_REWRITE.md.

Fidelity Vocabulary At A Glance

Migration reports use two different vocabularies depending on direction. They measure different things at different times, and conflating them is the most common way to misread a dry-run report:

Direction Vocabulary What it measures
Importing in (torrentngd migrate) trusted / hints / metadata-only / none How much piece state could be decoded from the source client at scan time
Exporting out (torrentngd export) recheck-free / complete-only / metadata-only / torrent-only How much of TorrentNG-client state survives being written into the target client's format

A torrent that imports as hints — the normal label for anything mid-download, see below — is not a lower grade of imported data. Once it's in TorrentNG-client state, it can still export later as recheck-free: hints only describes what the scanner could confirm about the source at import time, not what survived the import.

The torrentngd migrate command

For TorrentNG-client deployments, torrentngd migrate is the one-shot import path. It scans a source client's state directory read-only, prints a dry-run report, and (with --apply) writes TorrentNG-client DB rows and compatible fast-resume state together so complete torrents resume seeding without a full recheck.

torrentngd migrate --source <SRC> --from <DIR> [OPTIONS]
  • --source — rtorrent, qbittorrent, transmission, deluge, utorrent, biglybt, tixati, or generic
  • --from — the source client state directory (never modified)
  • --apply — perform the import; omit for a dry-run report (the default)
  • --policy — fast-resume trust: verify, trust-hints (default), or trust-all. trust-hints only trusts complete state when the data files are present at the expected sizes; anything else falls back to verification. If the source client looks like it crashed, the policy is lowered to verify automatically (see Crash detection).
  • --trust-unclean-source — keep --policy even when the source looks crashed
  • --allow-running-source — import even though the source client appears to be running (its state may be mid-write; refused by default)
  • --remap OLD=NEW — rewrite a save-path prefix (repeatable), e.g. --remap /downloads=/data when data moved into a container
  • --default-save-path DIR — fallback save path when the source recorded none
  • --report FILE — also write the markdown dry-run report to FILE
  • --config FILE — config file (else TORRENTNGD_CONFIG / defaults); this determines the TorrentNG-client DB and fast-resume target locations
  • --yes — skip the confirmation prompt with --apply

Crash detection

A resume bitfield saved just before a crash can claim data that never reached disk; importing it under a trusting policy reproduces the failure described in CRASH_SAFETY.md. Before applying, torrentngd migrate checks the source's shutdown state:

Source Signal Result
rTorrent rtorrent.lock (host:+pid) in the session directory no lock: clean. Lock and a live rtorrent process: running, refused unless --allow-running-source. Lock but no such process (or another host, or unreadable): crashed
qBittorrent lockfile (Qt QLockFile: pid, app, host) in the state directory or up to two parents lock found: same rules as rTorrent. Not found: unknown, not clean
Deluge deluged.pid (pid;port) in the state directory or its parent same as qBittorrent
any other source, or a non-standard location --source-lock FILE a lock or pid file you name: found and stale is a crash, found and live is running, missing means the client exited cleanly
every other source without --source-lock none TorrentNG can read unknown; nothing is assumed

The qBittorrent and Deluge file names, formats and locations come from those clients' documented behavior and were not verified against the real programs; if yours keeps its lock elsewhere, pass it with --source-lock.

A lock is judged by its process: gone means the last run did not finish; alive and named like the client means it is running now; alive but named like an unrelated program means the pid was reused (crash); a scripting-language host such as python counts as possibly the client. Process checks use /proc on Linux, kill and ps on macOS and FreeBSD, and OpenProcess on Windows.

For a crashed source the policy becomes verify, so every imported torrent is hash-checked before it is trusted; the dry-run prints this, and --trust-unclean-source overrides it. For sources with no signal, after any crash of that client pass --policy verify yourself.

The dry-run report and the post-apply summary both break torrents down into trusted / hints / metadata-only / none so you can see how much state will avoid a recheck before committing. Always run the dry-run first. The TorrentNG-client DB and fast-resume directory are taken from the resolved config, so point --config at the same config torrentngd uses, and back up the TorrentNG-client DB (see BACKUP_RESTORE.md) before --apply.

Reading the confidence summary

The four buckets describe what was decoded at scan time, not the final recheck decision:

  • trusted — piece state decoded and every backing file matched the expected size on disk. Resumes with no recheck.
  • hints — piece state decoded, but not all files validated by size at scan time. This is the normal label for an in-progress download: the partially-written file can never size-match the full torrent length, so the torrent is reported as hints even though its completed-piece map was fully imported.
  • metadata-only — no decodable piece state (e.g. an incomplete rTorrent torrent, or Tixati). Save path, labels, and counters import; content is rechecked.
  • none — only .torrent metadata was importable.

A hints partial still skips the full recheck under the default trust-hints policy. The TorrentNG client's startup guard invalidates only pieces whose backing files changed since import — not pieces that are merely incomplete — so a half-finished qBittorrent/Deluge download keeps its completed pieces and only fetches the rest. The label is a scan-time artifact of size-only file validation, not a downgrade of the imported piece state. Only --policy verify forces a full rehash of decoded state; trust-all trusts it even if files are absent. For this guard to hold, import after the data is in its final location (use --remap); moving or re-copying files after --apply changes their size/mtime and reverts the affected pieces to verification.

Migrating from rTorrent + ruTorrent

What gets imported

  • rTorrent session directory (.torrent files and .rtorrent resume files)
  • Save paths from resume state
  • Labels/categories stored in d.custom1
  • Tracker lists
  • Upload/download stats from resume state
  • Completed/seeding state (no forced recheck for complete torrents)

What does not import automatically

  • ruTorrent per-plugin settings (RSS rules, ratio groups, labels UI state)
  • ruTorrent views and saved searches
  • Plugin-stored custom metadata beyond d.custom1–5

Import into the TorrentNG client (recommended)

For TorrentNG-client deployments, import the rTorrent session directly with torrentngd migrate. Stop rTorrent (or work from a copy of the session directory) so resume files are not mid-write, then:

# 1. Dry-run: read-only, shows trusted/hints/metadata-only/none counts
torrentngd migrate --source rtorrent \
  --from ~/.rtorrent-session \
  --report /tmp/rtorrent-migration.md

# 2. Back up the TorrentNG-client DB (see BACKUP_RESTORE.md), then apply
torrentngd migrate --source rtorrent \
  --from ~/.rtorrent-session \
  --remap /old/downloads=/data \
  --apply

Complete torrents whose data files are present at the expected sizes import as trusted and resume seeding with no full recheck. Incomplete downloads and any torrent whose files are missing or resized fall back to verification — rTorrent partial resume state is not decoded, so in-progress downloads will recheck. Use --remap if the download paths differ in the new deployment. The rTorrent session directory is read only; nothing is written to it.

Procedure (compatible rTorrent integration)

Use this path when keeping rTorrent as the transfer client (integration or comparison).

Step 1: Run the diagnostic first

./scripts/healthcheck.sh /path/to/rtorrent.sock

Fix any issues it reports before migrating.

Step 2: Back up everything

# rTorrent session
cp -r ~/.rtorrent-session ~/rtorrent-session.bak

# ruTorrent settings
cp -r /var/www/rutorrent/conf ~/rutorrent-conf.bak
cp -r /var/www/rutorrent/share ~/rutorrent-share.bak

Step 3: Note your current config

cat ~/.rtorrent.rc

Key values to record:

  • session.path — session directory
  • directory.default.set — default download path
  • network.port_range.set — listen port
  • scgi_local or scgi_port — RPC socket

Step 4: Deploy the Phase 1 bundle

docker compose -f deploy/docker/compose.phase1.yml up --build

Map your existing session and data directories to the container volumes:

# In your compose override:
volumes:
  - /your/existing/rtorrent-session:/session
  - /your/existing/downloads:/data
  - /your/existing/config:/config

Place any .rtorrent.rc overrides in /config/rtorrent.rc.

More deployment details are in DEPLOYMENT.md.

Step 5: Verify

After starting the new container:

  1. Check the ruTorrent torrent list — all torrents should appear in their previous state
  2. Verify no torrents have started unnecessary rechecks
  3. Check that tracker announces are working
  4. Confirm save paths are correct

Notes

  • Complete torrents that were seeding will resume seeding. They are not forced to recheck.
  • Partial downloads resume from where they left off (rTorrent resume state is preserved).
  • If a torrent shows "hash check" on startup, the resume file may be missing or corrupt — this is from rTorrent's own resume logic, not a migration artifact.

Migrating from qBittorrent

What gets imported

  • BT_backup/ directory (.torrent files and .fastresume files)
  • Categories
  • Tags
  • Save paths
  • Torrent states (paused/seeding/downloading)
  • Tracker lists

Procedure

Step 1: Locate qBittorrent data

Common locations:

  • Linux: ~/.local/share/data/qBittorrent/BT_backup/
  • Docker: inside the container at the above path

Step 2: Copy BT_backup to a staging directory

cp -r ~/.local/share/data/qBittorrent/BT_backup /tmp/qbt-migration

Step 3: Run dry-run scan and TorrentNG-client import

torrentngd migrate --source qbittorrent --from /tmp/qbt-migration --report /tmp/qbt.md
torrentngd migrate --source qbittorrent --from /tmp/qbt-migration --apply

This pairs .torrent files with .fastresume sidecars and imports save path, category, tags, trackers, completion state, file rows, transfer counters, and ratio. qBittorrent's libtorrent-style resume data carries piece state, so complete and partial torrents both import their progress and avoid a full recheck under the default trust-hints policy when the data files are present.

Manual fallback remains available: load each .torrent through the TorrentNG API, pointing at the existing file path, then let the TorrentNG-client recheck job verify and resume without downloading.

Migrating from Transmission

What gets imported

  • Torrent files from ~/.config/transmission/torrents/
  • Resume data from ~/.config/transmission/resume/
  • Download directories

Procedure

Run torrentngd migrate against the Transmission config directory (the parent of torrents/ and resume/):

torrentngd migrate --source transmission --from ~/.config/transmission --report /tmp/tr.md
torrentngd migrate --source transmission --from ~/.config/transmission --apply

Transmission progress bitfields are decoded, so completed and partial torrents avoid a full recheck when their data is present. Manual fallback remains:

  1. Export torrent files from Transmission (right-click → "Export .torrent")
  2. Add each via the TorrentNG API, pointing to the existing download path
  3. Run a TorrentNG-client recheck job so verified pieces resume without re-downloading

Migrating from Deluge, uTorrent, BitTorrent Classic, BiglyBT/Vuze, or Tixati

Pass the matching --source (deluge, utorrent, biglybt, or tixati) to torrentngd migrate with the client's state/config directory as --from. The scanner imports .torrent metadata, save paths, labels/categories, transfer counters, lifecycle timestamps, active/paused state, per-file wanted/priority state, per-file completed bytes, trackers, generic tracker timing/scrape state, and any compatible fast-resume piece state it can decode.

Current fast-resume confidence:

  • rTorrent: complete .rtorrent sidecars can synthesize seed piece state when matching files are present; incomplete or unsupported sidecars fall back to verification.
  • Deluge: libtorrent-style resume data is imported like qBittorrent.
  • uTorrent/BitTorrent classic: aggregate resume.dat entries keyed by raw, hex, or base32 info-hash are imported. The per-torrent have/bitfield piece map is decoded, so completed pieces resume without a full recheck under the default trust-hints policy — the same as qBittorrent/Deluge.
  • BiglyBT/Vuze: aggregate downloads.config entries keyed by info-hash are imported, including nested resume bitfields when present.
  • Tixati: .torrent metadata, save paths, labels, and counters import, but progress state is an undocumented proprietary binary format and is verification-first by design — Tixati torrents are rechecked rather than risk trusting guessed piece state. This is a deliberate safety choice; lifting it needs a real Tixati state corpus and a strict decoder.

Use dry-run output before applying any import. The source directories are read only. Dry-run reports include trusted, hints, metadata-only, and none counts so operators can see how much state will avoid a full recheck. Low-confidence or unsupported resume data is downgraded to normal verification rather than trusted as complete. Piece-state length mismatches are reported and normalized to the torrent piece count before TorrentNG-client fast-resume state is written.

Path remapping

If your download directories are in different locations in the new setup, you need to remap paths before adding torrents.

# Example: old path /downloads, new path /data
# Set in rtorrent.rc or via API after adding:
# d.directory.set=<new_path>/<torrent_name>

The TorrentNG client and compatible-client service both support per-torrent save path updates. Bulk moves should use the TorrentNG storage planning flow so conflicts, capacity, copy/rename mode, rollback, and destructive delete approval are visible before data moves.

Pass remaps to torrentngd migrate with repeatable --remap OLD=NEW flags (e.g. --remap /downloads=/data). Each remap is applied both to file-hint validation — so trusted fast-resume state is still recognized after moving data into a container — and to the TorrentNG-client DB save path written during import. The longest matching prefix wins when multiple remaps apply.

torrentngd migrate --apply is the combined TorrentNG-client import path: DB rows and compatible fast-resume state are written together under one audited summary.

Leaving TorrentNG (torrentngd export)

Reverse migration is a first-class, no-lock-in path. torrentngd export reads TorrentNG-client state read-only (the session DB, persisted .torrent blobs, and fast-resume state) and writes another client's on-disk layout so you can resume seeding elsewhere — without a full recheck wherever the target format can carry piece state.

torrentngd export --format <FMT> --to <DIR> [OPTIONS]
  • --format — generic, libtorrent (qBittorrent/Deluge), transmission, rtorrent, utorrent, or biglybt
  • --to — output directory for the exported layout
  • --apply — write the export; omit for a dry-run report (the default)
  • --report FILE — also write the markdown dry-run report
  • --config FILE — config file (else TORRENTNGD_CONFIG / defaults); this locates the TorrentNG-client DB, .torrent blobs, and fast-resume directory
  • --yes — skip the confirmation prompt with --apply

The TorrentNG-client state is never modified. The dry-run report and post-apply summary break torrents into recheck-free / complete-only / metadata-only / torrent-only so you can see how much seeding state survives the move:

Target Layout written Fidelity
libtorrent <hash>.torrent + <hash>.fastresume (qBittorrent BT_backup / Deluge state dir) Recheck-free, partials included
transmission torrents/ + resume/ Recheck-free, partials included
rtorrent <hash>.torrent + <hash>.rtorrent session resume file Recheck-free for complete torrents; partials metadata-only (rTorrent's format has no decodable partial bitfield)
utorrent aggregate resume.dat Recheck-free at whole-piece granularity
biglybt aggregate downloads.config Recheck-free at whole-piece granularity
generic <hash>.torrent files + manifest.json Always correct; destination rechecks

Example — leave for qBittorrent with seeding state intact:

torrentngd export --format libtorrent --to /tmp/leaving --report /tmp/exit.md
# review the fidelity summary, then:
torrentngd export --format libtorrent --to /tmp/leaving --apply

Then point qBittorrent/Deluge at the exported directory (qBittorrent: copy into BT_backup/). generic is the universal safety valve: it always produces usable .torrent files plus a manifest of save paths, categories, tags, counters, and completed-piece counts.

Torrents imported via torrentngd migrate --apply persist their .torrent blob into the session directory, so a migrate-in then export-out round trip works without first starting torrentngd. A torrent with no TorrentNG-client fast-resume state exports as torrent-only (the destination rechecks) — that is honest, not a failure.

Rolling back

If migration causes problems:

  1. Stop the new container.
  2. Restore the TorrentNG-client DB/session backup created before import.
  3. Keep the original source client session untouched and restart it if needed.
  4. Re-run the dry-run report after correcting path/category/tag remaps.

No migration step modifies the session directory in-place. The import is read-only from the source. See BACKUP_RESTORE.md for TorrentNG-client DB backup and restore commands.