Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .agents/plugins/marketplace.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"name": "studio",
"interface": {
"displayName": "studio"
},
"plugins": [
{
"name": "data-liberation",
"source": {
"source": "local",
"path": "./packages/data-liberation-agent"
},
"policy": {
"installation": "AVAILABLE",
"authentication": "ON_INSTALL"
},
"category": "Coding"
}
]
}
1 change: 1 addition & 0 deletions .agents/skills/codex.marketplace.json
1 change: 1 addition & 0 deletions .agents/skills/marketplace.json
13 changes: 13 additions & 0 deletions .buildkite/commands/run-lint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,16 @@ npm run lint

echo '--- :typescript: Typecheck'
npm run typecheck

# The data-liberation plugin ships its MCP server as a committed esbuild
# bundle (the plugin installer copies the package from git verbatim — no
# npm install, no build). Rebuild it and fail if the committed artifact
# doesn't match src/, so the two can never drift. The build is
# byte-deterministic for a given lockfile, so a clean tree means fresh.
echo '--- :package: Verify data-liberation MCP bundle is fresh'
npm -w data-liberation run build:mcp-bundle
if ! git diff --exit-code -- packages/data-liberation-agent/dist/mcp-server.bundle.mjs; then
echo "^^^ +++"
echo "The committed data-liberation MCP bundle is stale. Run 'npm -w data-liberation run build:mcp-bundle' and commit the updated bundle."
exit 1
fi
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
{
"name": "data-liberation",
"name": "studio",
"owner": {
"name": "Automattic"
},
"plugins": [
{
"name": "data-liberation",
"source": "./",
"source": "./packages/data-liberation-agent",
"description": "Extract content from closed web platforms (GoDaddy Websites & Marketing, Hostinger, HubSpot, Shopify, Squarespace, Webflow, Weebly, Wix) into WordPress-compatible WXR files. Inspect, extract, QA, and import to WordPress."
}
]
Expand Down
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,10 @@ cli/vendor/
dist
dist-hosted
dist-local
# …except the data-liberation plugin's committed MCP server bundle (the plugin
# installer copies the package without running npm install or a build). The
# package-level .gitignore keeps everything else in its dist/ ignored.
!packages/data-liberation-agent/dist

# Playwright traces
test-results
Expand Down
1 change: 1 addition & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 4 additions & 7 deletions packages/data-liberation-agent/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
{
"name": "data-liberation",
"description": "Extract content from closed web platforms (GoDaddy Websites & Marketing, Hostinger, HubSpot, Shopify, Squarespace, Webflow, Weebly, Wix) into WordPress-compatible WXR files. Inspect, extract, QA, and import to WordPress.",
"version": "0.2.2",
"version": "1.0.0",
"author": {
"name": "Automattic"
},
"homepage": "https://github.com/Automattic/data-liberation-agent",
"repository": "https://github.com/Automattic/data-liberation-agent",
"homepage": "https://github.com/Automattic/studio",
"repository": "https://github.com/Automattic/studio",
"license": "GPL-2.0-or-later",
"keywords": [
"content-extraction",
Expand All @@ -23,8 +23,5 @@
"wordpress",
"wxr"
],
"mcp": {
"command": "npx",
"args": ["tsx", "src/mcp-server.ts"]
}
"mcpServers": "./.mcp.json"
}
6 changes: 3 additions & 3 deletions packages/data-liberation-agent/.codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
{
"name": "data-liberation",
"description": "Extract content from closed web platforms (GoDaddy Websites & Marketing, Hostinger, HubSpot, Shopify, Squarespace, Webflow, Weebly, Wix) into WordPress-compatible WXR files. Inspect, extract, QA, and import to WordPress.",
"version": "0.2.2",
"version": "1.0.0",
"author": {
"name": "Automattic"
},
"homepage": "https://github.com/Automattic/data-liberation-agent",
"homepage": "https://github.com/Automattic/studio",
"license": "GPL-2.0-or-later",
"skills": "./skills/",
"mcpServers": "./.mcp.json",
Expand All @@ -20,7 +20,7 @@
"Read",
"Write"
],
"websiteURL": "https://github.com/Automattic/data-liberation-agent",
"websiteURL": "https://github.com/Automattic/studio",
"defaultPrompt": [
"Inspect a site for liberation",
"Extract this site into WordPress-compatible WXR",
Expand Down
8 changes: 7 additions & 1 deletion packages/data-liberation-agent/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@ skills/replicate-workspace/
node_modules/
.DS_Store
*.log
dist/
# dist/ is build output, EXCEPT the committed plugin bundle: the Claude/Codex
# plugin installer copies this package verbatim (no npm install, no build), so
# the self-contained MCP server bundle must live in git.
dist/*
!dist/mcp-server.bundle.mjs
docs/plans/
docs/superpowers/
qa-screenshots/
Expand All @@ -15,6 +19,8 @@ qa-screenshots/
.worktrees/
.tmp-test/
TODOS.md
# Local discovery log — see CONTRIBUTING.md; intentionally not committed
DISCOVERIES.md

# Streaming compose-then-install scratch dir
.tmp-compose/
Expand Down
6 changes: 3 additions & 3 deletions packages/data-liberation-agent/.mcp.json
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
{
"mcpServers": {
"data-liberation": {
"command": "npx",
"args": ["tsx", "${CLAUDE_PLUGIN_ROOT:-.}/src/mcp-server.ts"],
"cwd": "${CLAUDE_PLUGIN_ROOT:-.}"
"command": "node",
"args": ["${CLAUDE_PLUGIN_ROOT}/scripts/mcp-launcher.mjs"],
"cwd": "${CLAUDE_PLUGIN_ROOT}"
}
}
}
1 change: 1 addition & 0 deletions packages/data-liberation-agent/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ When `adminToken` is present, `shopifyAdapter.extract` fetches products via the
## Non-obvious Details

- **MCP server runs `tsx src/mcp-server.ts` as ONE long-lived process — editing `src/` does NOT hot-reload it.** ESM modules are cached per process, so a re-called `liberate_*` tool keeps running the code that was loaded at server start. After editing handler/lib source, either restart the MCP server (so all callers pick it up) or, for a quick one-off re-run, drive the handler from a fresh `tsx` process (import the handler, pass a minimal `HandlerContext` with `textResult`/`errorResult`). Unit tests (`vitest`) always use the on-disk source, so they reflect edits immediately — the staleness is only the running MCP server.
- **`.mcp.json` starts the MCP server through `scripts/mcp-launcher.mjs`: dev checkouts run `src/` via tsx; plugin installs run the COMMITTED bundle `dist/mcp-server.bundle.mjs`.** The launcher only uses the bundle when the package's dependencies don't resolve, so a dev environment never runs a stale dist. The bundle must be committed because plugin installs are plain git copies: Claude's installer restores npm deps only when a per-package `package-lock.json` exists (this workspace package has none), and Codex's installer NEVER restores deps — so `claude/codex plugin marketplace add Automattic/studio` only works with a prebuilt self-contained artifact in the tree. After changing `src/` or dependencies, regenerate with `npm run build:mcp-bundle` and commit the result — CI (Buildkite Lint) rebuilds it and fails on drift. `playwright` and `single-file-cli` stay external to the bundle (guarded dynamic imports that degrade with install guidance). See `scripts/build-mcp-bundle.mjs` for the import.meta.url asset-resolution seam.
- DLA consumes `@automattic/blocks-engine` through the local pre-publish override `"file:../blocks-engine/packages/blocks-engine"`; standalone `npm install` therefore needs the sibling `blocks-engine` checkout until the package is published. On publish, flip the dependency to the authorized semver range (for example `"^0.x.y"`). Publishing itself remains gated on explicit user authorization.
- **`tsx scripts/*.ts` that call `page.evaluate` must create the page via `newShimmedPage` from `scripts/_pw.ts`** — tsx's esbuild keepNames rewrites named functions to `__name(...)`, which is undefined in the browser context Playwright serializes into (`ReferenceError: __name is not defined`). The shim installs a `__name` identity via `addInitScript`. vitest-run code is unaffected (its transform doesn't keepNames).
- WXR builder targets WXR 1.2 spec compliance
Expand Down
2 changes: 1 addition & 1 deletion packages/data-liberation-agent/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ If you can't test against a real site, note that in the PR and explain what you

### 3. Add to DISCOVERIES.md

This is required. Add a new entry at the top of [DISCOVERIES.md](./DISCOVERIES.md) in this format:
This is required. DISCOVERIES.md is a local, untracked log (gitignored — it is not committed to the repo); create it at the package root if it doesn't exist yet. Add a new entry at the top in this format:

```markdown
## YYYY-MM-DD — Short title
Expand Down
Loading
Loading