Skip to content

feat(plugins): #1247 standalone markdown plugin #1457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 35 commits into from
Sep 6, 2025
Merged

feat(plugins): #1247 standalone markdown plugin #1457

merged 35 commits into from
Sep 6, 2025
+2,416 −802

Conversation

thescientist13
Copy link
Member

@thescientist13 thescientist13 commented Mar 19, 2025 •
edited
Loading

Related Issue

resolves #1247

Documentation

  1. Needs documentation for the following - document new (standalone) markdown plugin www.greenwoodjs.dev#229
    • move markdown docs to plugins page (w/ Netlify redirect) (BREAKING)
    • delete markdown configuration docs (BREAKING)
    • add preServe lifecycle docs
    • tocHeadings - https://greenwoodjs.dev/docs/content-as-data/pages-data/#table-of-contents
    • "breaks" the getting started guide, will need to have the markdown plugin added as part of the guide
    • serve lifecycle resource plugin recommendations (set statusText and status)
    • update serve pages docs to reflect expectation of needing extensions (BREAKING)

Summary of Changes

  1. Created standalone unified based markdown plugin (BREAKING)
  2. Moved over all relevant markdown deps from the CLI to the new plugin
  3. Converted GraphQL plugin to use Content as data APIs, but now requires usage of activeContent flag (BREAKING)
  4. Refactored all CLI test cases to use HTML, moved relevant markdown test cases to new markdown plugin
  5. Refined how servePage option works for resource plugins; now relies on extension (BREAKING)
  6. Support "standalone" markdown processing usage

TODO

  1. how to support GraphQL and ToC data?
    • can it just resolve to the content server instead? - yes!
    • for now, will disable the unit tests though
  2. Confirm live reload of markdown content
  3. refactor Greenwood test cases using markdown
  4. Port Greenwood markdown test cases to new markdown package
    • frontmatter imports
    • layouts and pages
    • theme pack test case / slides
    • <x-counter></x-counter> (Demo Page Example)
    • getting started
  5. .mp3 test case is failing all GitHub Actions 🤔
  6. flaky windows test cases around Response body consumption - feat(plugins): #1247 standalone markdown plugin #1457 (comment)
  7. static router issue with layouts / HTML page (see xdescribe since we have a hardcoding on .html per inconsistent HTML output merging hierarchy for top level pages and layouts #1475 ) and create a test case in markdown plugin suite
  8. duplicated <body> tags test cases popping up (see xdescribe for Custom Sources Plugin and Custom Layout)
  9. custom dynamic loaders test case not working (Custom Static and Dynamic Page Loaders)
  10. Migrate custom elements wrapper logic
  11. better way to check for initContents instead of all the negate cases (in standard html plugin)
  12. do we even need the frontmatter plugin in the markdown processing section? - get pages frontmatter from remark plugin instead of additional frontmatter package #531
  13. integrate shouldPreServe / preServe in all places (e.g. loader.js), and handling response in serve all serve lifecyles?
  14. Custom plugin options - rebase against - Support passing options to markdown plugins #1479
    • update configuration and types (including test greenwood config)
  15. with pre lifecycles, do we even need the distinction of greenwood vs non-greenwood plugins at the top of serve.js?
  16. ~~avoid async reduce in preServe, per enhancement/issue 823 ensure greenwood cli and plugins are favoring idiomatic usages of async #1546 ~~
  17. String constructor - https://github.com/thescientist13/greenwood-native-typescript/blob/unified-in-plugins/external-content-source-plugin.ts#L38
  18. apply publint recommendation - chore/update package json repository field with valid workspace values #1494
  19. I think we can rollback all the plugin changes now that we've abandoned the perServe approach and any other tweaks we've made in the serve lifecycle
  20. Plugin Documentation
    • README
    • options (plugins, table of content headings)

Thoughts / Questions

  1. this is a breaking change to our getting started guide
  2. Have we introduced a change to serve lifecycle signature? would need to update our plugin types / docs
  3. This might impact our theme pack starter and so plugin-markdown would now need to be a peer dependency?
  4. does this help with some hot reloading of content as data? - it could, but not a focus right now support hot reloading content as data changes during development #1278
  5. Updates for Content as Data v2 #1286 in regards to pushing to the graph from a plugin? - N / A
  6. (nice to have) - standalone usage - standalone unifiedjs based markdown plugin #1247 (comment)

@thescientist13 thescientist13 added documentation Greenwood specific docs Plugins Greenwood Plugins CLI breaking Types(cript) Requires type definition or TypeScript related work / documentation labels Mar 19, 2025
@thescientist13 thescientist13 self-assigned this Mar 19, 2025
@thescientist13 thescientist13 force-pushed the feature/issue-1247-standalone-markdown-plugin branch from 9b41d02 to 26ae467 Compare April 27, 2025 21:17
@thescientist13 thescientist13 added the feature New feature or request label Apr 27, 2025
@thescientist13 thescientist13 force-pushed the feature/issue-1247-standalone-markdown-plugin branch from a31971f to ddb8721 Compare August 17, 2025 20:11
@thescientist13
Copy link
Member Author

thescientist13 commented Aug 18, 2025 •
edited
Loading

Hmm, not sure why the .mp3 develop command test case is failing all builds?
ex: https://github.com/ProjectEvergreen/greenwood/actions/runs/17025736392/job/48260854107

2875 passing (8m)
  214 pending
  4 failing

  1) Develop Greenwood With: 
       Default Greenwood Configuration and Workspace
         Develop command with audio format (.mp3) behavior
           should return a 200 status:

      AssertionError: expected 500 to equal 200
      + expected - actual

      -500
      +200
      
      at Context.<anonymous> (file:///home/runner/work/greenwood/greenwood/packages/cli/test/cases/develop.default/develop.default.spec.js:479:36)
      at process.processImmediate (node:internal/timers:485:21)

  2) Develop Greenwood With: 
       Default Greenwood Configuration and Workspace
         Develop command with audio format (.mp3) behavior
           should return the correct content type:

      AssertionError: expected 'text/plain; charset=utf-8' to equal 'audio/mpeg'
      + expected - actual

      -text/plain; charset=utf-8
      +audio/mpeg
      
      at Context.<anonymous> (file:///home/runner/work/greenwood/greenwood/packages/cli/test/cases/develop.default/develop.default.spec.js:484:57)
      at process.processImmediate (node:internal/timers:485:21)

  3) Develop Greenwood With: 
       Default Greenwood Configuration and Workspace
         Develop command with audio format (.mp3) behavior
           should return the correct content length:

      AssertionError: expected '0' to equal '5425061'
      + expected - actual

      -0
      +5425061
      
      at Context.<anonymous> (file:///home/runner/work/greenwood/greenwood/packages/cli/test/cases/develop.default/develop.default.spec.js:489:59)
      at process.processImmediate (node:internal/timers:485:21)

  4) Develop Greenwood With: 
       Default Greenwood Configuration and Workspace
         Develop command with audio format (.mp3) behavior
           should return the correct response body:
     AssertionError: expected '' to include 'ID3'
      at Context.<anonymous> (file:///home/runner/work/greenwood/greenwood/packages/cli/test/cases/develop.default/develop.default.spec.js:494:25)
      at process.processImmediate (node:internal/timers:485:21)

With logging added, we get to the root cause! 😮‍💨
https://github.com/ProjectEvergreen/greenwood/actions/runs/17027496341/job/48264721653

TypeError: Response.clone: Body has already been consumed.
    at webidl.errors.exception (node:internal/deps/undici/undici:3610:14)
    at _Response.clone (node:internal/deps/undici/undici:9192:31)
    at file:///home/runner/work/greenwood/greenwood/packages/cli/src/lifecycles/serve.js:122:49
    at async file:///home/runner/work/greenwood/greenwood/packages/cli/src/lifecycles/serve.js:103:5
    at async file:///home/runner/work/greenwood/greenwood/packages/cli/src/lifecycles/serve.js:58:5

@thescientist13 thescientist13 changed the title feature/issue 1247 standalone markdown plugin feat(plugins): issue 1247 standalone markdown plugin Aug 24, 2025
@thescientist13 thescientist13 force-pushed the feature/issue-1247-standalone-markdown-plugin branch from 9771fbf to f030ee6 Compare August 26, 2025 02:03
@thescientist13 thescientist13 marked this pull request as ready for review August 26, 2025 02:30
@thescientist13
Copy link
Member Author

thescientist13 commented Aug 30, 2025 •
edited
Loading

Getting a bunch of intermittent errors on Windows to due Response bodies being consumed in the dev server 😞

****************************************************************************
TypeError: Response.clone: Body has already been consumed.
    at webidl.errors.exception (node:internal/deps/undici/undici:3610:14)
    at _Response.clone (node:internal/deps/undici/undici:9192:31)
    at file:///C:/Users/owenbuckley/Workspace/greenwood/packages/cli/src/lifecycles/serve.js:70:45
    at async file:///C:/Users/owenbuckley/Workspace/greenwood/packages/cli/src/lifecycles/serve.js:49:5

edit: it looks like it can even happen from time to time on my mac 😬

maybe this has been part of the random flakiness we've seen from time to time in GitHub Actions

@thescientist13 thescientist13 force-pushed the feature/issue-1247-standalone-markdown-plugin branch from d86e18d to a880798 Compare August 30, 2025 20:55
@thescientist13 thescientist13 changed the title feat(plugins): issue 1247 standalone markdown plugin feat(plugins): #1247 standalone markdown plugin Aug 31, 2025
@thescientist13 thescientist13 mentioned this pull request Aug 31, 2025
Closed
5 tasks
@thescientist13 thescientist13 linked an issue Aug 31, 2025 that may be closed by this pull request
get pages frontmatter from remark plugin instead of additional frontmatter package #531
Closed
5 tasks
This was referenced Aug 31, 2025
Draft
Open
@KaiPrince KaiPrince mentioned this pull request Sep 1, 2025
Closed
This was referenced Sep 1, 2025
Open
Closed
@thescientist13
migrate markdown 404 page test case to markdown plugin
bff0645
@thescientist13
misc rebase sync changes
6fa5002
@thescientist13
refactoring markdown out of all CLI test cases
ff8564b
@thescientist13
refine 404 handling
8d21b0c
@thescientist13
add context plugin test cases to markdown plugin
119b391
@thescientist13
refine production server response statusText
22c2307
@thescientist13
handle markdown and custom elements p tag wrapping
98cdb3e
@thescientist13
enable test logging for failing test case
d8dc871
@thescientist13
refactor preServe transforming in development server
127b3e4
@thescientist13
clean up resource plugin filtering in serve lifecycle
a19c42b
@thescientist13
one more attempt at dev server test case failures
1e220f1
@thescientist13
refactor away from preServe lifecycle
9f11d4b
@thescientist13
cleanup
c5c49fe
@thescientist13
more preServe cleanup
a2842dc
@thescientist13
add clone
00ad749
@thescientist13
bump alpha version
dd8d956
@thescientist13
update lock file
8186ef0
@thescientist13
rollback preServe related changes
f0ba508
@thescientist13
revert back to async reduce pattern in serve lifecycle for dev server…
c053f1a 
… serve and intercept related resource plugin processing
@thescientist13
restore commander
feb3df5
@thescientist13
fix static router test cases
4c1bb57
@thescientist13
restore more test cases
8fe7348
@thescientist13
refactor servePage option for custom page formats
7af354b
@thescientist13
remove frontmatter dependency
e986403
@thescientist13
document markdown plugin options
95a9c7b
@thescientist13
restore frontmatter plugin
eb5a27b
@thescientist13
standalone markdown processing function
c9abc33
@thescientist13 thescientist13 force-pushed the feature/issue-1247-standalone-markdown-plugin branch from 89adcea to c9abc33 Compare September 4, 2025 01:33
@thescientist13 thescientist13 merged commit 8e9286b into master Sep 6, 2025
10 checks passed
@thescientist13 thescientist13 deleted the feature/issue-1247-standalone-markdown-plugin branch September 6, 2025 00:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@thescientist13 thescientist13

Labels
breaking CLI Content as Data documentation Greenwood specific docs feature New feature or request Plugins Greenwood Plugins Types(cript) Requires type definition or TypeScript related work / documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

standalone unifiedjs based markdown plugin get pages frontmatter from remark plugin instead of additional frontmatter package
1 participant
@thescientist13