diff --git a/README.md b/README.md index 85e308bc3..df43ce7f6 100644 --- a/README.md +++ b/README.md @@ -67,12 +67,13 @@ These are plain English instructions, not code. Changes take effect on the next ## Default Sources -### Podcasts (5) +### Podcasts (6) - [Latent Space](https://www.youtube.com/@LatentSpacePod) - [Training Data](https://www.youtube.com/playlist?list=PLOhHNjZItNnMm5tdW61JpnyxeYH5NDDx8) - [No Priors](https://www.youtube.com/@NoPriorsPodcast) - [Unsupervised Learning](https://www.youtube.com/@RedpointAI) - [Data Driven NYC](https://www.youtube.com/@DataDrivenNYC) +- [AI & I by Every](https://www.youtube.com/playlist?list=PLuMcoKK9mKgHtW_o9h5sGO2vXrffKHwJL) ### AI Builders on X (25) [Andrej Karpathy](https://x.com/karpathy), [Swyx](https://x.com/swyx), [Josh Woodward](https://x.com/joshwoodward), [Kevin Weil](https://x.com/kevinweil), [Peter Yang](https://x.com/petergyang), [Nan Yu](https://x.com/thenanyu), [Madhu Guru](https://x.com/realmadhuguru), [Amanda Askell](https://x.com/AmandaAskell), [Cat Wu](https://x.com/_catwu), [Thariq](https://x.com/trq212), [Google Labs](https://x.com/GoogleLabs), [Amjad Masad](https://x.com/amasad), [Guillermo Rauch](https://x.com/rauchg), [Alex Albert](https://x.com/alexalbert__), [Aaron Levie](https://x.com/levie), [Ryo Lu](https://x.com/ryolu_), [Garry Tan](https://x.com/garrytan), [Matt Turck](https://x.com/mattturck), [Zara Zhang](https://x.com/zarazhangrui), [Nikunj Kothari](https://x.com/nikunj), [Peter Steinberger](https://x.com/steipete), [Dan Shipper](https://x.com/danshipper), [Aditya Agarwal](https://x.com/adityaag), [Sam Altman](https://x.com/sama), [Claude](https://x.com/claudeai) diff --git a/README.zh-CN.md b/README.zh-CN.md index a4206f00a..cf9fdd481 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -60,12 +60,13 @@ Skill 使用纯文本 prompt 文件来控制内容的摘要方式。你可以通 ## 默认信息源 -### 播客（5个） +### 播客（6个） - [Latent Space](https://www.youtube.com/@LatentSpacePod) - [Training Data](https://www.youtube.com/playlist?list=PLOhHNjZItNnMm5tdW61JpnyxeYH5NDDx8) - [No Priors](https://www.youtube.com/@NoPriorsPodcast) - [Unsupervised Learning](https://www.youtube.com/@RedpointAI) - [Data Driven NYC](https://www.youtube.com/@DataDrivenNYC) +- [AI & I by Every](https://www.youtube.com/playlist?list=PLuMcoKK9mKgHtW_o9h5sGO2vXrffKHwJL) ### X 上的 AI 建造者（25位） [Andrej Karpathy](https://x.com/karpathy), [Swyx](https://x.com/swyx), [Josh Woodward](https://x.com/joshwoodward), [Kevin Weil](https://x.com/kevinweil), [Peter Yang](https://x.com/petergyang), [Nan Yu](https://x.com/thenanyu), [Madhu Guru](https://x.com/realmadhuguru), [Amanda Askell](https://x.com/AmandaAskell), [Cat Wu](https://x.com/_catwu), [Thariq](https://x.com/trq212), [Google Labs](https://x.com/GoogleLabs), [Amjad Masad](https://x.com/amasad), [Guillermo Rauch](https://x.com/rauchg), [Alex Albert](https://x.com/alexalbert__), [Aaron Levie](https://x.com/levie), [Ryo Lu](https://x.com/ryolu_), [Garry Tan](https://x.com/garrytan), [Matt Turck](https://x.com/mattturck), [Zara Zhang](https://x.com/zarazhangrui), [Nikunj Kothari](https://x.com/nikunj), [Peter Steinberger](https://x.com/steipete), [Dan Shipper](https://x.com/danshipper), [Aditya Agarwal](https://x.com/adityaag), [Sam Altman](https://x.com/sama), [Claude](https://x.com/claudeai) diff --git a/SKILL.md b/SKILL.md index d7a72d8fb..33695f48e 100644 --- a/SKILL.md +++ b/SKILL.md @@ -334,7 +334,7 @@ internet connection. Otherwise, use whatever content is in the JSON. ### Step 3: Check for content -If `stats.podcastEpisodes` is 0 AND `stats.xBuilders` is 0, tell the user: +If `stats.podcastEpisodes` is 0 AND `stats.xBuilders` is 0 AND `stats.blogPosts` is 0, tell the user: "No new updates from your builders today. Check back tomorrow!" Then stop. ### Step 4: Remix content @@ -346,6 +346,7 @@ Read the prompts from the `prompts` field in the JSON: - `prompts.digest_intro` — overall framing rules - `prompts.summarize_podcast` — how to remix podcast transcripts - `prompts.summarize_tweets` — how to remix tweets +- `prompts.summarize_blogs` — how to remix blog posts - `prompts.translate` — how to translate to Chinese **Tweets (process first):** The `x` array has builders with tweets. Process one at a time: diff --git a/examples/sample-digest.md b/examples/sample-digest.md index 9d7bb5981..63bcf04b9 100644 --- a/examples/sample-digest.md +++ b/examples/sample-digest.md @@ -6,6 +6,40 @@ This is an example of what your AI Builders Digest looks like. AI Builders Digest — March 14, 2026 +X / TWITTER + +Andrej Karpathy (karpathy on X) +Shared a deep thread on why he thinks "Software 3.0" (natural language programming) +will make traditional coding a niche skill within 5 years. Key argument: the compile +target is changing from machine code to LLM prompts. Sparked massive debate. +Also released a new Eureka Labs tutorial on building a code interpreter from scratch. +https://x.com/karpathy/status/example1 +https://x.com/karpathy/status/example2 + +Guillermo Rauch (rauchg on X) +Announced Vercel's new "v0 Teams" — collaborative AI prototyping where multiple +people can prompt and iterate on the same UI simultaneously. Called it "Google Docs +for vibe coding." Ships next week. +https://x.com/rauchg/status/example3 + +Amanda Askell (AmandaAskell on X) +Published a nuanced take on AI safety benchmarks: "We're measuring what's easy to +measure, not what matters. Capability evals tell you what the model CAN do. +Alignment evals should tell you what it WILL do unprompted." Linked to a new +Anthropic research paper on behavioral evaluations. +https://x.com/AmandaAskell/status/example4 + + +OFFICIAL BLOGS + +Anthropic Engineering: "Building Reliable AI Agents with Tool Use" +The Anthropic team shares lessons from building production agent systems. Key finding: +tool selection accuracy drops sharply when agents have more than 15 tools. The fix +is dynamic tool curation per task, not smarter models. Includes benchmark data and +a reference architecture for multi-step tool chains. +https://www.anthropic.com/engineering/reliable-ai-agents + + PODCASTS Latent Space — "Why Agents Keep Failing (And How to Fix Them)" @@ -33,28 +67,4 @@ Key insights: - He's surprisingly bullish on open-source: "The gap will narrow to months, not years." https://youtube.com/watch?v=example456 - -X / TWITTER - -Andrej Karpathy (@karpathy) -Shared a deep thread on why he thinks "Software 3.0" (natural language programming) -will make traditional coding a niche skill within 5 years. Key argument: the compile -target is changing from machine code to LLM prompts. Sparked massive debate. -Also released a new Eureka Labs tutorial on building a code interpreter from scratch. -https://x.com/karpathy/status/example1 -https://x.com/karpathy/status/example2 - -Guillermo Rauch (@rauchg) -Announced Vercel's new "v0 Teams" — collaborative AI prototyping where multiple -people can prompt and iterate on the same UI simultaneously. Called it "Google Docs -for vibe coding." Ships next week. -https://x.com/rauchg/status/example3 - -Amanda Askell (@AmandaAskell) -Published a nuanced take on AI safety benchmarks: "We're measuring what's easy to -measure, not what matters. Capability evals tell you what the model CAN do. -Alignment evals should tell you what it WILL do unprompted." Linked to a new -Anthropic research paper on behavioral evaluations. -https://x.com/AmandaAskell/status/example4 - -Reply to adjust your settings, sources, or summary style. +Generated through the Follow Builders skill: https://github.com/zarazhangrui/follow-builders diff --git a/scripts/package.json b/scripts/package.json index c8513db65..1cc0e4436 100644 --- a/scripts/package.json +++ b/scripts/package.json @@ -5,7 +5,8 @@ "type": "module", "scripts": { "generate-feed": "node generate-feed.js", - "prepare-digest": "node prepare-digest.js" + "prepare-digest": "node prepare-digest.js", + "test": "node test-docs-consistency.js" }, "dependencies": { "dotenv": "^16.4.0", diff --git a/scripts/test-docs-consistency.js b/scripts/test-docs-consistency.js new file mode 100644 index 000000000..92eeefbd9 --- /dev/null +++ b/scripts/test-docs-consistency.js @@ -0,0 +1,189 @@ +#!/usr/bin/env node + +// ============================================================================ +// Documentation Consistency Tests +// ============================================================================ +// Verifies that READMEs, SKILL.md, prompts, and sample digest stay in sync +// with the actual source config and each other. +// +// Usage: node test-docs-consistency.js +// Exit code: 0 = pass, 1 = failures found +// ============================================================================ + +import { readFile } from 'fs/promises'; +import { join } from 'path'; + +const SCRIPT_DIR = decodeURIComponent(new URL('.', import.meta.url).pathname); +const ROOT = join(SCRIPT_DIR, '..'); + +let failures = 0; + +function pass(name) { + console.log(` ✓ ${name}`); +} + +function fail(name, detail) { + console.error(` ✗ ${name}`); + console.error(` ${detail}`); + failures++; +} + +// -- Helpers ----------------------------------------------------------------- + +async function readText(relPath) { + return readFile(join(ROOT, relPath), 'utf-8'); +} + +async function readJSON(relPath) { + return JSON.parse(await readFile(join(ROOT, relPath), 'utf-8')); +} + +// -- Tests ------------------------------------------------------------------- + +async function testPodcastCountInReadmes() { + const sources = await readJSON('config/default-sources.json'); + const actualCount = sources.podcasts.length; + + const readmeEn = await readText('README.md'); + const readmeZh = await readText('README.zh-CN.md'); + + // English README + const enMatch = readmeEn.match(/### Podcasts \((\d+)\)/); + if (!enMatch) { + fail('README.md podcast count', 'Could not find "### Podcasts (N)" heading'); + } else if (parseInt(enMatch[1]) !== actualCount) { + fail('README.md podcast count', `Says ${enMatch[1]} but default-sources.json has ${actualCount}`); + } else { + pass('README.md podcast count matches default-sources.json'); + } + + // Chinese README + const zhMatch = readmeZh.match(/### 播客（(\d+)个）/); + if (!zhMatch) { + fail('README.zh-CN.md podcast count', 'Could not find "### 播客（N个）" heading'); + } else if (parseInt(zhMatch[1]) !== actualCount) { + fail('README.zh-CN.md podcast count', `Says ${zhMatch[1]} but default-sources.json has ${actualCount}`); + } else { + pass('README.zh-CN.md podcast count matches default-sources.json'); + } + + // Verify each podcast name appears in both READMEs + for (const podcast of sources.podcasts) { + if (!readmeEn.includes(podcast.name)) { + fail(`README.md missing podcast`, `"${podcast.name}" not listed`); + } + if (!readmeZh.includes(podcast.name)) { + fail(`README.zh-CN.md missing podcast`, `"${podcast.name}" not listed`); + } + } +} + +async function testSkillMdBlogPromptReference() { + const skill = await readText('SKILL.md'); + + // SKILL.md must reference summarize_blogs prompt + if (skill.includes('prompts.summarize_blogs')) { + pass('SKILL.md references prompts.summarize_blogs'); + } else { + fail('SKILL.md missing blog prompt', 'Step 4 should list prompts.summarize_blogs'); + } +} + +async function testSkillMdNoContentCheck() { + const skill = await readText('SKILL.md'); + + // The "no content" check must include blogPosts + if (skill.includes('stats.blogPosts')) { + pass('SKILL.md no-content check includes blogPosts'); + } else { + fail('SKILL.md no-content check', 'Should check stats.blogPosts alongside podcastEpisodes and xBuilders'); + } +} + +async function testSampleDigestNoAtHandles() { + const sample = await readText('examples/sample-digest.md'); + + // @ followed by a word character indicates a Twitter handle with @ + // Exclude URLs like https://x.com/@... which are valid + const lines = sample.split('\n'); + const badLines = []; + for (const line of lines) { + // Skip URL lines + if (line.trim().startsWith('http')) continue; + // Check for @handle pattern (not inside URLs) + if (/@\w+/.test(line.replace(/https?:\/\/\S+/g, ''))) { + badLines.push(line.trim()); + } + } + + if (badLines.length === 0) { + pass('Sample digest contains no @handles in non-URL text'); + } else { + fail('Sample digest has @handles', `Found: ${badLines[0]}`); + } +} + +async function testSampleDigestSectionOrder() { + const sample = await readText('examples/sample-digest.md'); + + // digest-intro.md specifies: X/TWITTER → OFFICIAL BLOGS → PODCASTS + const xPos = sample.indexOf('X / TWITTER'); + const blogsPos = sample.indexOf('OFFICIAL BLOGS') !== -1 + ? sample.indexOf('OFFICIAL BLOGS') + : sample.indexOf('BLOGS'); + const podcastsPos = sample.indexOf('PODCASTS'); + + if (xPos === -1) { + fail('Sample digest section order', 'Missing X / TWITTER section'); + return; + } + if (podcastsPos === -1) { + fail('Sample digest section order', 'Missing PODCASTS section'); + return; + } + + if (xPos < podcastsPos && (blogsPos === -1 || (xPos < blogsPos && blogsPos < podcastsPos))) { + pass('Sample digest section order: X → Blogs → Podcasts'); + } else { + fail('Sample digest section order', `Expected X (${xPos}) < Blogs (${blogsPos}) < Podcasts (${podcastsPos})`); + } +} + +async function testPrepareDigestLoadsAllPrompts() { + const prepare = await readText('scripts/prepare-digest.js'); + + const expectedPrompts = [ + 'summarize-podcast.md', + 'summarize-tweets.md', + 'summarize-blogs.md', + 'digest-intro.md', + 'translate.md' + ]; + + for (const prompt of expectedPrompts) { + if (prepare.includes(prompt)) { + pass(`prepare-digest.js references ${prompt}`); + } else { + fail(`prepare-digest.js missing prompt`, `${prompt} not found in PROMPT_FILES`); + } + } +} + +// -- Run --------------------------------------------------------------------- + +console.log('\nDocumentation Consistency Tests\n'); + +await testPodcastCountInReadmes(); +await testSkillMdBlogPromptReference(); +await testSkillMdNoContentCheck(); +await testSampleDigestNoAtHandles(); +await testSampleDigestSectionOrder(); +await testPrepareDigestLoadsAllPrompts(); + +console.log(''); +if (failures > 0) { + console.error(`${failures} test(s) failed`); + process.exit(1); +} else { + console.log('All tests passed'); +}