-
Notifications
You must be signed in to change notification settings - Fork 595
some notes from trying to put this into practice #3
Description
I think the idea behind this repo is excellent, but I encountered several issues when trying to put it to use.
For starters, Claude Code (1.0.44) could not access the commands in .claude/commands/PRPs. Apparently, the existence of another folder with the same name in the project was tripping it up, because it worked once I renamed .claude/commands/PRPs to .claude/commands/PRP-commands.
I then ran /prp-base-create with claude 4 opus, feeding it a fairly detailed, high-level description of an entire project that I wanted to create (project folder was empty aside from the prp-related files/folders). It ran for about 15 minutes and created a PRP file which largely resembled the examples from your repo, though it wasnt using keywords like CREATE, MOVE, ADD, ENSURE etc in the task list. Rather, it split the "Implementation Blueprint" section into 5 Phases (with some pseudocode for each) and 20 Tasks (each of which only consisted of a name, e.g. "1. [ ] Set up project structure and dependencies"). Surprisingly, it also created several .md and .sql files in my project root which contained research on some of the frameworks it intended to use for the project (e.g. langchain / langgraph) and example schemas for the database that it wanted to implement.
While a bit unexpected, I figured I'd just try and see what the eventual result was going to be, so I ran /prp-base-execute with the newly created PRP file, this time using claude 4 sonnet. It churned through the list of 20 tasks in about 90 minutes, although it did stop after tasks 8, 12 and 16 without being able to explain why, and had to be prompted to resume execution.
The resulting project looks well-structured and documented, but failed to execute. I used a fresh session to start debugging the issues, but after fixing the first 5 or so realized this would likely prove to be a very long process. Same goes for manual debugging, as the project is almost 15k lines of code.
Not that this should be surprising - I did not expect coding agents to be able to write a complete, reasonably complex, functional project from just a high-level description, and indeed they weren't. I figured you might be interested in my experience though, particularly the ways in which claude code behaved unexpectedly when creating / executing the PRP.