feat: shape annotations + animation presets for AI-authored overlays#46
Open
Misterr-H wants to merge 1 commit into
Open
feat: shape annotations + animation presets for AI-authored overlays#46Misterr-H wants to merge 1 commit into
Misterr-H wants to merge 1 commit into
Conversation
Adds a new `.shape` clip type (rect, oval, circle, arrow, line) and a
named-preset animation engine (pop-in, draw-on, shake, spin, slide-{in,out},
fade-{in,out}, un-draw). Shape clips render as `CAShapeLayer` overlays
following the same pattern text clips already use — never as composition
tracks. The composition rebuild path explicitly skips both text and shape
clips, so adding overlays never triggers an AVAsset load.
Two new MCP tools: `add_shapes` (batch placement with optional enter/exit/
loop animations baked into one call) and `apply_animation` (preset by name
on an existing clip). Each tool is one undoable action, follows the strict-
allowed-keys validation pattern from add_texts, and reuses every existing
keyframe track — no new keyframe infrastructure.
Phases:
- Data model: ClipType (.shape), ShapeStyle.swift, Clip.shapeStyle +
Clip.strokeProgressTrack, AnimatableProperty.strokeProgress
- Renderer: ShapeLayerController (CAShapeLayer overlay tree + export bake
via AVVideoCompositionCoreAnimationTool), PreviewNSView setShapeRoot,
VideoEngine syncShapeLayers, TimelineRenderer + ExportService bake hook
- VM: EditorViewModel.placeShapeClips, shape-aware property mutation
routing (syncShapeLayers instead of composition rebuild)
- Presets: AnimationPresets engine — 17 named presets, pure functions,
unit-testable, returns keyframe rows that drop into existing tracks
- MCP: add_shapes, apply_animation tool definitions and executor;
AgentInstructions guidance for tutorial-annotation workflows
Build clean. Existing tests pass. No new keyframe infrastructure, no new
runtime, no new external dependencies.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Misterr-H
marked this pull request as ready for review
June 19, 2026 21:21
TimLai666
added a commit
to TimLai666/fronda
that referenced
this pull request
Jun 23, 2026
…mier-io#95, palmier-io#94, palmier-io#92, palmier-io#108, palmier-io#46, palmier-io#105, palmier-io#114) in rewrite specs Adds Upstream change tracking sections to: - 01-foundation-and-project-model.md: clip data model, project.json - 03-timeline-editor-and-preview.md: compositor, captions, preview - 04-export-rendering-and-interchange.md: stall watcher, resolutions - 05-agent-mcp-and-chat.md: agent tools, transform merge
TimLai666
added a commit
to TimLai666/fronda
that referenced
this pull request
Jun 23, 2026
- Add ShapeStyle model (ShapeKind, Stroke, Fill, Arrowhead, Endpoints) - Add ClipType::Shape variant - Add shape_style and stroke_progress_track fields to Clip - Add add_shapes and apply_animation tool definitions - Add AddShapesInput and ApplyAnimationInput validation structs - Add serialization tests and tool validation tests - Update tool count to 39 across all crates
TimLai666
added a commit
to TimLai666/fronda
that referenced
this pull request
Jul 3, 2026
The compositor now rasterizes shape clips instead of skipping them: rasterize_shape draws a filled + border-stroked rectangle or ellipse into an RGBA buffer, and compose_frame blits it through the same transform/rotation/opacity/blend path as media (procedural, so fetch_source is not consulted). visible_clips now includes shape clips (empty media_ref allowed); Text still needs a rasterizer and Arrow/ Line are follow-ups. So shape annotations (PR palmier-io#46) now appear in both the preview canvas and the exported video. +3 tests: rect fully filled, oval fills center not corners, a shape clip composites its fill onto the canvas.
TimLai666
added a commit
to TimLai666/fronda
that referenced
this pull request
Jul 4, 2026
The Rust-only stroke_progress_track (palmier-io#46 shape draw-on animation) is a clip-relative-frame KeyframeTrack, but was omitted from all three places that maintain the other six tracks: rescale_clip_keyframes (fps change), clamp_clip_keyframes_to_duration, and split_all_clip_keyframe_tracks. So a shape's stroke animation would desync on an fps change and leak/duplicate across a clip split. Also added to count_keyframes for accurate reporting. No render impact yet (the compositor doesn't sample the track), but the stored data is now maintained correctly for when it is wired up. Tests cover the rescale (50→100 at 2x) and split partitioning.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses #45.
Opening this as a draft per CONTRIBUTING.md — happy to leave it parked until there's room to review, or close it if the direction doesn't fit. The issue lays out the user-problem framing; this PR is the implementation.
Summary
.shapeclip type (rect, oval, circle, arrow, line) rendered asCAShapeLayeroverlays — same pattern text clips already use, never composition tracks.pop-in,draw-on,shake-{subtle,strong},spin,fade-{in,out},slide-{in,out}-{up,down,left,right},un-draw) that compile to keyframe rows on the existing tracks.add_shapes(batch, with enter/exit/loop animations in one call) andapply_animation(preset on an existing clip). Each is one undoable action; strict allowed-keys validation matchesadd_texts.position,scale,rotation,opacity); adds one newstrokeProgressTrackto drive draw-on / un-draw.File map
Models/ClipType.swift,Models/ShapeStyle.swift(new),Models/Timeline.swift,Models/Keyframe.swift,UI/AppTheme.swift.shapecase, style payload,Clip.shapeStyle+Clip.strokeProgressTrack,AnimatableProperty.strokeProgress, track colorPreview/ShapeLayerController.swift(new),Preview/PreviewView.swift,Preview/VideoEngine.swift,Preview/CompositionBuilder.swift,Preview/TimelineRenderer.swift,Export/ExportService.swiftAVVideoCompositionCoreAnimationTool;CompositionBuilderfilters out shape clips (same as text) so they never become AVAsset-backed composition tracksEditor/ViewModel/EditorViewModel+ClipMutations.swift,EditorViewModel+Keyframes.swift,EditorViewModel+MediaLibrary.swift,EditorViewModel+Tracks.swiftplaceShapeClips(mirrorsplaceTextClips), shape-aware property mutation routing (syncShapeLayersinstead of composition rebuild)Editor/Animations/AnimationPresets.swift(new)ClipAgent/Tools/ToolDefinitions.swift,ToolExecutor.swift,ToolExecutor+Shapes.swift(new),ToolExecutor+Generate.swift,ToolExecutor+Timeline.swift,AgentInstructions.swiftadd_shapes,apply_animationtool definitions + executor;.shaperejected at generate/timeline-inspect paths; agent guidance for tutorial-annotation patternsGeneration/Edit/*.swift,Generation/GenerationService.swift,Generation/UI/GenerationView.swift,MediaPanel/MediaTab/MediaTab.swift.shapeto existingClipTypeexhaustive switches; disambiguates SwiftUI'sShapeStyleprotocol where it collides with the new model type27 files, +1402 / -55, 4 new files.
What the agent call looks like end-to-end
One undoable action; the preview's
CAShapeLayertree updates immediately viasyncShapeLayers(); no composition rebuild required.Test plan
swift buildclean (no new warnings introduced)swift test— existing tests pass; no behavioral regressions in keyframe sampling, composition build for non-shape timelines, or text-clip placementtools/listexposesadd_shapes+apply_animationwith their schemastools/call add_shapesagainst a live MCP server returns success + clip IDslibconvexmobile.abuilt for SDK 26.2 vs deployment 26.0). Code paths verified via MCP responses and OSLog. Will fold in a recording on a healthy machine when reviewed.Out of scope (intentional)
ShapeTabinspector mirroringTextTab)linear/hold/smoothHow I'd split this if you'd rather review smaller chunks
Happy to rebase into that shape on request. Also happy to close this if the direction doesn't fit — no hard feelings either way.
🤖 Generated with Claude Code