Skip to content

Conversation

mfts
Copy link
Owner

@mfts mfts commented Sep 7, 2025

Add M4A and MP3 audio file support, leveraging existing video infrastructure for playback and analytics.


Slack Thread

Open in Cursor Open in Web

Summary by CodeRabbit

  • New Features

    • Added support for audio formats (M4A, MP3) across upload, detection, and extension handling.
    • Introduced a unified Media Player (replacing Video Player) with download disabled by default.
    • Updated file icons to recognize new audio types.
  • Bug Fixes

    • Prevented unintended video processing when content type is missing or not a video.
    • Hardened video analytics with strict event validation, safer per-second aggregation, and defensive calculations.
    • Added API response validation to avoid processing malformed analytics data.

Copy link

cursor bot commented Sep 7, 2025

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
Learn more about Cursor Agents

Copy link

vercel bot commented Sep 7, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
papermark Ready Ready Preview Comment Sep 15, 2025 7:05pm

Copy link
Contributor

coderabbitai bot commented Sep 7, 2025

Walkthrough

Renames the video playback component to MediaPlayer with prop updates and a backward-compatible alias. Expands supported MIME types to include several audio types and maps them to “video” handling. Tightens video-processing conditions to require video MIME types. Hardens video analytics validation and aggregation. Minor cosmetic changes to imports and a trailing newline.

Changes

Cohort / File(s) Summary
Player rename and props alignment
components/view/viewer/video-player.tsx, components/view/viewer/video-viewer.tsx
Renamed VideoPlayer to MediaPlayer; prop videoSrcmediaSrc; default preventDownload set to true; maintained alias export const VideoPlayer = MediaPlayer; updated imports/usages in VideoViewer.
MIME support expansion
lib/constants.ts, lib/utils/get-content-type.ts, lib/utils/get-file-icon.tsx
Added audio MIME types: audio/mp4, audio/x-m4a, audio/m4a, audio/mpeg. Classified as “video” in getSupportedContentType; extensions mapped (m4a, mp3); icons updated to use VideoIcon; included in supported/accepted types.
Video processing gate hardening
lib/api/documents/process-document.ts, pages/api/.../versions/index.ts
Gated processing to type === "video" with contentType?.startsWith("video/") and exclusion of video/mp4; avoids triggering when contentType is missing or non-video.
Analytics validation and aggregation
pages/api/.../video-analytics.ts
Added event schema validation, per-second aggregation safeguards, unique viewer tracking per second, defensive metrics (avg duration, first/last viewed), and response validation for analytics fetch.
Cosmetic/reordering
components/upload-zone.tsx, components/welcome/special-upload.tsx
Added trailing newline; reordered some imports without logic changes.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Client
  participant API as Versions API
  participant Proc as Video Processor

  Client->>API: Upload document version (type, contentType, url)
  alt contentType startsWith("video/") && type === "video" && contentType != "video/mp4"
    API->>Proc: trigger processVideo(url, teamId, documentVersionId, ...)
    Note right of Proc: Starts async processing pipeline
  else Not a qualifying video
    API-->>Client: Skip video processing path
  end
Loading
sequenceDiagram
  autonumber
  participant Client
  participant API as Video Analytics API
  participant TB as Tinybird

  Client->>API: GET /video-analytics?docId=...&videoLength=...
  API->>TB: Query analytics data
  TB-->>API: data[]
  alt data is valid array
    API->>API: Validate and filter events (types, times, view_id)
    API->>API: Aggregate per-second buckets and unique viewers
    API-->>Client: Aggregated metrics and distributions
  else invalid/missing data
    API->>API: Log error
    API-->>Client: 500 with error message
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title "Add m4a and mp3 audio support" accurately and concisely describes the PR's primary change—adding M4A/MP3 MIME types and related support (constants, content-type handling, icons, and viewer/analytics)—and is clear and scan-friendly. Other edits in the changeset (MediaPlayer rename, import reordering, defensive checks) are secondary and do not make the title misleading or irrelevant.
✨ Finishing touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch cursor/add-m4a-and-mp3-audio-support-1c1b

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@mfts mfts marked this pull request as ready for review September 15, 2025 18:59
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
pages/api/teams/[teamId]/documents/[id]/video-analytics.ts (1)

78-85: Unique views should be computed from validated events (and skip falsy view_id).

Counting from all events can inflate unique_views and avg_view_duration denominator with invalid/undefined view_ids.

Apply this diff:

-    const uniqueViewIds = new Set(events.map((e) => e.view_id));
+    const uniqueViewIds = new Set(
+      validEvents.map((e) => e.view_id).filter((v): v is string => Boolean(v))
+    );

Also applies to: 144-157

lib/constants.ts (1)

67-102: Map audio/ to the correct size limit (treat as video or add a dedicated audio limit)*

getFileSizeLimit in lib/utils/get-file-size-limits.ts currently falls through to limits.document for audio/*; handle audio explicitly — either add an "audio" limit to FileSizeLimits/getFileSizeLimits or at minimum add if (fileType.startsWith("audio/")) return limits.video;. Location: lib/utils/get-file-size-limits.ts → getFileSizeLimit.

components/upload-zone.tsx (1)

595-608: Populate audio extension arrays in lib/constants.ts to restore extension-based MIME resolution.

upload-zone resolves MIME by matching file extensions against acceptableDropZoneFileTypes (components/upload-zone.tsx lines 593–601); lib/constants.ts currently has empty arrays for "audio/mp4", "audio/x-m4a", "audio/m4a", and "audio/mpeg" (lib/constants.ts lines 145–148), so .m4a/.mp3 dropped from folders won't match — add the appropriate extensions (e.g. [".m4a", ".mp3"]).

🧹 Nitpick comments (13)
components/upload-zone.tsx (2)

720-723: Update hint text to reflect new audio support (.m4a, .mp3).

The drop hint omits audio types now accepted for paid plans.

Apply this diff:

-                    : `Only *.pdf, *.pptx, *.docx, *.xlsx, *.xls, *.csv, *.ods, *.ppt, *.odp, *.doc, *.odt, *.dwg, *.dxf, *.png, *.jpg, *.jpeg, *.mp4, *.mov, *.avi, *.webm, *.ogg`}
+                    : `Only *.pdf, *.pptx, *.docx, *.xlsx, *.xls, *.csv, *.ods, *.ppt, *.odp, *.doc, *.odt, *.dwg, *.dxf, *.png, *.jpg, *.jpeg, *.mp4, *.mov, *.avi, *.webm, *.ogg, *.m4a, *.mp3`}

676-677: Add missing dependency to keep extension mapping in sync with plan changes.

getFilesFromEvent closes over acceptableDropZoneFileTypes but doesn’t list it in deps.

Apply this diff:

-  );
+  , [folderPathName, endpointTargetType, teamInfo, acceptableDropZoneFileTypes]
+  );
lib/utils/get-file-icon.tsx (1)

59-61: Include all m4a MIME variants for consistent icon rendering.

audio/x-m4a and audio/m4a aren’t covered; they’ll fall to default FileIcon.

Apply this diff:

     case "video/x-msvideo":
     case "video":
+    case "audio/x-m4a":
+    case "audio/m4a":
     case "audio/mp4":
     case "audio/mpeg":
       return <VideoIcon className={className} isLight={isLight} />;
components/welcome/special-upload.tsx (1)

94-96: Clarify unsupported-format message (now misleading if media is allowed here).

If this flow can accept any supported types, broaden the copy. If it’s intentionally docs-only, ignore.

Apply this diff:

-          "Unsupported file format. Please upload a PDF or Excel file.",
+          "Unsupported file format.",
lib/constants.ts (1)

145-148: Populate extensions for new audio MIME types to fix folder-upload mime resolution.

Empty arrays break the extension-based fix in upload-zone for files with type "" (Firefox).

Apply this diff:

-  "audio/mp4": [], // ".m4a"
-  "audio/x-m4a": [], // ".m4a"
-  "audio/m4a": [], // ".m4a"
-  "audio/mpeg": [], // ".mp3"
+  "audio/mp4": [".m4a"], // ".m4a"
+  "audio/x-m4a": [".m4a"], // ".m4a"
+  "audio/m4a": [".m4a"], // ".m4a"
+  "audio/mpeg": [".mp3"], // ".mp3"
pages/api/teams/[teamId]/documents/[id]/video-analytics.ts (2)

92-97: Off-by-one: avoid creating an extra, unused time bucket.

Initialize 0..videoLength-1 for per-second buckets.

Apply this diff:

-    for (let t = 0; t <= videoLength; t++) {
+    for (let t = 0; t < Math.ceil(videoLength); t++) {

216-219: Use event-derived fallback length when document length is missing.

Hardcoding 51 can filter out valid events (end_time <= videoLength + 10). Derive from data.

Apply this diff:

-    const videoLength = document.versions[0]?.length || 51;
-    if (!videoLength) {
-      return res.status(400).json({ message: "Video length not found" });
-    }
+    const storedLength = document.versions[0]?.length;
...
-      const analytics = calculateAnalytics(response.data, videoLength);
+      const derivedLength =
+        Array.isArray(response.data) && response.data.length
+          ? Math.max(
+              1,
+              Math.ceil(
+                Math.max(
+                  ...response.data
+                    .map((e: any) => e?.end_time)
+                    .filter((n: any) => typeof n === "number"),
+                ),
+              ),
+            )
+          : 0;
+      const effectiveLength = storedLength ?? derivedLength ?? 0;
+      if (!effectiveLength) {
+        return res.status(400).json({ message: "Video length not found" });
+      }
+      const analytics = calculateAnalytics(response.data, effectiveLength);

Also applies to: 242-243

components/view/viewer/video-player.tsx (1)

64-68: Drop redundant or add type; pick one source mechanism.

Using both src and a bare is unnecessary and can confuse content-type negotiation. With only a URL, the src attribute suffices.

Apply this diff:

-          src={mediaSrc ?? videoSrc ?? ""}
-        >
-          <source src={mediaSrc ?? videoSrc ?? ""} />
+          src={mediaSrc ?? videoSrc ?? ""}
+        >
components/view/viewer/video-viewer.tsx (1)

112-117: Use media‑generic copy to match audio support

Now that audio uses this path, prefer “media” over “video” in user‑facing errors.

Apply:

-  console.error("Video playback error:", error);
-  toast.error("Error playing video. Please try again.");
+  console.error("Media playback error:", error);
+  toast.error("Error playing media. Please try again.");
lib/api/documents/process-document.ts (1)

174-174: Gate still skips non‑mp4 videos when contentType is missing

The new check avoids processing audio (good), but if contentType is absent you’ll now skip valid non‑mp4 videos (regression risk).

Consider a safe fallback to extension when contentType is falsy:

-  if (type === "video" && contentType !== "video/mp4" && contentType?.startsWith("video/")) {
+  if (
+    type === "video" &&
+    (
+      (contentType?.startsWith("video/") && contentType !== "video/mp4") ||
+      (!contentType && getExtension(name)?.toLowerCase() !== "mp4")
+    )
+  ) {
lib/utils/get-content-type.ts (2)

37-41: Audio MIME types routed to “video” path

This achieves reuse of the player/analytics pipeline for m4a/mp3.

If you want broader real‑world coverage without adding many cases, consider also accepting common non‑standard aliases like "audio/mp3" and "audio/x-mp3".


105-110: Add content‑type normalization to handle parameters/case

Strict equality can miss values like audio/mpeg; charset=binary or mixed case. Normalize once per function.

Apply near the top of each function:

-export function getSupportedContentType(contentType: string): string | null {
-  switch (contentType) {
+export function getSupportedContentType(contentType: string): string | null {
+  const ct = contentType?.split(";")[0]?.toLowerCase();
+  switch (ct) {
-export function getExtensionFromContentType(
-  contentType: string,
-): string | null {
-  switch (contentType) {
+export function getExtensionFromContentType(
+  contentType: string,
+): string | null {
+  const ct = contentType?.split(";")[0]?.toLowerCase();
+  switch (ct) {

Optionally add:

+    case "audio/mp3":
+    case "audio/x-mp3":
+      return "mp3";
pages/api/teams/[teamId]/documents/[id]/versions/index.ts (1)

136-140: Same gating concern: add fallback when contentType is missing

Without contentType, valid non‑mp4 videos won’t be processed.

Minimal, URL‑based fallback (keys look like paths here):

-      if (
-        type === "video" &&
-        contentType !== "video/mp4" &&
-        contentType?.startsWith("video/")
-      ) {
+      if (
+        type === "video" &&
+        (
+          (contentType?.startsWith("video/") && contentType !== "video/mp4") ||
+          (!contentType && !url.toLowerCase().endsWith(".mp4"))
+        )
+      ) {
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 458247c and dd44fe6.

📒 Files selected for processing (10)
  • components/upload-zone.tsx (1 hunks)
  • components/view/viewer/video-player.tsx (3 hunks)
  • components/view/viewer/video-viewer.tsx (2 hunks)
  • components/welcome/special-upload.tsx (1 hunks)
  • lib/api/documents/process-document.ts (1 hunks)
  • lib/constants.ts (2 hunks)
  • lib/utils/get-content-type.ts (2 hunks)
  • lib/utils/get-file-icon.tsx (1 hunks)
  • pages/api/teams/[teamId]/documents/[id]/versions/index.ts (1 hunks)
  • pages/api/teams/[teamId]/documents/[id]/video-analytics.ts (4 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
components/view/viewer/video-viewer.tsx (1)
components/view/viewer/video-player.tsx (1)
  • MediaPlayer (19-72)
🔇 Additional comments (3)
components/welcome/special-upload.tsx (1)

23-31: Import re-ordering is fine.

components/view/viewer/video-player.tsx (1)

19-21: ```shell
#!/bin/bash
set -euo pipefail

echo "PWD: $(pwd)"
echo "---- rg 'preventDownload' ----"
rg -nP --hidden --no-ignore 'preventDownload' || true

echo
echo "---- rg 'prevent-download' ----"
rg -nP --hidden --no-ignore 'prevent-download' || true

echo
echo "---- rg 'controlsList' ----"
rg -nP --hidden --no-ignore 'controlsList' || true

echo
echo "---- rg for context menu handlers (onContextMenu|contextmenu|oncontextmenu) ----"
rg -nP --hidden --no-ignore -S 'onContextMenu|contextmenu|oncontextmenu' || true

echo
echo "---- rg MediaPlayer / VideoPlayer JSX usages ----"
rg -nP --hidden --no-ignore '<(?:MediaPlayer|VideoPlayer)\b' -S || true

echo
echo "---- rg for MediaPlayerProps / MediaPlayer type ----"
rg -nP --hidden --no-ignore 'MediaPlayerProps\b|type\s+MediaPlayerProps\b|interface\s+MediaPlayerProps\b' -S || true

echo
echo "---- Show components/view/viewer/video-player.tsx (if exists) ----"
if [ -f components/view/viewer/video-player.tsx ]; then
sed -n '1,400p' components/view/viewer/video-player.tsx
else
echo "file not found: components/view/viewer/video-player.tsx"
fi

echo
echo "---- Show components/view/viewer/video-viewer.tsx (if exists) ----"
if [ -f components/view/viewer/video-viewer.tsx ]; then
sed -n '1,400p' components/view/viewer/video-viewer.tsx
else
echo "file not found: components/view/viewer/video-viewer.tsx"
fi


</blockquote></details>
<details>
<summary>components/view/viewer/video-viewer.tsx (1)</summary><blockquote>

`10-10`: **Import swap to MediaPlayer looks good — verified.**

No remaining references to VideoPlayer or videoSrc; video-player.tsx still re-exports VideoPlayer for backward compatibility (components/view/viewer/video-player.tsx:76–77).

</blockquote></details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->

Comment on lines +11 to 17
interface MediaPlayerProps {
mediaSrc: string;
onError?: (error: Error) => void;
controls?: boolean;
className?: string;
preventDownload?: boolean;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Alias isn’t prop‑compatible; accept both mediaSrc and legacy videoSrc.

The alias export alone doesn’t preserve the old prop name; existing usages will break. Support both to truly be backward‑compatible.

Apply this diff:

-interface MediaPlayerProps {
-  mediaSrc: string;
+interface MediaPlayerProps {
+  mediaSrc?: string;
+  // Back-compat with old VideoPlayer prop
+  videoSrc?: string;
   onError?: (error: Error) => void;
   controls?: boolean;
   className?: string;
   preventDownload?: boolean;
 }
@@
-        mediaSrc,
+        mediaSrc,
+        videoSrc,
         onError,
         controls = true,
         className = "",
         preventDownload = true,
       },
@@
-          src={mediaSrc}
+          src={mediaSrc ?? videoSrc ?? ""}
         >
-          <source src={mediaSrc} />
+          <source src={mediaSrc ?? videoSrc ?? ""} />
           Your browser does not support the media tag.
         </video>
@@
 // Keep the old VideoPlayer export for backward compatibility
 export const VideoPlayer = MediaPlayer;

Also applies to: 23-29, 64-68, 76-78

🤖 Prompt for AI Agents
In components/view/viewer/video-player.tsx around lines 11-17 (and similarly at
23-29, 64-68, 76-78), the new prop alias only exports mediaSrc and will break
existing callers using the legacy videoSrc prop; update the component props and
runtime handling to accept both names by adding videoSrc?: string to the
interface, normalize inside the component (const src = mediaSrc ?? videoSrc) and
use src for rendering and error callbacks, and update any prop-types or exported
types so both props are accepted and documented; ensure TypeScript types mark
one as optional and preserve onError/controls/className/preventDownload
behavior.

@mfts mfts merged commit af40772 into main Sep 15, 2025
9 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Sep 15, 2025
@mfts mfts deleted the cursor/add-m4a-and-mp3-audio-support-1c1b branch September 16, 2025 14:39
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants