Skip to content

Fix audio/video sync: publish audio from camera when enabled and and … #119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

Fix audio/video sync: publish audio from camera when enabled and and … #119

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0330383
Fix audio/video sync: publish audio from camera when enabled and and …
Yasir761 Oct 5, 2025
81ec8ad
fix: conditionally render ListenToRemoteAudio based on camera window …
Yasir761 Oct 6, 2025
281cd04
fix: track camera window open in callState
Yasir761 Oct 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 28 additions & 23 deletions tauri/src/components/ui/call-center.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -236,18 +236,19 @@ export function ConnectedActions() {
</TooltipProvider>
)}
<Button
className="w-full border-red-500 text-red-600 flex flex-row gap-2"
className="w-full border-red-500 text-red-600 flex flex-row gap-1.5 text-sm"
variant="gradient-white"
onClick={handleEndCall}
>
<HiOutlinePhoneXMark className="size-4" />
<HiOutlinePhoneXMark className="size-3.5" />
End call
</Button>
</div>
</div>
</div>
</div>
<ListenToRemoteAudio />
{/* Render remote audio only if camera is NOT enabled */}
{!callTokens?.hasCameraEnabled && <ListenToRemoteAudio />} diable to solve the issue related to sync audio playback
Copy link
Contributor

@iparaskev iparaskev Oct 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not 100% correct, hasCameraEnabled is set when the current user is publishing their local camera. We want to disable this when the camera window is open, which could also happen if another participant is publishing their camera. We could update CallState to keep track of the camera window is opened or not.

</>
);
}
Expand Down Expand Up @@ -343,7 +344,7 @@ function MicrophoneIcon({ setMicEnabled }: { setMicEnabled: (enabled: boolean) =
[Colors.mic.text]: hasAudioEnabled,
[Colors.deactivatedIcon]: !hasAudioEnabled,
})}
className="hover:outline-solid hover:outline-1 hover:outline-slate-300 focus:ring-0 focus-visible:ring-0 hover:bg-slate-200 size-4 rounded-xs p-0 border-0 shadow-none hover:shadow-xs"
className=" p-0.5 text-xs rounded-sm hover:outline-solid hover:outline-1 hover:outline-slate-300 focus:ring-0 focus-visible:ring-0 hover:bg-slate-200 size-4 border-0 shadow-none hover:shadow-xs"
/>
<SelectPortal container={document.getElementsByClassName("container")[0]}>
<SelectContent align="center">
Expand Down Expand Up @@ -509,7 +510,7 @@ function ScreensharingEventListener({
return <div />;
}

function CameraIcon() {
function CameraIcon({ micEnabled }: { micEnabled: boolean }) {
Copy link
Contributor

@iparaskev iparaskev Oct 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for this. We don't want to change the way we capture the audio and camera. We only want to change the way the audio is played.

const { updateCallTokens, callTokens } = useStore();
const [retry, setRetry] = useState(0);
const tracks = useTracks([Track.Source.Camera], {});
Expand Down Expand Up @@ -541,23 +542,27 @@ function CameraIcon() {

const isDisabled = cameraDevices.length === 0;



const handleCameraToggle = () => {
clickedCameraRef.current = true;
let newCameraEnabled = !cameraEnabled;
updateCallTokens({
...callTokens,
hasCameraEnabled: newCameraEnabled,
});
if (!newCameraEnabled) {
const cameraTrack = localParticipant
.getTrackPublications()
.filter((track) => track.source === Track.Source.Camera)[0];
if (cameraTrack && cameraTrack.track && cameraTrack.track instanceof LocalTrack) {
localParticipant.unpublishTrack(cameraTrack.track);
}
clickedCameraRef.current = true;
const newCameraEnabled = !cameraEnabled;
updateCallTokens({ ...callTokens, hasCameraEnabled: newCameraEnabled });

if (newCameraEnabled && micEnabled) {
// Publish a camera track with audio
localParticipant.setMicrophoneEnabled(true, { noiseSuppression: true, echoCancellation: true });
} else if (!newCameraEnabled) {
// Unpublish camera track and audio
const cameraTrack = localParticipant
.getTrackPublications()
.find((track) => track.source === Track.Source.Camera);
if (cameraTrack && cameraTrack.track instanceof LocalTrack) {
localParticipant.unpublishTrack(cameraTrack.track);
}
};

localParticipant.setMicrophoneEnabled(false);
}
};
const handleCameraChange = (value: string) => {
console.debug("Selected camera: ", value);
setActiveCameraDevice(value);
Expand Down Expand Up @@ -604,7 +609,7 @@ function CameraIcon() {
}
size="unsized"
disabled={isDisabled}
className={clsx("flex-1 min-w-0", {
className={clsx("flex-1 min-w-0 text-xs", {
[Colors.deactivatedText]: !cameraEnabled,
[`${Colors.camera.text} ${Colors.camera.ring}`]: cameraEnabled,
})}
Expand All @@ -623,7 +628,7 @@ function CameraIcon() {
return (
device.deviceId !== "" && (
<SelectItem key={device.deviceId} value={device.deviceId}>
<span className="text-xs truncate">
<span className="text-xs truncate ">
{device.label || `Camera ${device.label.slice(0, 8)}...`}
</span>
</SelectItem>
Expand Down Expand Up @@ -716,7 +721,7 @@ function MediaDevicesSettings() {
return (
<div className="flex flex-row gap-1 w-full">
<MicrophoneIcon setMicEnabled={setMicEnabled} />
<CameraIcon />
<CameraIcon micEnabled={micEnabled}/>
<ScreenShareIcon
callTokens={callTokens}
setCallTokens={setCallTokens}
Expand Down
17 changes: 16 additions & 1 deletion tauri/src/windows/camera/main.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import ReactDOM from "react-dom/client";
import { Toaster } from "react-hot-toast";
import { useDisableNativeContextMenu } from "@/lib/hooks";
import { tauriUtils } from "../window-utils";
import { LiveKitRoom, useTracks, VideoTrack } from "@livekit/components-react";
import { LiveKitRoom, useLocalParticipant, useTracks, VideoTrack } from "@livekit/components-react";
import { Track } from "livekit-client";
import { getCurrentWebviewWindow } from "@tauri-apps/api/webviewWindow";
import { PhysicalSize, LogicalPosition, currentMonitor } from "@tauri-apps/api/window";
Expand Down Expand Up @@ -194,7 +194,22 @@ function CameraWindow() {
const [isSelfHidden, setIsSelfHidden] = useState(false);
const [livekitUrl, setLivekitUrl] = useState<string>("");
const [isExpanded, setIsExpanded] = useState(false);
// this is to resolve the issue of audio sync playback
const { localParticipant } = useLocalParticipant();
const [micEnabled, setMicEnabled] = useState(false);

useEffect(() => {
if (!localParticipant) return;

if (micEnabled) {
localParticipant.setMicrophoneEnabled(true, {
noiseSuppression: true,
echoCancellation: true,
});
} else {
localParticipant.setMicrophoneEnabled(false);
}
}, [micEnabled]);
useEffect(() => {
// Set correct window size
CameraWindowSize({ numOfTracks: 0, expansionFactor: isExpanded ? EXPANSION_FACTOR : 1 });
Expand Down