Skip to content

Fix audio recording and deal with audio watermarking in sub tasks #413

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
julien-nc merged 7 commits into from
Dec 4, 2025
Merged

Fix audio recording and deal with audio watermarking in sub tasks #413

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3083271
record to wav, not to mp3
julien-nc Dec 3, 2025
68485be
enh(audiochat): do not watermark audio output of audio chat, only sup…
julien-nc Dec 3, 2025
bbb6b95
fix(audio-recording): add necessary CSP connect domain
julien-nc Dec 3, 2025
b1fb17c
fix: require php 8.2
julien-nc Dec 3, 2025
f33923c
enh: remove unused audio utils
julien-nc Dec 3, 2025
c13b3de
fix: update composer.lock after changing config.platform in composer.…
julien-nc Dec 3, 2025
eb0e15d
enh: keep supporting NC >= 30
julien-nc Dec 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/appstore-build-publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ on:
types: [published]

env:
PHP_VERSION: 8.1
PHP_VERSION: 8.2

jobs:
build_and_publish:
Expand Down
2 changes: 1 addition & 1 deletion appinfo/info.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Known providers:

More details on how to set this up in the [admin docs](https://docs.nextcloud.com/server/latest/admin_manual/ai/index.html)
]]> </description>
<version>2.10.0</version>
<version>2.11.0-dev</version>
<licence>agpl</licence>
<author>Julien Veyssier</author>
<namespace>Assistant</namespace>
Expand Down
134 changes: 25 additions & 109 deletions composer.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions lib/Listener/CSPListener.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ public function handle(Event $event): void {

$csp = new ContentSecurityPolicy();
$csp->addAllowedWorkerSrcDomain('blob:');
$csp->addAllowedConnectDomain('blob:');
$event->addPolicy($csp);
}
}
4 changes: 4 additions & 0 deletions lib/TaskProcessing/AudioToAudioChatProvider.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,10 @@ public function process(?string $userId, array $input, callable $reportProgress)
Application::APP_ID . ':internal',
$userId,
);
// the setIncludeWatermark method was introduced in NC 33
if (method_exists($task, 'setIncludeWatermark')) {
$task->setIncludeWatermark(false);
}
$taskOutput = $this->taskProcessingService->runTaskProcessingTask($task);
$outputAudioFileId = $taskOutput['speech'];

Expand Down
4 changes: 4 additions & 0 deletions lib/TaskProcessing/ContextAgentAudioInteractionProvider.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,10 @@ public function process(?string $userId, array $input, callable $reportProgress)
Application::APP_ID . ':internal',
$userId,
);
// the setIncludeWatermark method was introduced in NC 33
if (method_exists($task, 'setIncludeWatermark')) {
$task->setIncludeWatermark(false);
}
$ttsTaskOutput = $this->taskProcessingService->runTaskProcessingTask($task);
$outputAudioFileId = $ttsTaskOutput['speech'];
$outputAudioFileContent = $this->taskProcessingService->getOutputFileContent($outputAudioFileId);
Expand Down
48 changes: 0 additions & 48 deletions src/audioUtils.js
View file
Open in desktop

This file was deleted.

2 changes: 1 addition & 1 deletion src/components/ChattyLLM/InputArea.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ export default {
const url = generateOcsUrl('/apps/assistant/api/v1/input-file')
const formData = new FormData()
formData.append('data', blob)
formData.append('filename', 'chat-input.mp3')
formData.append('filename', 'chat-input.wav')
axios.post(url, formData).then(response => {
this.$emit('submit-audio', response.data.ocs.data.fileId)
}).catch(error => {
Expand Down
4 changes: 1 addition & 3 deletions src/components/fields/AudioRecorderWrapper.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@ import NcButton from '@nextcloud/vue/components/NcButton'

import { showError } from '@nextcloud/dialogs'

import { convertWavToMp3 } from '../../audioUtils.js'
import { MediaRecorder, register } from 'extendable-media-recorder'
import { connect } from 'extendable-media-recorder-wav-encoder'

Expand Down Expand Up @@ -210,8 +209,7 @@ export default {
this.killStreams()
if (!this.aborted) {
const wavBlob = new Blob(this.chunks, { type: this.mediaRecorder.mimeType })
const mp3Blob = await convertWavToMp3(wavBlob)
this.$emit('new-recording', mp3Blob)
this.$emit('new-recording', wavBlob)
this.$emit('update:is-recording', false)
}
this.resetComponentData()
Expand Down
2 changes: 1 addition & 1 deletion src/components/fields/ListOfMediaField.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ export default {
const url = generateOcsUrl('/apps/assistant/api/v1/input-file')
const formData = new FormData()
formData.append('data', blob)
formData.append('filename', 'recording.mp3')
formData.append('filename', 'recording.wav')
axios.post(url, formData).then(response => {
const fileId = response.data.ocs.data.fileId
if (this.value === null) {
Expand Down
2 changes: 1 addition & 1 deletion src/components/fields/MediaField.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ export default {
const url = generateOcsUrl('/apps/assistant/api/v1/input-file')
const formData = new FormData()
formData.append('data', blob)
formData.append('filename', 'recording.mp3')
formData.append('filename', 'recording.wav')
axios.post(url, formData).then(response => {
this.$emit('update:value', response.data.ocs.data.fileId)
this.filePath = response.data.ocs.data.filePath
Expand Down