Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,5 +1,107 @@
[[Transcription]]
= Transcription API

Spring AI provides support for OpenAI's Transcription API.
When additional providers for Transcription are implemented, a common `AudioTranscriptionModel` interface will be extracted.
Spring AI provides a unified API for Speech-to-Text transcription through the `TranscriptionModel` interface. This allows you to write portable code that works across different transcription providers.

== Supported Providers

- xref:api/audio/transcriptions/openai-transcriptions.adoc[OpenAI's Whisper API]
- xref:api/audio/transcriptions/azure-openai-transcriptions.adoc[Azure OpenAI Whisper API]

== Common Interface

All transcription providers implement the following shared interface:

=== TranscriptionModel

The `TranscriptionModel` interface provides methods for converting audio to text:

[source,java]
----
public interface TranscriptionModel extends Model<AudioTranscriptionPrompt, AudioTranscriptionResponse> {

/**
* Transcribes the audio from the given prompt.
*/
AudioTranscriptionResponse call(AudioTranscriptionPrompt transcriptionPrompt);

/**
* A convenience method for transcribing an audio resource.
*/
default String transcribe(Resource resource) {
AudioTranscriptionPrompt prompt = new AudioTranscriptionPrompt(resource);
return this.call(prompt).getResult().getOutput();
}

/**
* A convenience method for transcribing an audio resource with options.
*/
default String transcribe(Resource resource, AudioTranscriptionOptions options) {
AudioTranscriptionPrompt prompt = new AudioTranscriptionPrompt(resource, options);
return this.call(prompt).getResult().getOutput();
}
}
----

=== AudioTranscriptionPrompt

The `AudioTranscriptionPrompt` class encapsulates the input audio and options:

[source,java]
----
Resource audioFile = new FileSystemResource("/path/to/audio.mp3");
AudioTranscriptionPrompt prompt = new AudioTranscriptionPrompt(
audioFile,
options
);
----

=== AudioTranscriptionResponse

The `AudioTranscriptionResponse` class contains the transcribed text and metadata:

[source,java]
----
AudioTranscriptionResponse response = model.call(prompt);
String transcribedText = response.getResult().getOutput();
AudioTranscriptionResponseMetadata metadata = response.getMetadata();
----

== Writing Provider-Agnostic Code

One of the key benefits of the shared transcription interface is the ability to write code that works with any transcription provider without modification. The actual provider (OpenAI, Azure OpenAI, etc.) is determined by your Spring Boot configuration, allowing you to switch providers without changing application code.

=== Basic Service Example

The shared interface allows you to write code that works with any transcription provider:

[source,java]
----
@Service
public class TranscriptionService {

private final TranscriptionModel transcriptionModel;

public TranscriptionService(TranscriptionModel transcriptionModel) {
this.transcriptionModel = transcriptionModel;
}

public String transcribeAudio(Resource audioFile) {
return transcriptionModel.transcribe(audioFile);
}

public String transcribeWithOptions(Resource audioFile, AudioTranscriptionOptions options) {
AudioTranscriptionPrompt prompt = new AudioTranscriptionPrompt(audioFile, options);
AudioTranscriptionResponse response = transcriptionModel.call(prompt);
return response.getResult().getOutput();
}
}
----

This service works seamlessly with OpenAI, Azure OpenAI, or any other transcription provider, with the actual implementation determined by your Spring Boot configuration.

== Provider-Specific Features

While the shared interface provides portability, each provider also offers specific features through provider-specific options classes (e.g., `OpenAiAudioTranscriptionOptions`, `AzureOpenAiAudioTranscriptionOptions`). These classes implement the `AudioTranscriptionOptions` interface while adding provider-specific capabilities.

For detailed information about provider-specific features, see the individual provider documentation pages.