Getting error with word_timestamps = true

[crsici]

I'm getting below error when running whisper in executor, it's not happen if max_worker=1. Please help if you have any idea!

default_thread_pool` = ThreadPoolExecutor(max_workers=3)

loop = asyncio.get_event_loop()

execFunction = partial(getModel().transcribe, fileName, 
temperature = temperature, word_timestamps = settings.word_timestamps, 
hallucination_silence_threshold = settings.hallucination_silence_threshold, 
**decode_options)

with default_thread_pool as executor:
    transcribe = await loop.run_in_executor(executor, execFunction)

File "root/.venv/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "root/.venv/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in call
return await self.app(scope, receive, send)
File "root/.venv/lib/python3.9/site-packages/fastapi/applications.py", line 1054, in call
await super().call(scope, receive, send)
File "root/.venv/lib/python3.9/site-packages/starlette/applications.py", line 113, in call
await self.middleware_stack(scope, receive, send)
File "root/.venv/lib/python3.9/site-packages/starlette/middleware/errors.py", line 187, in call
raise exc
File "root/.venv/lib/python3.9/site-packages/starlette/middleware/errors.py", line 165, in call
await self.app(scope, receive, _send)
File "root/.venv/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 62, in call
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "root/.venv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "root/.venv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "root/.venv/lib/python3.9/site-packages/starlette/routing.py", line 715, in call
await self.middleware_stack(scope, receive, send)
File "root/.venv/lib/python3.9/site-packages/starlette/routing.py", line 735, in app
await route.handle(scope, receive, send)
File "root/.venv/lib/python3.9/site-packages/starlette/routing.py", line 288, in handle
await self.app(scope, receive, send)
File "root/.venv/lib/python3.9/site-packages/starlette/routing.py", line 76, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "root/.venv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "root/.venv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "root/.venv/lib/python3.9/site-packages/starlette/routing.py", line 73, in app
response = await f(request)
File "root/.venv/lib/python3.9/site-packages/fastapi/routing.py", line 301, in app
raw_response = await run_endpoint_function(
File "root/.venv/lib/python3.9/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
return await dependant.call(**values)
File "root/app/main.py", line 138, in test1
segments, outputLang = await transcribeFile("tests/ja3mb.mp3", {})
File "root/app/main.py", line 154, in transcribeFile
transcribe = await loop.run_in_executor(executor, func_with_kwargs)
File "/Users/lw12838/.pyenv/versions/3.9.9/lib/python3.9/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "root/.venv/lib/python3.9/site-packages/whisper/transcribe.py", line 402, in transcribe
add_word_timestamps(
File "root/.venv/lib/python3.9/site-packages/whisper/timing.py", line 300, in add_word_timestamps
alignment = find_alignment(model, tokenizer, text_tokens, mel, num_frames, **kwargs)
File "root/.venv/lib/python3.9/site-packages/whisper/timing.py", line 197, in find_alignment
logits = model(mel.unsqueeze(0), tokens.unsqueeze(0))[0]
File "root/.venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "root/.venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
File "root/.venv/lib/python3.9/site-packages/whisper/model.py", line 296, in forward
return self.decoder(tokens, self.encoder(mel))
File "root/.venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "root/.venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
File "root/.venv/lib/python3.9/site-packages/whisper/model.py", line 242, in forward
x = block(x, xa, mask=self.mask, kv_cache=kv_cache)
File "root/.venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "root/.venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
File "root/.venv/lib/python3.9/site-packages/whisper/model.py", line 169, in forward
x = x + self.cross_attn(self.cross_attn_ln(x), xa, kv_cache=kv_cache)[0]
File "root/.venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "root/.venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1845, in _call_impl
return inner()
File "root/.venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1806, in inner
hook_result = hook(self, args, result)
File "root/.venv/lib/python3.9/site-packages/whisper/timing.py", line 189, in
lambda _, ins, outs, index=i: QKs.setitem(index, outs[-1][0])
TypeError: 'NoneType' object is not subscriptable

[Advait251206]

This looks much more like a concurrency/thread-safety issue than a problem with word_timestamps=True itself.

The strongest clue is:

It doesn't happen if max_workers=1

but fails when:

ThreadPoolExecutor(max_workers=3)

is used.

What's actually failing?

The exception occurs inside Whisper's word-alignment code:

File "whisper/timing.py", line 189

lambda _, ins, outs, index=i:
    QKs.__setitem__(index, outs[-1][0])

TypeError: 'NoneType' object is not subscriptable

This means:

outs is None

when the forward hook executes.

Normally Whisper expects:

outs

to contain attention outputs from the decoder.

Instead it receives:

None

which suggests that the internal state used by the alignment code has become corrupted or overwritten.

Why only with word_timestamps=True?

When you enable:

word_timestamps=True

Whisper performs an additional alignment pass:

transcription
    ↓
attention extraction
    ↓
token-word alignment

Internally, it registers temporary forward hooks on attention layers.

The relevant code path is roughly:

find_alignment(...)

which stores attention matrices in shared structures such as:

QKs

during model execution.

These mechanisms were not designed with concurrent inference in mind.

Most likely cause

You're sharing a single model instance:

getModel().transcribe(...)

across multiple threads.

Something like:

global_model = whisper.load_model(...)

Thread A -> transcribe()
Thread B -> transcribe()
Thread C -> transcribe()

works for normal transcription much of the time.

However, when word_timestamps=True:

Thread A registers hooks
Thread B registers hooks
Thread C registers hooks

↓
hooks overwrite each other's state

↓
alignment data becomes invalid

↓
outs == None

and the alignment code crashes.

Evidence

The traceback passes through:

add_word_timestamps()
find_alignment()

which are exactly the functions that use temporary hooks and attention extraction.

These are known to be much less concurrency-friendly than standard decoding.

Recommended fixes

Option 1: Serialize access to the model

Use a lock around transcription:

model_lock = threading.Lock()

def transcribe_safe(...):
    with model_lock:
        return model.transcribe(
            ...,
            word_timestamps=True
        )

This is the safest fix.

Option 2: One model per worker

Instead of:

1 model
3 threads

use:

Worker 1 -> Model 1
Worker 2 -> Model 2
Worker 3 -> Model 3

More memory usage, but avoids shared-state issues.

Option 3: Separate process workers

Prefer:

ProcessPoolExecutor

or multiple service processes over multiple threads sharing one Whisper model.

Speech models generally behave better with process isolation than with aggressive multithreading.

Option 4: Disable word timestamps

You've already observed:

word_timestamps=False

works.

This further supports the theory that the failure occurs in the alignment-specific code rather than decoding itself.

Additional note

Your code also does:

with default_thread_pool as executor:

inside the request handler.

That means the executor is being entered and exited repeatedly.

Typically you'd create:

default_thread_pool = ThreadPoolExecutor(...)

once at application startup and reuse it.

This is probably unrelated to the crash, but worth cleaning up.

My diagnosis

The most likely root cause is that OpenAI Whisper's word-timestamp alignment code is not thread-safe when multiple requests share the same model instance. The forward hooks used by find_alignment() interfere with each other under concurrent execution, eventually causing:

outs == None

and leading to:

TypeError: 'NoneType' object is not subscriptable

If you serialize access to the model (or use separate model instances/processes per worker), the issue will likely disappear.