Update Dockerfile with CUDA 11.8 #635
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
docker image里的cuda好像要升级一下，不然有这个报错: server log:2024-11-09 12:31:36 2024-11-09 20:31:36,085 INFO send inference response 2024-11-09 12:31:36 0%| | 0/1 [00:00<?, ?it/s]2024-11-09 20:31:36,093 INFO synthesis text 你好,我是通义生成式语音大模型,请问有什么可以帮您的吗? 2024-11-09 12:31:44 0%| | 0/1 [00:08<?, ?it/s] 2024-11-09 12:31:44 2024-11-09 20:31:44,105 ERROR Exception iterating responses: cuFFT error: CUFFT_INTERNAL_ERROR 2024-11-09 12:31:44 Traceback (most recent call last): 2024-11-09 12:31:44 File "/opt/conda/lib/python3.10/site-packages/grpc/_server.py", line 589, in _take_response_from_response_iterator 2024-11-09 12:31:44 return next(response_iterator), True 2024-11-09 12:31:44 File "/opt/CosyVoice/CosyVoice/runtime/python/grpc/server.py", line 62, in Inference 2024-11-09 12:31:44 for i in model_output: 2024-11-09 12:31:44 File "/opt/CosyVoice/CosyVoice/runtime/python/grpc/../../../cosyvoice/cli/cosyvoice.py", line 61, in inference_sft 2024-11-09 12:31:44 for model_output in self.model.tts(**model_input, stream=stream, speed=speed): 2024-11-09 12:31:44 File "/opt/CosyVoice/CosyVoice/runtime/python/grpc/../../../cosyvoice/cli/model.py", line 191, in tts 2024-11-09 12:31:44 this_tts_speech = self.token2wav(token=this_tts_speech_token, 2024-11-09 12:31:44 File "/opt/CosyVoice/CosyVoice/runtime/python/grpc/../../../cosyvoice/cli/model.py", line 138, in token2wav 2024-11-09 12:31:44 tts_speech, tts_source = self.hift.inference(speech_feat=tts_mel, cache_source=hift_cache_source) 2024-11-09 12:31:44 File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context 2024-11-09 12:31:44 return func(*args, **kwargs) 2024-11-09 12:31:44 File "/opt/CosyVoice/CosyVoice/runtime/python/grpc/../../../cosyvoice/hifigan/generator.py", line 410, in inference 2024-11-09 12:31:44 generated_speech = self.decode(x=speech_feat, s=s) 2024-11-09 12:31:44 File "/opt/CosyVoice/CosyVoice/runtime/python/grpc/../../../cosyvoice/hifigan/generator.py", line 350, in decode 2024-11-09 12:31:44 s_stft_real, s_stft_imag = self._stft(s.squeeze(1)) 2024-11-09 12:31:44 File "/opt/CosyVoice/CosyVoice/runtime/python/grpc/../../../cosyvoice/hifigan/generator.py", line 334, in _stft 2024-11-09 12:31:44 spec = torch.stft( 2024-11-09 12:31:44 File "/opt/conda/lib/python3.10/site-packages/torch/functional.py", line 641, in stft 2024-11-09 12:31:44 return _VF.stft(input, n_fft, hop_length, win_length, window, # type: ignore[attr-defined] 2024-11-09 12:31:44 RuntimeError: cuFFT error: CUFFT_INTERNAL_ERROR
|
这个好像有人提过一个pr，似乎是跟rtx显卡有关的问题，这个dockerfile是在v100上验证过的 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
docker image里的cuda好像要升级一下，不然有这个报错: server log:2024-11-09 12:31:36 2024-11-09 20:31:36,085 INFO send inference response 2024-11-09 12:31:36
0%| | 0/1 [00:00<?, ?it/s]2024-11-09 20:31:36,093 INFO synthesis text 你好,我是通义生成式语音大模型,请问有什么可以帮您的吗?
2024-11-09 12:31:44
0%| | 0/1 [00:08<?, ?it/s]
2024-11-09 12:31:44 2024-11-09 20:31:44,105 ERROR Exception iterating responses: cuFFT error: CUFFT_INTERNAL_ERROR
2024-11-09 12:31:44 Traceback (most recent call last):
2024-11-09 12:31:44 File "/opt/conda/lib/python3.10/site-packages/grpc/_server.py", line 589, in _take_response_from_response_iterator
2024-11-09 12:31:44 return next(response_iterator), True
2024-11-09 12:31:44 File "/opt/CosyVoice/CosyVoice/runtime/python/grpc/server.py", line 62, in Inference
2024-11-09 12:31:44 for i in model_output:
2024-11-09 12:31:44 File "/opt/CosyVoice/CosyVoice/runtime/python/grpc/../../../cosyvoice/cli/cosyvoice.py", line 61, in inference_sft
2024-11-09 12:31:44 for model_output in self.model.tts(**model_input, stream=stream, speed=speed):
2024-11-09 12:31:44 File "/opt/CosyVoice/CosyVoice/runtime/python/grpc/../../../cosyvoice/cli/model.py", line 191, in tts
2024-11-09 12:31:44 this_tts_speech = self.token2wav(token=this_tts_speech_token,
2024-11-09 12:31:44 File "/opt/CosyVoice/CosyVoice/runtime/python/grpc/../../../cosyvoice/cli/model.py", line 138, in token2wav
2024-11-09 12:31:44 tts_speech, tts_source = self.hift.inference(speech_feat=tts_mel, cache_source=hift_cache_source)
2024-11-09 12:31:44 File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
2024-11-09 12:31:44 return func(*args, **kwargs)
2024-11-09 12:31:44 File "/opt/CosyVoice/CosyVoice/runtime/python/grpc/../../../cosyvoice/hifigan/generator.py", line 410, in inference
2024-11-09 12:31:44 generated_speech = self.decode(x=speech_feat, s=s)
2024-11-09 12:31:44 File "/opt/CosyVoice/CosyVoice/runtime/python/grpc/../../../cosyvoice/hifigan/generator.py", line 350, in decode
2024-11-09 12:31:44 s_stft_real, s_stft_imag = self._stft(s.squeeze(1))
2024-11-09 12:31:44 File "/opt/CosyVoice/CosyVoice/runtime/python/grpc/../../../cosyvoice/hifigan/generator.py", line 334, in _stft
2024-11-09 12:31:44 spec = torch.stft(
2024-11-09 12:31:44 File "/opt/conda/lib/python3.10/site-packages/torch/functional.py", line 641, in stft
2024-11-09 12:31:44 return _VF.stft(input, n_fft, hop_length, win_length, window, # type: ignore[attr-defined]
2024-11-09 12:31:44 RuntimeError: cuFFT error: CUFFT_INTERNAL_ERROR