whisper 推理两路并发报错: Failed to asyn execute

一、问题现象：
按照whisper教程编译好medium模型之后, 用flask启动了一个服务, 单路推理没问题, 但是用jmeter 请求2路并发就有问题, 一请求直接报错, 报错如下:

2025-03-29 09:10:08.807 +08:00] [58217] [140164571588288] [rt] [ERROR] [ContextImpl.cpp:760] : Failed to asyn execute.                                               | 0/500 [00:00<?, ?frames/s]
  0%|                                                                                                                                                                | 0/500 [00:00<?, ?frames/s]
[2025-03-29 09:10:08,808] ERROR in app: Exception on /asr [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1511, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 919, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 917, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 902, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/test/whisper_base64_server.py", line 62, in asr
    result = asr_do(wave_data)
             ^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/test/whisper_base64_server.py", line 37, in asr_do
    result = model.transcribe(audio=wave_data,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/Whisper/whisper/whisper/transcribe.py", line 279, in transcribe
    result: DecodingResult = decode_with_fallback(mel_segment)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/Whisper/whisper/whisper/transcribe.py", line 195, in decode_with_fallback
    decode_result = model.decode(segment, options)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/Whisper/whisper/whisper/decoding.py", line 834, in decode
    result = DecodingTask(model, options).run(mel)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/Whisper/whisper/whisper/decoding.py", line 747, in run
    tokens, sum_logprobs, no_speech_probs = self._main_loop(audio_features, tokens)
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/Whisper/whisper/whisper/decoding.py", line 695, in _main_loop
    logits, cache_dyn, cache_sta = self.inference.logits(tokens, audio_features, cache_dyn_, cache_sta)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/Whisper/whisper/whisper/decoding.py", line 159, in logits
    logits, cache_dyn, _ = self.model.decoder(
                           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/Whisper/whisper/whisper/model.py", line 235, in forward
    logits, cache_dyn, _ = self.mindietorch_decode_model(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
  File "code/__torch__/whisper/model/___torch_mangle_1882.py", line 14, in forward
    __torch___whisper_model____torch_mangle_1882_TextDecoder_aie_engine_0 = getattr(input_0, "__torch__.whisper.model.___torch_mangle_1882.TextDecoder_aie_engine_0")
    _0 = [input_4, input_5, input_1, input_3]
    _1 = ops.mrt.execute_engine(_0, __torch___whisper_model____torch_mangle_1882_TextDecoder_aie_engine_0)
         ~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
    _2, _3, = _1
    return (_3, _2, input_5)

Traceback of TorchScript, original code (most recent call last):
RuntimeError: [ERROR thrown at ascend-inference-ptplugin/core/runtime/execute_engine.cpp:542] copy size error between host and device

10.144.214.68 - - [29/Mar/2025 09:10:08] "POST /asr HTTP/1.1" 500 -
begin asr....
                                                                                                                                                                                                [2025-03-29 09:10:10.787 +08:00] [58217] [140165100033728] [rt] [ERROR] [AIEStream.cpp:58] : Failed to synchronize stream, ret507015.                                 | 0/500 [00:00<?, ?frames/s]
  0%|                                                                                                                                                                | 0/500 [00:02<?, ?frames/s]
[2025-03-29 09:10:10,788] ERROR in app: Exception on /asr [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1511, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 919, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 917, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 902, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/test/whisper_base64_server.py", line 62, in asr
    result = asr_do(wave_data)
             ^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/test/whisper_base64_server.py", line 37, in asr_do
    result = model.transcribe(audio=wave_data,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/Whisper/whisper/whisper/transcribe.py", line 279, in transcribe
    result: DecodingResult = decode_with_fallback(mel_segment)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/Whisper/whisper/whisper/transcribe.py", line 195, in decode_with_fallback
    decode_result = model.decode(segment, options)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/Whisper/whisper/whisper/decoding.py", line 834, in decode
    result = DecodingTask(model, options).run(mel)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/Whisper/whisper/whisper/decoding.py", line 747, in run
    tokens, sum_logprobs, no_speech_probs = self._main_loop(audio_features, tokens)
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/Whisper/whisper/whisper/decoding.py", line 695, in _main_loop
    logits, cache_dyn, cache_sta = self.inference.logits(tokens, audio_features, cache_dyn_, cache_sta)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/Whisper/whisper/whisper/decoding.py", line 159, in logits
    logits, cache_dyn, _ = self.model.decoder(
                           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/lkq/vllm/modelzoo/Whisper/whisper/whisper/model.py", line 243, in forward
    cache_dyn = cache_dyn.cpu()
                ^^^^^^^^^^^^^^^
RuntimeError: [ERROR thrown at ascend-inference-ptplugin/core/npu_device/override_operators.cpp:126] SynchronizeStream error, please check AIE or ACL log

二、软件版本:
-- CANN 版本: 8.0.0  
--Pytorch 版本: 2.1.0+cpu
--Python 版本: 3.11.6
--操作系统版本: ubuntu 18.04

Ascend/ModelZoo-PyTorch

内容风险标识

评论 (0)

Ascend/ModelZoo-PyTorch .gitee-modal { width: 500px !important; }

内容风险标识