Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Client cannot find deployment error #504

Open
heiseon opened this issue Jul 12, 2024 · 0 comments
Open

Client cannot find deployment error #504

heiseon opened this issue Jul 12, 2024 · 0 comments

Comments

@heiseon
Copy link

heiseon commented Jul 12, 2024

When I start the server in persistent mode, it runs very well. However, when I open the client, it reports the following error.

Traceback (most recent call last):
File "/root/miniconda3/lib/python3.11/site-packages/mii/config.py", line 392, in get_mii_config
mii_config = import_score_file(deployment_name, DeploymentType.LOCAL).mii_config
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.11/site-packages/mii/utils.py", line 111, in import_score_file
spec.loader.exec_module(score)
File "", line 936, in exec_module
File "", line 1073, in get_code
File "", line 1130, in get_data
FileNotFoundError: [Errno 2] No such file or directory: '/root/autodl-tmp/hf/hub/Qwen1.5-7B/score.py'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/root/miniconda3/lib/python3.11/site-packages/mii/config.py", line 397, in get_mii_config
mii_config = import_score_file(deployment_name,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.11/site-packages/mii/utils.py", line 111, in import_score_file
spec.loader.exec_module(score)
File "", line 936, in exec_module
File "", line 1073, in get_code
File "", line 1130, in get_data
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/mii_cache/Qwen1.5-7B-mii-deployment/score.py'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/root/autodl-tmp/llm/LLM-quickstart/deepspeed/vs/client2-7b.py", line 6, in
client = mii.client(model_path)
^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.11/site-packages/mii/api.py", line 109, in client
mii_config = get_mii_config(model_or_deployment_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.11/site-packages/mii/config.py", line 400, in get_mii_config
raise DeploymentNotFoundError(
mii.errors.DeploymentNotFoundError: Could not find a deployment named /root/autodl-tmp/hf/hub/Qwen1.5-7B or Qwen1.5-7B-mii-deployment

When I use curl to access the server, it returns results normally.
How should I troubleshoot the problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant