Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[misc] add hint for AttributeError #5462

Merged
merged 3 commits into from
Jun 12, 2024
Merged

Conversation

youkaichao
Copy link
Member

try to fix #5456

Copy link
Collaborator

@mgoin mgoin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@youkaichao I like the error message, however needing to manually add a decorator for each function seems messy.

Could you possibly try this method of attaching the decorator to all functions in the _custom_ops module? I haven't tested this, but it should be something like this

def hint_on_error(fn):

    def wrapper(*args, **kwargs):
        try:
            return fn(*args, **kwargs)
        except AttributeError as e:
            msg = (
                "Error in calling custom op %s: %s\n"
                "Possibly you have built or installed an obsolete version of vllm.\n"
                "Please try a clean build and install of vllm,"
                "or remove old built files such as vllm/*.so and build/ .")
            logger.error(msg, fn.__name__, e)
            raise e

    return wrapper

import sys
import inspect

# Get the current module
current_module = sys.modules[__name__]

# Iterate through all functions in the current module
for name, obj in inspect.getmembers(current_module, inspect.isfunction):
    # Apply the decorator to each function
    setattr(current_module, name, hint_on_error(obj))

@youkaichao
Copy link
Member Author

@mgoin I updated the code with auto-decoration. here is the example message I get:

ERROR 06-12 12:21:06 _custom_ops.py:41] Error in calling custom op reshape_and_cache: '_OpNamespace' '_C_cache_ops' object has no attribute 'reshape_and_cache'
ERROR 06-12 12:21:06 _custom_ops.py:41] Possibly you have built or installed an obsolete version of vllm.
ERROR 06-12 12:21:06 _custom_ops.py:41] Please try a clean build and install of vllm,or remove old built files such as vllm/*.so and build/ .
[rank0]: Traceback (most recent call last):

I think this should be clear enough.

Copy link
Collaborator

@mgoin mgoin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job, I like your approach as a shim

@youkaichao youkaichao enabled auto-merge (squash) June 12, 2024 20:55
@youkaichao youkaichao merged commit 622d451 into vllm-project:main Jun 12, 2024
56 of 57 checks passed
@youkaichao youkaichao deleted the error_hint branch June 12, 2024 21:46
robertgshaw2-neuralmagic pushed a commit to neuralmagic/nm-vllm that referenced this pull request Jun 16, 2024
joerunde pushed a commit to joerunde/vllm that referenced this pull request Jun 17, 2024
xjpang pushed a commit to xjpang/vllm that referenced this pull request Jun 27, 2024
xjpang pushed a commit to xjpang/vllm that referenced this pull request Jul 8, 2024
xjpang pushed a commit to xjpang/vllm that referenced this pull request Jul 24, 2024
Temirulan pushed a commit to Temirulan/vllm-whisper that referenced this pull request Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Outdated binaries when re-building vLLM from source
2 participants