-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Dynamic Shape] More correctness guards #276
Conversation
Moreover, there is a subtle bug where the symbol registry gets rewritten multiple times if there exists inputs which promises the same shape, but runtime inputs have different shapes. So added a check for that as well. |
I have no idea why the gpt2 test passed before, there was a similar error with the llama implementation, where the compiled model reinterpreted an int64 tensor as an int32 tensor, producing random outputs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @Aalanli.
for i, (traced, new) in enumerate(zip(self.meta_data.inputs, inputs)): | ||
if ir.data_type(traced.dtype) != new.dtype: | ||
raise RuntimeError( | ||
f"dtype mismatch at arg {i} between original: {traced.dtype} and new: {new.dtype}" | ||
) | ||
traced_shape = traced.shape | ||
concrete_shape = new.shape | ||
if len(traced_shape) != len(concrete_shape): | ||
raise RuntimeError( | ||
f"Rank of input {i} not equal to original. ({len(concrete_shape)} vs. {len(traced_shape)})" | ||
) | ||
for j, (orig_shape, new_shape) in enumerate(zip(traced_shape, concrete_shape)): | ||
if isinstance(orig_shape, int) and orig_shape != new_shape: | ||
raise RuntimeError( | ||
f'shape mismatch at dimension {j}, original: \ | ||
{orig_shape} vs. new: {new_shape}' | ||
) | ||
elif orig_shape not in symbol_map: | ||
symbol_map[orig_shape] = new_shape | ||
elif symbol_map[orig_shape] != new_shape: | ||
raise RuntimeError( | ||
f"There exists multiple instances of the same symbol {orig_shape}\ | ||
with different values in inputs (ex: {symbol_map[orig_shape]} and {new_shape})" | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same logic as the CompiledGraph. Consider implement as a utility function and put it at the same module of TensorSignature
.
Co-authored-by: Yaoyao Ding <[email protected]>
Co-authored-by: Yaoyao Ding <[email protected]>
Co-authored-by: Yaoyao Ding <[email protected]>
Co-authored-by: Yaoyao Ding <[email protected]>
Thanks @Aalanli ! |
Add support for dynamic shape assertions in the C++ runtime.
Further, add shape and dtype check for python runtime. With this being enabled by default.