-
Notifications
You must be signed in to change notification settings - Fork 19.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement tool for saved Keras model file inspection, diff, and patching. #19705
Comments
Sure, you are welcome to work on that. Do you have any experience with web development? I'm thinking this tool may benefit from an interactive js/html interface to be used in a notebook. |
I'm in the third year of computer science and engineering, so we already had to use JS and HTML for some projects, and I already have some knowledge of web development from some side projects I have done and am currently doing. I will also be doing this with @pedro-curto, who is in the same year and university as I am. |
Sounds great. Here's an example of a draft I wrote a long time ago, displaying a kind of summary of a file's content: def inspect_file(
filepath, reference_model=None, custom_objects=None, print_fn=print
):
filepath = str(filepath)
if filepath.endswith(".keras"):
with zipfile.ZipFile(filepath, "r") as zf:
print_fn(f"Keras model file '{filepath}'")
with zf.open(_CONFIG_FILENAME, "r") as f:
config = json.loads(f.read())
print_fn(
f"Model: {config['class_name']} name='{config['config']['name']}'"
)
if reference_model is None:
reference_model = deserialize_keras_object(
config, custom_objects=custom_objects
)
with zf.open(_METADATA_FILENAME, "r") as f:
metadata = json.loads(f.read())
print_fn(f"Saved with Keras {metadata['keras_version']}")
print_fn(f"Date saved: {metadata['date_saved']}")
archive = zipfile.ZipFile(filepath, "r")
weights_store = H5IOStore(
_VARS_FNAME + ".h5", archive=archive, mode="r"
)
print_fn("Weights file:")
inspect_nested_dict(weights_store.h5_file, print_fn, prefix=" ")
elif filepath.endswith(".weights.h5"):
print_fn(f"Keras weights file '{filepath}'")
weights_store = H5IOStore(
_VARS_FNAME + ".h5", archive=archive, mode="r"
)
inspect_nested_dict(weights_store.h5_file, print_fn)
else:
raise ValueError(
"Invalid filename: expected a `.keras` `.weights.h5` extension. "
f"Received: filepath={filepath}"
)
def inspect_nested_dict(store, print_fn=print, prefix=""):
for key in store.keys():
value = store[key]
if hasattr(value, "keys"):
skip = False
if (
list(value.keys()) == ["vars"]
and len(value["vars"].keys()) == 0
):
skip = True
if key == "vars" and len(value.keys()) == 0:
skip = True
if not skip:
print_fn(f"{prefix}{key}")
inspect_nested_dict(value, print_fn, prefix=prefix + " ")
if key == "vars":
for k in value.keys():
w = value[k]
print_fn(f"{prefix} {k}: {w.shape} {w.dtype}") (It relies on objects from I think we want the following features:
What do you think? |
That sounds fantastic! |
Sure, you can just ask questions in this thread. |
Hi @fchollet, Thank you very much for your assistance. |
Hello!
I saw this feature on "🚀 Contributing to Keras 🚀" and I want to know If I can start developing it.
The tool can:
The text was updated successfully, but these errors were encountered: