Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

key name in ip-adapter weight state_dict() #456

Open
shixishi opened this issue Feb 14, 2025 · 0 comments
Open

key name in ip-adapter weight state_dict() #456

shixishi opened this issue Feb 14, 2025 · 0 comments

Comments

@shixishi
Copy link

In tutorial_train.py, the code of IPAdapter Class
def load_from_checkpoint(self, ckpt_path: str):
# Calculate original checksums
orig_ip_proj_sum = torch.sum(torch.stack([torch.sum(p) for p in self.image_proj_model.parameters()]))
orig_adapter_sum = torch.sum(torch.stack([torch.sum(p) for p in self.adapter_modules.parameters()]))

    state_dict = torch.load(ckpt_path, map_location="cpu")

    # Load state dict for image_proj_model and adapter_modules
    self.image_proj_model.load_state_dict(state_dict["image_proj"], strict=True)
    self.adapter_modules.load_state_dict(state_dict["ip_adapter"], strict=True)

    # Calculate new checksums
    new_ip_proj_sum = torch.sum(torch.stack([torch.sum(p) for p in self.image_proj_model.parameters()]))
    new_adapter_sum = torch.sum(torch.stack([torch.sum(p) for p in self.adapter_modules.parameters()]))

    # Verify if the weights have changed
    assert orig_ip_proj_sum != new_ip_proj_sum, "Weights of image_proj_model did not change!"
    assert orig_adapter_sum != new_adapter_sum, "Weights of adapter_modules did not change!"

    print(f"Successfully loaded weights from checkpoint {ckpt_path}")

""
self.image_proj_model.load_state_dict(state_dict["image_proj"], strict=True)
self.adapter_modules.load_state_dict(state_dict["ip_adapter"], strict=True)
""
Why there are "image_proj" key name and "ip_adapter" key name in state_dict?
How are key names generated in state_dict?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant