You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Dear authors:
I read two of your articles on knowledge editing and benefited a lot.
Sorry to bother you, I have two questions I want to make sure of with you:
MEMIT's Equation 9, why specifically split into two items, looks like it can be combined into [1,u]. Does the specific split [1,n] and [n+1,u] have any special meanings?
ROME looks like it can also perform batch editing? The original paper is to calculate a set of [k*, v*] and then update W_proj. But what if I calculate [k*_1, k*_2...k*_n] with [v*_1, v*_2...v*_n], and then update W_proj? In this way, ROME can also be batch edited?
Looking forward to your reply, thank you
[1] Locating and Editing Factual Associations in GPT
[2] MASS-EDITING MEMORY IN A TRANSFORMER
The text was updated successfully, but these errors were encountered:
In MEMIT, authors said "In each individual layer l, we wish to store a large batch of u ≫ 1 memories. "u here means the new associations that we want the model to learn.
On the next line, authors said "assuming that the layer contains previously-stored memories that should be preserved." I believe i means the memorized associations that we want the model to preserve.
I guess the reason why author write like that is to emphasize the batch of associations updating. As we can see on author's previous work ROME (eq 2), they use :
Dear authors:
I read two of your articles on knowledge editing and benefited a lot.
Sorry to bother you, I have two questions I want to make sure of with you:
MEMIT's Equation 9, why specifically split into two items, looks like it can be combined into [1,u]. Does the specific split [1,n] and [n+1,u] have any special meanings?
![image](https://private-user-images.githubusercontent.com/20624157/243605891-1a61f0ac-4b86-44b3-9b36-cd205f999754.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTk1MzU3ODQsIm5iZiI6MTcxOTUzNTQ4NCwicGF0aCI6Ii8yMDYyNDE1Ny8yNDM2MDU4OTEtMWE2MWYwYWMtNGI4Ni00NGIzLTliMzYtY2QyMDVmOTk5NzU0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjI4VDAwNDQ0NFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTUyMjIwMzQxMjY3YjZhMTRhZDgxNjA3NjkwY2JkOGE3NzE5ZTc3NTQ5ODUzYjBjY2Q1NDA2MGQyOWEyNGZmNzkmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.ZKsBw70qS9PS9CuURT7deEibLOmtc9tclC9qV01g0I0)
ROME looks like it can also perform batch editing? The original paper is to calculate a set of [k*, v*] and then update W_proj. But what if I calculate [k*_1, k*_2...k*_n] with [v*_1, v*_2...v*_n], and then update W_proj? In this way, ROME can also be batch edited?
Looking forward to your reply, thank you
[1] Locating and Editing Factual Associations in GPT
[2] MASS-EDITING MEMORY IN A TRANSFORMER
The text was updated successfully, but these errors were encountered: