Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hello Defillama team!
I previously submitted a PR proposing the addition of a new tab page on this extension, with various other fixes/improvements. As discussed with @0xngmi , the fact that there is already a new tab page standalone extension and that adding this feature could be problematic for Chrome Web Store review, I decided to close the previous PR. But since I think my other performance improvements and code reorganization are still relevant, I submit to you this PR that contains only those. This introduces 0 changes to the end user, only changes that make the extension faster and the code (in my opinion) more maintainable.
Those are the changes :
IndexedDB/Dexie Databases
The phishing domains check uses IndexedDB managed by Dexie in the background script. The problem with this is that IndexedDB has a way of queueing requests that makes resolving results very slow during database updates (even when the db being updated is different from the one being read).
Some domain lists are pretty big (Metamask blocked lists is 97k entries), so the updates can last as long as 1min. You can notice the issue if you try to navigate when the extension has just been installed (and is going through its DB initialization). The logo will take a very long time to update when navigating to a new page.
New in this PR
The first approach that I implemented in my previous PR was to divide DB updates into chunks so the queries can come in between and be executed in reasonable time. The problem with this approach is that they can still take 2-10 seconds to resolve. So I added an additional optimisation, which consists in using the temporary arrays that are created by the update routine for searching the domain. Those arrays are in memory anyways while the script waits for the DBs to update, so using them doesn't have any cost. I also checked search time between those arrays and IndexedDB, and there is virtually no difference :
As you can see it's faster for small lists (allowed/fuzzy) and slower for the big blocked list, but we're talking 2ms vs 1ms, instead of 2-10 seconds.
Same as the Previous PR
I still kept update chunks (and increased their size to 1000) because it avoids doing massive DB updates with 100k+ lines, which is not recommended.
I also changed the way those updates are done, to avoid clearing the whole database every time. (this also prevents reading an empty database by accident).
Domains are now stored with an "updateId" key, that changes on each update. This way the update can bulkPut the domains that are fetched (overwriting existing ones and creating the new ones), and the old domains with an outdated updateId are removed after.
Background Script
I did various improvements to the Background script :
Minor stuff