Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve data structure #57

Open
cankisagun opened this issue Apr 8, 2020 · 1 comment
Open

Improve data structure #57

cankisagun opened this issue Apr 8, 2020 · 1 comment
Labels
Enclave help wanted Extra attention is needed rust

Comments

@cankisagun
Copy link
Contributor

Data is overwritten each time a user submits data - this can be improved but is hard. Currently user data is stored inside the enclave as a Rust HashMap indexed by the userId as its key, and the array of locations as its associated data. so everytime a new dataset is added to HashMap overwrites whatever prior entry was there for that key. Improving on this is hard because one would need to find data overlaps in terms of space and time with prior entries and do a proper merge.

@ainsleys ainsleys added the help wanted Extra attention is needed label Apr 9, 2020
@FishmanL
Copy link

Heh, I actually just had thoughts for this -- I'd recommend as a first pass splitting up the time/location stamps into 0.05 degree lat long square chunks in location and 15 minute increments of time, with user-data hashmaps stored at each point in that latXlongXts array (I'm assuming Rust can handle sparse vectors in O(1) with some sort of standard hashmap modification, otherwise this scales beautifully but has a constant cost of something like O(10^10) hashmaps created) (with 4 per 0.1 degree chunk, so you have overlap)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enclave help wanted Extra attention is needed rust
Projects
None yet
Development

No branches or pull requests

3 participants