Skip to content

memory imports#124

Closed
eva-cosma wants to merge 4 commits intomainfrom
dev/global_store
Closed

memory imports#124
eva-cosma wants to merge 4 commits intomainfrom
dev/global_store

Conversation

@eva-cosma
Copy link
Copy Markdown
Collaborator

Pull Request Overview

This pull request adds/changes/fixes...

Testing Strategy

This pull request was tested by...

TODO or Help Wanted

This pull request still needs...

Formatting

  • Ran cargo fmt
  • Ran cargo check
  • Ran cargo build
  • Ran cargo doc
  • Ran nix fmt

Github Issue

This pull request closes <GITHUB_ISSUE>

@codecov
Copy link
Copy Markdown

codecov Bot commented Jan 29, 2025

Codecov Report

Attention: Patch coverage is 73.29545% with 94 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/execution/interpreter_loop.rs 64.05% 78 Missing ⚠️
src/execution/store.rs 26.66% 11 Missing ⚠️
src/core/reader/types/import.rs 0.00% 1 Missing and 1 partial ⚠️
src/execution/lut.rs 93.54% 1 Missing and 1 partial ⚠️
src/validation/mod.rs 95.45% 1 Missing ⚠️
Files with missing lines Coverage Δ
src/execution/execution_info.rs 100.00% <100.00%> (ø)
src/execution/mod.rs 92.59% <100.00%> (+0.65%) ⬆️
src/validation/code.rs 63.85% <100.00%> (+0.04%) ⬆️
src/validation/mod.rs 82.60% <95.45%> (+1.39%) ⬆️
src/core/reader/types/import.rs 26.31% <0.00%> (ø)
src/execution/lut.rs 96.10% <93.54%> (+0.10%) ⬆️
src/execution/store.rs 71.42% <26.66%> (-13.99%) ⬇️
src/execution/interpreter_loop.rs 90.26% <64.05%> (-1.98%) ⬇️

@eva-cosma eva-cosma marked this pull request as draft January 31, 2025 10:08
Comment thread src/validation/mod.rs
Comment on lines +130 to +132
if imported_memories.len() > 1 {
return Err(Error::MoreThanOneMemory);
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if imported_memories.len() > 1 {
return Err(Error::MoreThanOneMemory);
}
#[cfg(not(multiple_memories))] // maybe without this
if local_memories.len() + imported_memories.len() > 1 {
return Err(Error::MoreThanOneMemory);
}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the multiple memories proposal, can you have more than 1 local memory? Or is it multiple local, multiple imported?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking through the code, I disagree. We have bits in execution_info that MUST have a single memory otherwise it panics.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@george-cosma Just to be sure that I understand correctly:

Our code assumes that there must be at least on local memory? Or do you mean there must bet exactly one local memory?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple memories supports more than 1 local memory, in particular I quote:

Add a memory index to all memory-related instructions.

[1]

Comment thread src/validation/mod.rs
Comment on lines +118 to 120
if local_memories.len() > 1 {
return Err(Error::MoreThanOneMemory);
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if local_memories.len() > 1 {
return Err(Error::MoreThanOneMemory);
}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not under #cfg?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking through the code, I disagree. We have bits in execution_info that MUST have a single memory otherwise it panics.

Comment thread src/execution/mod.rs Outdated
code_expr,
// TODO fix this ugly clone
// TODO: Not used any more. Should we remove it?
sidetable: sidetable.clone(),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should remove this, otherwise there might be confusion

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cemonem agree?

Copy link
Copy Markdown
Contributor

@cemonem cemonem Mar 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, looks good, you can completely remove the field and the comment

Comment thread src/execution/mod.rs Outdated
.insert(module_name.to_string(), self.modules.len());
self.modules.push(exec_info);

let local_sidetables = validation_info
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since Sidetables are not supposed to be modified after initialization, would it be better if we'd attach them to the FunctionDeclarations?

Mr. Titzer does the exact same thing where the Sidetable is attached to its corresponding FuncDecl.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Letting them attatched to FunctionInst gives us headaches in terms of rust optimizations

What I am doing there is not to modify the sidetables, I am just shoving them in an array (and syncing it with function indexing by inserting dummy tables for imported functions which are not used anyway)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the sidetable could be a good candidate for an Arc, in theory. No one needs to alter them past parsing a wasm module, but we need them in various places. If we have multiple instances of a given wasm module, each needs the same sidetable. Alternatively, a shared reference is also no bad option, but of course than it leaks to the user, which then needs to store the sidetable in a way that allows creation of sufficiently long lived values to hand out the references.

Copy link
Copy Markdown
Contributor

@cemonem cemonem Mar 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eventually we will have a per module sidetable. In terms of interface, my intuition is that the sidetable per module and the wasm binary per module should essentially be always together within the interpreter loop, as the instruction pointer and stp is. We use WasmReader object for purposes other than having a context for what module we are executing and where the ip is (for example for parsing during validation). If this wasn't the case, like if there was an object, functionally the same as WasmReader but just used within the interpreter loop, i would chuck the sidetable within this object and sort of force the programmer to update stp and pc together and not individually, since everytime, especially when doing big jumps at pc we also need to update the stp so that it points to the sidetable entry pertaining to the next branching instruction that would be hit. In the case of who has references to it, we would essentially treat it the same as we treat the WasmReader object in the interpreter loop.

Comment thread src/execution/interpreter_loop.rs Outdated
.store
.mems
.first()
.first_mut()
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think we should shrink down the interpreter loop massively through the use of macros

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought you loved +4000 lines of code changed D:
Jokes aside, I agree that we should abstract away some parts of the interpreter loop, though I'm not sure I agree with macros.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check element / look at the get_mem function. Abstracting it any more isn't worthwhile imho. ctrl+f works fast enough

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we should investigate optimizing the interpreter loop. However, I vow to postpone this for the next phase of erasing technical debts. Lets not implement new features and refactor at the same time :)

@eva-cosma eva-cosma marked this pull request as ready for review February 4, 2025 19:36
George Cosma added 3 commits February 4, 2025 21:43
Signed-off-by: George Cosma <george.cosma@oxidos.io>
Signed-off-by: George Cosma <george.cosma@oxidos.io>
Signed-off-by: George Cosma <george.cosma@oxidos.io>
Signed-off-by: Cem Onem <cem.oenem@dlr.de>
@cemonem cemonem mentioned this pull request Mar 26, 2025
5 tasks
@cemonem
Copy link
Copy Markdown
Contributor

cemonem commented May 20, 2025

closing for now since we are going with global store approach.

@cemonem cemonem closed this May 20, 2025
@florianhartung florianhartung deleted the dev/global_store branch May 27, 2025 13:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants