Skip to content

increase the size of uvm memory allocation #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
+25 −16
Open

increase the size of uvm memory allocation #11

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
0de451b
increase the size of uvm memory allocation
0x5459 Sep 17, 2024
7632b7c
Correctly set the value of UVM_KMALLOC_THRESHOLD
0x5459 Nov 4, 2024
32b874e
use `MAX_MCACHE_ORDER` instead of hard-coded
0x5459 Nov 13, 2024
d340014
Correctly call vfree
0x5459 Nov 13, 2024
4bdb132
Export the uvm_vmalloc_hdr_t structure to nv-nanos.h. And modify vfre…
0x5459 Nov 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion kernel-open/nvidia-uvm/uvm_kvmalloc.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -257,7 +257,7 @@ static void *alloc_internal(size_t size, bool zero_memory)
// Make sure that (sizeof(hdr) + size) is what it should be
BUILD_BUG_ON(sizeof(uvm_vmalloc_hdr_t) != offsetof(uvm_vmalloc_hdr_t *, ptr));

assert(size <= (1 << 16));
assert(size <= (1 << 32));
Copy link
Member

@francescolavra francescolavra Sep 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The (1 << 16) limit is there because the kernel heap used for these allocations supports a malloc-style interface (where you don't need to specify a size value when freeing memory) only for allocations sizes up to (1 << 16) (see the MAX_MCACHE_ORDER constant in the Nanos source at src/config.h). So it cannot be changed to a larger value.
If we want to remove this limit, we need to be able to use the vmalloc API when allocations larger that (1 << MAX_MCACHE_ORDER) are requested. This would entail:

  • changing the UVM_KMALLOC_THRESHOLD definition in kernel_open/nvidia_uvm/uvm_kvmalloc.h from infinity to (1 << MAX_MCACHE_ORDER)
  • changing the is_vmalloc_addr() definition in kernel_open/common/inc/nv_nanos.h from false to (objcache_from_object(u64_from_pointer(p), PAGESIZE_2M) == INVALID_ADDRESS)
  • changing the vfree() definition in kernel_open/common/inc/nv_nanos.h from kfree to NV_KFREE; this means the macro will take an additional size parameter, for which the macro invocations can use the alloc_size value of the corresponding uvm_vmalloc_hdr_t struct (all vmalloc allocations use this header)

if (size <= UVM_KMALLOC_THRESHOLD) {
if (zero_memory)
return kzalloc(size, NV_UVM_GFP_FLAGS);
Expand Down