Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is it possible to crawl multiple ranges and append them to one locked file? #88

Open
mgutt opened this issue Oct 14, 2020 · 1 comment

Comments

@mgutt
Copy link

mgutt commented Oct 14, 2020

This works to preload head and tail of one file, but it is not persistent:

head -c 60000000 "$file" > /dev/null
tail -c 1000000 "$file" > /dev/null

Now I tried the same through vmtouch as follows:

size=$(stat -c%s $file)
vmtouch -l -p 0-60000000 "$file" &
sleep 2
start_byte="$(($size-1000000))"
vmtouch -l -p $start_byte-$size "$file" &

But I found out that the second command does not append the crawled range to the already locked file. Instead it seems to overwrite the existing one. I mean not really. I have still two different processes for the same file:

ps aux | awk '{print $6/1024 " MB\t\t" $11}' | sort -n | grep vmtouch
2020-10-14 18_45_50

Is there an other combination of commands/flags possible that allows me to lock both file parts?

@mgutt mgutt changed the title Is it possible to crawl/lock different ranges of one file? Is it possible to crawl multiple ranges and append them to one locked file? Oct 14, 2020
@hoytech
Copy link
Owner

hoytech commented Oct 15, 2020

I tried a quick test and it seems to work how I expect. I have a 6.7 GB file. In window 1:

$ sudo vmtouch -p 0-100M -l file
LOCKED 25600 pages (100M)

In window 2:

$ sudo vmtouch -p 6.6G- -l file
LOCKED 28778 pages (112M)

Window 3:

$ vmtouch -e file
           Files: 1
     Directories: 0
   Evicted Pages: 1758928 (6G)
         Elapsed: 0.036811 seconds
$ vmtouch -v file
file
[o                                                          o] 54378/1758928

           Files: 1
     Directories: 0
  Resident Pages: 54378/1758928  212M/6G  3.09%
         Elapsed: 0.069962 seconds

Note how the -e doesn't evict the ranges at the start and end.

I agree though, it would be much nicer if it was possible to do this in a single command!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants