vboxwrapper: create the 'virtualbox home directory' in the project dir #6018
Conversation
As far as I can tell this dir is used only on Win; we run VBoxSVC.exe there, and look for a log file later. Should we remove it for other platforms? Also: there's some code to create a 'scratch' dir, projects/scratch. This seems like a bad idea; we shouldn't put random stuff in projects/, and also if there are multiple VM jobs they share the same dir. Should we get rid of this?
There was a problem hiding this comment.
Copilot wasn't able to review any files in this pull request.
Files not reviewed (1)
- samples/vboxwrapper/vbox_vboxmanage.cpp: Language not supported
|
This is the latest result with the 26209 (compared to the code): 2025-01-22 22:54:38 (13555): 2025-01-22 22:54:38 (13555): Can someone supply me this file compiled for testing? |
|
I'm not sure the changes I've been making (involving the 'virtualbox home directory') On the Mac, BOINC jobs run as a 'hidden user', boinc_project. I'm confused.
|
Yes it did work. When I switch to the 26207b wrapper, tasks are running but failing due to network connection..
Here I have the trace and replay of the task: The task: |
|
I can't see (in those logs) any problems except the cvmfs connect failure, |
That is true. The error I sent earlier here (Command: VBoxManage -q list systemproperties) is with the new wrapper. I'm trying to debug it now on my Mac. I cannot find it. With automatic launch of the wrapper, the project fails immediately after start. https://lhcathome.cern.ch/lhcathome/result.php?resultid=419017986 New info, the VirtualBox home directory is changed. The folder is made (without the .). BOINC is not allowed to create .VirtualBox on Mac since this is system only! (VirtualBox is allowed) https://lhcathome.cern.ch/lhcathome/result.php?resultid=419018059 The folder for the VirtualBox settings need to be changed since VirtualBox places them in ~ of which is not allowed for other users. I'm tired now. Going to bed and will try to continue tomorrow with testing on this. |
|
I just archived a couple of logs from the LHC@home server before they disappear.
I remember even @NenTech had valids reported by the previous app versions using vboxwrapper 26207 (unfortunately the logs are not available any more). This recent result from @NenTech failed although vboxwrapper 26207 has been used: Very unlikely that it is caused by the vdi file since
Host internal networking appears to work since "Mounting the shared directory" succeeded. The Hypervisor System Log reports lots of this: Could it be the app version setup? Another log from @NenTech's computer reports this: Not sure why this points to the user's directory. |
davidpanderson
changed the title
|
don't merge for now |
|
@computezrmle I did the testing with normal user account and via BOINC. The results with all versions of the wrapper are the same via normal user account as the 26207b via BOINC. Found the error and I have it fixed in my version 26210! There is still 1 issue left. User group boinc_project is not allowed to create the directory VirtualBox in BOINC_data. |
I'm surprised, since I believe all project applications run as user boinc_master and group boinc_project, and are able to traverse the directory tree into the slots and project directories and successfully create files in those subdirectories. I don't see any indication in the referenced result log. How do you know that VBoxWrapper is failing to create the directory due to a permission error? |
This is based on my reading of the current setprojectgrp.cpp, which is invoked using switcher in set_to_project_group() in sandbox.cpp which in turn is called from both ACTIVE_TASK::start() and ACTIVE_TASK::setup_file() in app_start.cpp as well as other places. |
The 'VBox home directory' is where VBox writes log files (which are read by vboxwrapper). If this is not specified by the env var VBOX_USER_HOME, we need to create it somewhere and set the env var to point there. Previously we put it in <datadir>/projects. That's no good because it's not a project, and the client erased it. We also tried putting it in the (real) user's home dir. That's no good because 1) we shouldn't mess with the home dir 2) in sandboxed configs we're running as user 'boinc_projects', and don't have access to the home dir. According to https://boinc.berkeley.edu/sandbox_design.php, the only places 'boinc_projects' can write are projects/, slots/, and their subdirectories. So the logical places to put .VirtualBox are this job's slot directory, or its project directory. I chose the latter.
|
According to https://boinc.berkeley.edu/sandbox_design.php, I'm submitting a PR for vboxwrapper where it creates .VirtuaBox |
|
I think this should work, but I can't currently test it. |
davidpanderson
changed the title
|
This modification works on Linux with a couple of vbox tasks from LHC@home but I'm not happy writing the vbox logs to Bad scenario Assume you run a BOINC client attached to at least 2 vbox projects Now the bad thing - remove project abc from the BOINC client. Possible solution (not yet tested) |
|
BTW |
That information is out of date. I changed it in commit aecfe8e on November 8, 2022 as part of a fix to "Show graphics" from Manager for MacOS 13. Please see my comments here and here |
That's where the VirtualBox directory was before these changes. See #5658. The problem with that was that every time the BOINC client is restarted, it cleans the BOINC Data/projects/ directory by removing any subdirectories that are not a currently attached project. So the VirtualBox directory was deleted each time the BOINC client was restarted. |
|
Beyond the show-stopper computezrmle reported, the current vboxwrapper-26209 test version was also working OK on Windows. |
|
I didn't realize that different jobs shared the same VBoxSVC instance. In the above example, the more immediate problem is that the fgh job will look for the log in projects/fgh/.Virtualbox/, and it's actually in projects/abc/.Virtualbox. So - since according to Charlie the data directory is in fact writeable to the boinc_projects user - the best thing is to put .VirtualBox in the data directory. I'll make this change. |
Note: there's a single instance of VBoxSVC, not one per job.
|
Another thing to realize: As I understand, VirtualBox on Intel Macs can only emulate Linux for x86_64 machines, and VirtualBox on Apple Silicon macs (M1, M2, etc.) can only emulate Linux for arm64 machines. So any VirtualBox project applications must be matched to the architecture of the Mac on which they are run. |
|
Thanks, LGTM Once the artifacts are available |
Right. |
That is my expectation from reading the code. But @computezrmle wrote:
@computezrmle You haven't told us how you know that it's a permissions problem. Did you get an error message indicating a permissions problem? Might the failure to create the directory have been due to the environment variable not being set (PR #6019) and not a permissions issue at all? |
|
Just a heads up that I will be tied up with other matters and may not be able to read or respond to this discussion any more until after the coming weekend. |
The permissions issue was an assumption based on NenTech's task Logs 2 days ago. |
|
I just ran some tests and found that tasks (project applications) are running as user boinc_project and group boinc_project so files written by the project applications have user boinc_project and group boinc_project, but files written by the client have user group_master and group boinc_project. But the BOINC Data directory has the following permissions: Since the BOINC Data directory is not writable by others, I believe this may indeed mean that project applications don't have permission to create files or directories at the top level of the BOINC Data directory. One solution might be to have the BOINC client (or the BOINC Mac installer) create the "BOINC Data/.VirtualBox" directory with the same permissions as the projects and slots directories. This would also require additional code in the Mac_SA_Secure.sh script. However, I won't have time to update the Mac client or installer code until next week, so if @NenTech wants to test the current VBoxWrapper artifact I'd say go ahead. Here is the output from my tests: |
Sorry, I was wrong about it being out of date. That information is correct. I was confusing the ownership of the executable file on disk with the user under which it is run. I apologize for my confusion. |
|
I think I have a simpler solution. The original problem #5658 was that the VirtualBox directory was in the Projects subdirectory, and every time the BOINC client is restarted, it cleans the "BOINC Data/projects/" directory by removing any subdirectories that are not a currently attached project. So the VirtualBox directory was deleted each time the BOINC client was restarted. The simplest solution might be to modify the BOINC client code which cleans up the "BOINC Data/projects/" directory to recognize the VirtualBox directory as a special case and not delete it. Then we can revert the location of the VirtualBox directly to "/Library/Application Support/BOINC Data/VirtualBox/" (with or without the leading period.) I recommend that it be VirtualBox as before and not .VirtualBox so it is not a hidden file, for ease of user support and debugging. And this has the added bonus of providing backward compatibility with earlier versions of VBoxWrapper. What do you all think? @davidanderson can you make those changes? Unfortunately, due to Apple's security restrictions I'll have to build the installer from this branch and run it through Apple's "Notarization" process before @NenTech can run it to test it. I won;t have time to do that until after the weekend. |
|
I checked the permission issue in BOINC_data simply by viewing if the folder was made. After I added boinc_project to the BOINC_data folder, the wrapper was be able to make the folder. I think indeed that the best solution would be to put the VirtualBox folder without leading . to the projects folder. |
|
Oops - I just read charlie's earlier comment.
Is this correct? If so, what's the lesser of these evils? |
|
BTW, the next version of the client will not remove unrecognized stuff in projects/ |
Yes, it worked fine for LHC@home until last week. |
|
Any user can traverse the top level BOINC Data directory (user, group and other all have x permission) but only user and group boinc_master can write to it. So neither user nor group boinc_project can create a directory there. But after traversing the top level directory they can write into and create files and directories in the projects and slots directories which have user boinc_master and group boinc_project. |
|
Yeah. But we don't want to put .Virtualbox in either projects/ or slots/, |
|
From my point of view the right place should be I agree with Charlie Fenton that it shouldn't be a hidden directory (hiding it doesn't make sense). For older clients it may be fine to tell the volunteers to manually create the directory and set the permissions. |
|
So it worked for LHC before last week. Good. OK, it put it in projects/. The path of least resistance is to continue putting it there. |
That's pretty much the only option. boinc_projects can't write to data dir. Older clients will delete this on startup; new clients won't.
They used an artifact from here based on v26207 +PRs up to #5608.
In projects/ |
That of course would be ideal. But isn't there a way for projects to send certain tasks only to clients with a minimum version number? They could tell clients to upgrade BOINC in Notices. I've seen messages in the Event Log saying that a project has tasks available but not for my computer or other criteria. Is there such message saying it has tasks for a newer version of BOINC? |
|
Yes, but I'd like to minimize complexity here. |
|
I tested 2 other scenarios with vboxwrapper v26209 on Windows with creating the .VirtualBox directory under e.g. /projects/lhcathome.cern.ch_lhcathome
2025-01-25 14:15:01 (1548): 2025-01-25 14:15:54 (1548): 2025-01-25 14:16:47 (1548): 2025-01-25 14:17:39 (1548): 2025-01-25 14:18:31 (1548): 2025-01-25 14:19:19 (1548): 2025-01-25 14:20:11 (1548): 2025-01-25 14:21:03 (1548): 2025-01-25 14:21:56 (1548): 2025-01-25 14:22:53 (1548): 2025-01-25 14:23:40 (1548): 2 clients could be usefull, because of differentiating sub-projects like LHC's Theory. ATLAS and CMS or 3rd scenario: two different projects using VM's. I started testing this but Rosetta does not have VBox-tasks at the moment. |
|
3rd scenario: Running VMs of different projects: When you have a running VM of project 1 and start a VBox-task of project 2, the VBox-xml and -log of project 1 are used. Important remark: With VirtualBox Manager the BOINC VMs are not shown! So suspending both tasks again and open VirtualBox Manager (empty list of VMs) and resuming the BOINC-tasks, |
|
The wrapper does work. I still have the issue with the failing tasks. I think this is also part of the folder permissions. But for now, I still can not find the bug.
|
The 'VBox home directory' is where VBox writes log files
(which are read by vboxwrapper).
If this is not specified by the env var VBOX_USER_HOME,
we need to create it somewhere and set the env var to point there.
Previously we put it in /projects.
That's no good because it's not a project, and the client erased it.
We also tried putting it in the (real) user's home dir.
That's no good because
and don't have access to the home dir.
According to https://boinc.berkeley.edu/sandbox_design.php,
the only places 'boinc_projects' can write are
projects/, slots/, and their subdirectories.
So the logical places to put .VirtualBox are
this job's slot directory, or its project directory.
I chose the latter.