generated from PoCInnovation/open-source-project-template
-
Notifications
You must be signed in to change notification settings - Fork 0
Feat/api integration #3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 30 commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
f6296cf
added test repository to figure out how to create a functionnal vm
skl1017 041fc30
feat: (wip) added new user-data configuration that provides network c…
skl1017 f1dd24d
fix: fixed network connectivity problem by using standard interface, …
skl1017 7dfc8be
feat: added guest agent and xubuntu-desktop to cloud-init, resized qc…
skl1017 eacfc47
feat: added libvirt dependencies for python in libvirt-install.sh
skl1017 bc57258
feat: changed create-vm.sh to create-distribox-image.sh, and added cl…
skl1017 fdc18d4
fix: changed permissions and groups in init script, added distribox g…
skl1017 e9df176
fix: added removal of seed.iso and os.qcow2
skl1017 110d182
feat(api): added requirements.txt
skl1017 92279e0
feat(api): added pydantic vm models
skl1017 09cfd15
feat(api): added core folder with constants and config files
skl1017 3b4ab62
feat(api): added build_xml function to core utilities
skl1017 783f7ae
feat(api): added vm service, with create_vm and start_vm methods
skl1017 8d92159
feat: added 2 routes, one for vm creation and another for vm start
skl1017 da83d84
Merge branch 'main' into feat/api/create-vm
skl1017 65d9de6
Merge branch 'main' into feat/api/create-vm
skl1017 0b3f901
feat(api): added an ORM model for vms and finished developping endpoi…
skl1017 94276fd
feat(api): GET /vms/:id endpoint
skl1017 8b15c7c
feat(api): added POST /vm/:id/start endpoint and changed config to ma…
skl1017 3eb35df
Merge branch 'feat/api/create-vm' into feat/api-integration
skl1017 a4fbfef
Merge branch 'feat/api/get-vm' into feat/api-integration
skl1017 52190fc
Merge branch 'feat/api/start-vm' into feat/api-integration
skl1017 3725efc
feat(api): added routes for /vms
skl1017 09dd932
feat(api): added router for /vms
skl1017 4cf8377
feat(api): added POST /vm/:id/stop endpoint
skl1017 b6376d9
fix(api): removed status field from vms table, as it is a constantly …
skl1017 d0111a2
Merge branch 'feat/api-integration' of github.com:PoCInnovation/Distr…
skl1017 58964cb
feat(api): added error handling for resources not found
skl1017 8e2df96
feat(api): added get_all vms endpoint (/vms)
skl1017 f6c6956
feat(api): added PUT /:id/password to generate and set random passwor…
skl1017 4d257fd
fix: Change VM_STATE_NAMES for blocked state to English
lg-epitech File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| import libvirt | ||
| from app.orm.vm import VmORM | ||
| from dotenv import load_dotenv | ||
| from os import getenv | ||
| from sqlmodel import create_engine, SQLModel | ||
|
|
||
| load_dotenv() | ||
|
|
||
| db_name = getenv("POSTGRES_NAME", "distribox") | ||
| db_user = getenv("POSTGRES_USER", "distribox_user") | ||
| db_pass = getenv("POSTGRES_PASSWORD", "distribox_password") | ||
| db_port = getenv("POSTGRES_PORT", "5432") | ||
|
|
||
| database_url = f"postgresql+psycopg2://{db_user}:{db_pass}@localhost:{db_port}/{db_name}" | ||
| engine = create_engine(database_url, echo=True) | ||
| SQLModel.metadata.create_all(engine) | ||
|
|
||
| class QEMUConfig: | ||
| qemu_conn = None | ||
|
|
||
| @classmethod | ||
| def get_connection(cls): | ||
| if cls.qemu_conn is None or cls.qemu_conn.isAlive() == 0: | ||
| try: | ||
| qemu_conn = libvirt.open("qemu:///system") | ||
| except libvirt.libvirtError: | ||
| raise | ||
| return qemu_conn | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| from pathlib import Path | ||
| import libvirt | ||
|
|
||
| BASE_DIR = Path('/var/lib/distribox/') | ||
| VMS_DIR = BASE_DIR / 'vms' | ||
| IMAGES_DIR = BASE_DIR / 'images' | ||
|
|
||
| VM_STATE_NAMES = { | ||
| libvirt.VIR_DOMAIN_NOSTATE: "No state", | ||
| libvirt.VIR_DOMAIN_RUNNING: "Running", | ||
| libvirt.VIR_DOMAIN_BLOCKED: "Bloqué sur E/S", | ||
| libvirt.VIR_DOMAIN_PAUSED: "Paused", | ||
| libvirt.VIR_DOMAIN_SHUTDOWN: "Stopping", | ||
| libvirt.VIR_DOMAIN_SHUTOFF: "Stopped", | ||
| libvirt.VIR_DOMAIN_CRASHED: "Crashed", | ||
| libvirt.VIR_DOMAIN_PMSUSPENDED: "Suspended (power management)", | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| from lxml import etree | ||
| from app.models.vm import VmRead | ||
| from app.core.constants import VMS_DIR | ||
|
|
||
| def build_xml(vm_read: VmRead): | ||
|
|
||
| domain = etree.Element("domain", type="kvm") | ||
|
|
||
| etree.SubElement(domain, "name").text = str(vm_read.id) | ||
| etree.SubElement(domain, "memory", unit="MiB").text = str(vm_read.mem) | ||
| etree.SubElement(domain, "vcpu", placement="static").text = str(vm_read.vcpus) | ||
|
|
||
| os_el = etree.SubElement(domain, "os") | ||
| etree.SubElement(os_el, "type", arch="x86_64", machine="pc").text = "hvm" | ||
| etree.SubElement(os_el, "boot", dev="hd") | ||
|
|
||
| features = etree.SubElement(domain, "features") | ||
| for feature in ["acpi", "apic", "pae"]: | ||
| etree.SubElement(features, feature) | ||
|
|
||
| etree.SubElement(domain, "cpu", mode="host-passthrough") | ||
|
|
||
| etree.SubElement(domain, "clock", offset="utc") | ||
|
|
||
| etree.SubElement(domain, "on_poweroff").text = "destroy" | ||
| etree.SubElement(domain, "on_reboot").text = "restart" | ||
| etree.SubElement(domain, "on_crash").text = "destroy" | ||
|
|
||
| devices = etree.SubElement(domain, "devices") | ||
|
|
||
| disk_main = etree.SubElement(devices, "disk", type="file", device="disk") | ||
| etree.SubElement(disk_main, "driver", name="qemu", type="qcow2") | ||
| etree.SubElement(disk_main, "source", file=str(VMS_DIR / str(vm_read.id) / f"distribox-{vm_read.os}.qcow2")) | ||
| etree.SubElement(disk_main, "target", dev="vda", bus="virtio") | ||
|
|
||
| channel = etree.SubElement(devices, "channel", type="unix") | ||
| etree.SubElement(channel, "source", mode="bind") | ||
| etree.SubElement(channel, "target", type="virtio", name="org.qemu.guest_agent.0") | ||
|
|
||
| # disk_seed = etree.SubElement(devices, "disk", type="file", device="cdrom") | ||
| # etree.SubElement(disk_seed, "driver", name="qemu", type="raw") | ||
| # etree.SubElement(disk_seed, "source", file="/var/lib/distribox/images/seed.iso") | ||
| # etree.SubElement(disk_seed, "target", dev="hdb", bus="ide") | ||
| # etree.SubElement(disk_seed, "readonly") | ||
|
|
||
| iface = etree.SubElement(devices, "interface", type="network") | ||
| etree.SubElement(iface, "source", network="default") | ||
| etree.SubElement(iface, "model", type="virtio") | ||
|
|
||
| etree.SubElement(devices, "graphics", type="vnc", port="-1", autoport="yes", listen="127.0.0.1") | ||
|
|
||
| etree.SubElement(devices, "console", type="pty") | ||
| etree.SubElement(devices, "input", type="keyboard", bus="ps2") | ||
| etree.SubElement(devices, "input", type="tablet", bus="usb") | ||
|
|
||
| xml_string = etree.tostring(domain, pretty_print=True, encoding="utf-8").decode() | ||
| return xml_string |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,15 +1,20 @@ | ||
| from typing import Union | ||
|
|
||
| from fastapi import FastAPI | ||
|
|
||
| from fastapi import FastAPI, HTTPException | ||
| from fastapi.responses import JSONResponse | ||
| from app.routes import vm | ||
| app = FastAPI() | ||
|
|
||
|
|
||
| @app.get("/") | ||
| def read_root(): | ||
| return {"Hello": "World"} | ||
|
|
||
| @app.exception_handler(HTTPException) | ||
| async def global_exception_handler(_, exc: HTTPException): | ||
| return JSONResponse( | ||
| status_code=exc.status_code, | ||
| content={"detail": exc.detail} | ||
| ) | ||
|
|
||
| @app.get("/items/{item_id}") | ||
| def read_item(item_id: int, q: Union[str, None] = None): | ||
| return {"item_id": item_id, "q": q} | ||
| @app.exception_handler(Exception) | ||
| async def global_exception_handler(_, exc: Exception): | ||
| return JSONResponse( | ||
| status_code=500, | ||
| content={"detail": str(exc)} | ||
| ) | ||
| app.include_router(vm.router, prefix="/vms") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| from pydantic import BaseModel | ||
| from typing import Optional | ||
|
|
||
| class VmBase(BaseModel): | ||
| os: str | ||
| mem: int | ||
| vcpus: int | ||
| disk_size: int | ||
|
|
||
| class VmRead(VmBase): | ||
| id: str | ||
| state: str | ||
|
|
||
| class VmCreate(VmBase): | ||
| pass |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| from sqlmodel import SQLModel, Field | ||
| import uuid | ||
|
|
||
| class VmORM(SQLModel, table=True, ): | ||
| __tablename__ = "vms" | ||
|
|
||
| id: uuid.UUID = Field(default_factory=uuid.uuid4, primary_key=True) | ||
| os: str | ||
| mem: int | ||
| vcpus: int | ||
| disk_size: int | ||
| password: str | None = None |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| from fastapi import status, APIRouter | ||
| from app.models.vm import VmCreate | ||
| from app.services.vm_service import VmService | ||
|
|
||
| router = APIRouter() | ||
|
|
||
| @router.get('/', status_code=status.HTTP_200_OK) | ||
| def get_vm_list(): | ||
| vm_list = VmService.get_vm_list() | ||
| return vm_list | ||
|
|
||
| @router.get("/{vm_id}", status_code=status.HTTP_200_OK) | ||
| def get_vm(vm_id: str): | ||
| vm = VmService.get_vm(vm_id) | ||
| return vm | ||
|
|
||
| @router.get("/{vm_id}/state", status_code=status.HTTP_200_OK) | ||
| def get_vm(vm_id: str): | ||
| state = VmService.get_state(vm_id) | ||
| return state | ||
|
|
||
| @router.post("/{vm_id}/start", status_code=status.HTTP_200_OK) | ||
| def start_vm(vm_id: str): | ||
| vm = VmService.start_vm(vm_id) | ||
| return vm | ||
|
|
||
| @router.post("/{vm_id}/stop", status_code=status.HTTP_200_OK) | ||
| def stop_vm(vm_id: str): | ||
| vm = VmService.stop_vm(vm_id) | ||
| return vm | ||
|
|
||
| @router.post("/", status_code=status.HTTP_201_CREATED) | ||
| def create_vm(vm:VmCreate): | ||
| created_vm = VmService.create_vm(vm) | ||
| return created_vm | ||
|
|
||
| @router.put("/{vm_id}/password", status_code=status.HTTP_200_OK) | ||
| def set_vm_password(vm_id): | ||
| return VmService.set_vm_password(vm_id) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,172 @@ | ||
| import uuid | ||
| import shutil | ||
| import libvirt | ||
| import hashlib | ||
| from app.utils.vm import wait_for_state | ||
| from typing import Optional | ||
| from app.core.constants import VMS_DIR, IMAGES_DIR, VM_STATE_NAMES | ||
| from app.models.vm import VmCreate | ||
| from app.core.xml_builder import build_xml | ||
| from app.core.config import QEMUConfig, engine | ||
| from sqlmodel import Session, select, update | ||
| from app.orm.vm import VmORM | ||
| from fastapi import status, HTTPException | ||
|
|
||
| class Vm: | ||
| def __init__(self, vm_create: VmCreate): | ||
| self.id = uuid.uuid4() | ||
| self.os = vm_create.os | ||
| self.mem = vm_create.mem | ||
| self.vcpus = vm_create.vcpus | ||
| self.disk_size = vm_create.disk_size | ||
| self.state: Optional[str] = None | ||
|
|
||
|
|
||
| def create(self): | ||
| if self.state : | ||
| return self | ||
| self.state = 'Stopped' | ||
| vm_dir = VMS_DIR / str(self.id) | ||
| distribox_image_dir = IMAGES_DIR / f"distribox-{self.os}.qcow2" | ||
| try: | ||
| vm_dir.mkdir(parents=True, exist_ok=True) | ||
| shutil.copy(distribox_image_dir, vm_dir) | ||
| vm_xml = build_xml(self) | ||
| conn = QEMUConfig.get_connection() | ||
| conn.defineXML(vm_xml) | ||
| with Session(engine) as session: | ||
| vm_record = VmORM( | ||
| id = self.id, | ||
| os = self.os, | ||
| mem = self.mem, | ||
| vcpus = self.vcpus, | ||
| disk_size= self.disk_size, | ||
| ) | ||
| session.add(vm_record) | ||
| session.commit() | ||
| return self | ||
| except Exception: | ||
| raise | ||
|
|
||
| @classmethod | ||
| def get(cls, vm_id: str): | ||
| try: | ||
| conn = QEMUConfig.get_connection() | ||
| vm = conn.lookupByName(vm_id) | ||
| vm_state, _ = vm.state() | ||
| with Session(engine) as session: | ||
| vm_record = session.get(VmORM, uuid.UUID(vm_id)) | ||
| vm_instance = cls.__new__(cls) | ||
| vm_instance.id = vm_record.id | ||
| vm_instance.os = vm_record.os | ||
| vm_instance.mem = vm_record.mem | ||
| vm_instance.vcpus = vm_record.vcpus | ||
| vm_instance.disk_size =vm_record.disk_size | ||
| vm_instance.state = VM_STATE_NAMES.get(vm_state, 'None') | ||
| except libvirt.libvirtError as e: | ||
| if e.get_error_code() == libvirt.VIR_ERR_NO_DOMAIN: | ||
| raise HTTPException(status.HTTP_404_NOT_FOUND, f'Vm {vm_id} not found') | ||
| except Exception: | ||
| raise | ||
| return vm_instance | ||
|
|
||
| @classmethod | ||
| def get_all(cls): | ||
| try: | ||
| with Session(engine) as session: | ||
| statement = select(VmORM) | ||
| vm_records = session.scalars(statement).all() | ||
| vm_list = [] | ||
| for vm_record in vm_records: | ||
| vm_list.append(cls.get(str(vm_record.id))) | ||
| return vm_list | ||
| except Exception: | ||
| raise | ||
|
|
||
| def start(self): | ||
| try: | ||
| conn = QEMUConfig.get_connection() | ||
| vm = conn.lookupByName(str(self.id)) | ||
| if vm.isActive() == 0: | ||
| vm.create() | ||
| except libvirt.libvirtError as e: | ||
| if e.get_error_code() == libvirt.VIR_ERR_NO_DOMAIN: | ||
| raise HTTPException(status.HTTP_404_NOT_FOUND, f'Vm {vm.id} not found') | ||
| except Exception: | ||
| raise | ||
| state_code = wait_for_state(vm, libvirt.VIR_DOMAIN_RUNNING, 0.5, 10) | ||
| self.state=VM_STATE_NAMES.get(state_code, 'None') | ||
| return self | ||
|
|
||
| def stop(self): | ||
| try: | ||
| conn = QEMUConfig.get_connection() | ||
| vm = conn.lookupByName(str(self.id)) | ||
| if vm.isActive() == 1: | ||
| vm.shutdown() | ||
| except libvirt.libvirtError as e: | ||
| if e.get_error_code() == libvirt.VIR_ERR_NO_DOMAIN: | ||
| raise HTTPException(status.HTTP_404_NOT_FOUND, f'Vm {vm.id} not found') | ||
| except Exception: | ||
| raise | ||
| state_code = wait_for_state(vm, libvirt.VIR_DOMAIN_SHUTOFF, 5, 10) | ||
| self.state=VM_STATE_NAMES.get(state_code, 'None') | ||
| return self | ||
|
|
||
| def get_state(self): | ||
| try: | ||
| conn = QEMUConfig.get_connection() | ||
| vm = conn.lookupByName(str(self.id)) | ||
| state, _ = vm.state() | ||
| except Exception: | ||
| raise | ||
| return {"state" : VM_STATE_NAMES.get(state, 'None')} | ||
|
|
||
| def generate_password(self): | ||
| try: | ||
| random_uuid = str(uuid.uuid4()) | ||
| uuid_bytes = random_uuid.encode('utf-8') | ||
| hasher = hashlib.sha256() | ||
| hasher.update(uuid_bytes) | ||
| password = hasher.hexdigest() | ||
| with Session(engine) as session: | ||
| statement = update(VmORM).where(VmORM.id == self.id).values(password=password) | ||
| session.exec(statement) | ||
| session.commit() | ||
| return {"password": password} | ||
| except Exception: | ||
| raise | ||
|
|
||
|
|
||
| class VmService: | ||
|
|
||
| def get_vm_list(): | ||
| vm = Vm.get_all() | ||
| return vm | ||
| def get_vm(vm_id: str): | ||
| vm = Vm.get(vm_id) | ||
| return vm | ||
|
|
||
| def get_state(vm_id:str): | ||
| vm = Vm.get(vm_id) | ||
| state = vm.get_state() | ||
| return state | ||
|
|
||
| def create_vm(vm_create: VmCreate): | ||
| vm = Vm(vm_create) | ||
| vm.create() | ||
| return vm | ||
|
|
||
| def start_vm(vm_id: str): | ||
| vm = Vm.get(vm_id) | ||
| return vm.start() | ||
|
|
||
| def stop_vm(vm_id: str): | ||
| vm = Vm.get(vm_id) | ||
| return vm.stop() | ||
|
|
||
| def set_vm_password(vm_id: str): | ||
| vm = Vm.get(vm_id) | ||
| return vm.generate_password() | ||
|
|
||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| from time import sleep | ||
| def wait_for_state(vm, state_code: int, timeout: float, retries: int,): | ||
| for _ in range(0, retries): | ||
| code, _ = vm.state() | ||
| if state_code == code: | ||
| return code | ||
| sleep(timeout) | ||
| return code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,3 @@ | ||
| fastapi[standard] | ||
| libvirt-python | ||
| lxml |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.