Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DigitalOcean] droplet integration #3832

Merged
merged 85 commits into from
Jan 2, 2025
Merged
Changes from 1 commit
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
96c5e81
init digital ocean droplet integration
asaiacai Aug 14, 2024
fa8a6bb
abbreviate cloud name
asaiacai Aug 20, 2024
cc8384e
switch to pydo
asaiacai Aug 20, 2024
80b5941
adjust polling logic and mount block storage to instance
asaiacai Aug 31, 2024
49c411b
merge
asaiacai Aug 31, 2024
a741f11
filter by paginated
asaiacai Aug 31, 2024
8702acd
lint
asaiacai Sep 2, 2024
1de819c
sky launch, start, stop functional
asaiacai Sep 2, 2024
fafc71d
fix credential file mounts, autodown works now
asaiacai Sep 2, 2024
e50126c
set gpu droplet image
asaiacai Sep 3, 2024
8532fcf
cleanup
asaiacai Sep 3, 2024
13628ad
remove more tests
asaiacai Sep 3, 2024
34d1916
atomically destroy instance and block storage simulatenously
asaiacai Sep 3, 2024
5eab8f9
install docker
asaiacai Sep 3, 2024
c992161
disable spot test
asaiacai Sep 3, 2024
a868b1a
fix ip address bug for multinode
asaiacai Sep 6, 2024
d4f7794
lint
asaiacai Sep 6, 2024
30ead7b
patch ssh from job/serve controller
asaiacai Sep 6, 2024
6791a7d
switch to EA slugs
asaiacai Sep 6, 2024
af8e5e9
do adaptor
asaiacai Sep 10, 2024
3a31a0a
lint
asaiacai Sep 11, 2024
ce900ed
Update sky/clouds/do.py
asaiacai Sep 17, 2024
391fea1
Update sky/clouds/do.py
asaiacai Sep 17, 2024
1703b40
comment template
asaiacai Sep 17, 2024
66f0314
comment patch
asaiacai Sep 17, 2024
817f3b3
add h100 test case
asaiacai Sep 17, 2024
5d8368c
comment on instance name length
asaiacai Sep 17, 2024
74856df
Update sky/clouds/do.py
asaiacai Sep 18, 2024
cbbb36b
Update sky/clouds/service_catalog/do_catalog.py
asaiacai Sep 18, 2024
ee98000
comment on max node char len
asaiacai Sep 23, 2024
d6da5e8
comment on weird azure import
asaiacai Sep 23, 2024
79aac0a
comment acc price is included in instance price
asaiacai Sep 23, 2024
71c9f9a
fix return type
asaiacai Sep 23, 2024
4fc8fe8
switch with do_utils
asaiacai Sep 23, 2024
113d24d
remove broad except
asaiacai Sep 23, 2024
1e0f9ec
Update sky/provision/do/instance.py
asaiacai Sep 23, 2024
4ab385b
Update sky/provision/do/instance.py
asaiacai Sep 23, 2024
daa7446
remove azure
asaiacai Sep 23, 2024
0d71031
comment on non_terminated_only
asaiacai Sep 23, 2024
dd8c238
add open port debug message
asaiacai Sep 29, 2024
cf7947b
wrap start instance api
asaiacai Sep 29, 2024
56163c0
use f-string
asaiacai Sep 29, 2024
0d39425
wrap stop
asaiacai Sep 29, 2024
0f8a53b
wrap instance down
asaiacai Sep 29, 2024
2881508
assert credentials and check against all contexts
asaiacai Sep 29, 2024
ae76a80
assert client is None
asaiacai Sep 29, 2024
8056bc8
remove pending instances during instance restart
asaiacai Sep 29, 2024
9bdf9df
wrap rename
asaiacai Sep 29, 2024
6cccf6a
rename ssh key var
asaiacai Oct 4, 2024
901ed4e
fix tags
asaiacai Oct 4, 2024
7d57980
add tags for block device
asaiacai Oct 4, 2024
e8d1782
f strings for errors
asaiacai Oct 4, 2024
2e51c59
support image ids
asaiacai Oct 21, 2024
b5fe945
update do tests
Oct 24, 2024
6565fff
only store head instance id
Oct 24, 2024
c6a4583
Merge branch 'skypilot-org:master' into droplet
asaiacai Oct 25, 2024
fde2bc2
rename image slugs
Oct 25, 2024
baf5b48
Merge branch 'droplet' of https://github.com/asaiacai/skypilot into d…
Oct 25, 2024
ff87fe7
add digital ocean alias
Oct 25, 2024
c49c330
wait for docker to be available
Oct 25, 2024
c857fe9
Merge branch 'skypilot-org:master' into droplet
asaiacai Oct 25, 2024
40b2134
update requirements and tests
Oct 25, 2024
65bfc03
increase docker timeout
Oct 25, 2024
812f747
lint
Oct 26, 2024
031777a
Merge branch 'skypilot-org:master' into droplet
asaiacai Nov 4, 2024
3132acb
merge
asaiacai Dec 19, 2024
279dc93
move tests
asaiacai Dec 19, 2024
3b093b0
lint
asaiacai Dec 19, 2024
3023938
patch test
asaiacai Dec 19, 2024
b98b189
lint
asaiacai Dec 19, 2024
335b646
typo fix
asaiacai Dec 19, 2024
61e0ae9
fix typo
asaiacai Dec 19, 2024
b33edad
patch tests
asaiacai Dec 19, 2024
e0e4701
fix tests
asaiacai Dec 19, 2024
72cd751
no_mark spot test
asaiacai Dec 19, 2024
4bb73ce
handle 2cpu serve tests
asaiacai Dec 19, 2024
18e4e8c
lint
asaiacai Dec 19, 2024
97f8c4e
lint
asaiacai Dec 19, 2024
cd31370
use logger.debug
asaiacai Dec 19, 2024
717c75e
fix none cred path
asaiacai Dec 19, 2024
3d47236
lint
asaiacai Dec 19, 2024
3d5f073
handle get_cred path
asaiacai Dec 19, 2024
396d782
pylint
asaiacai Dec 19, 2024
24d3c4d
patch for DO test_optimizer_dryruns.py
asaiacai Dec 31, 2024
3bf98c8
revert optimizer dryrun
asaiacai Dec 31, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 24 additions & 6 deletions tests/test_optimizer_dryruns.py
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,15 @@ def test_partial_cpus(monkeypatch):


def test_partial_memory(monkeypatch):
_test_resources_launch(monkeypatch, memory=32)
_test_resources_launch(monkeypatch, memory='32')
enabled_clouds = list(clouds.CLOUD_REGISTRY.values())
# DO has no 32 mem instances
enabled_clouds.remove(clouds.CLOUD_REGISTRY['do'])
_test_resources_launch(monkeypatch,
memory=32,
enabled_clouds=enabled_clouds)
_test_resources_launch(monkeypatch,
memory='32',
enabled_clouds=enabled_clouds)
_test_resources_launch(monkeypatch, memory='32+')


Expand Down Expand Up @@ -250,7 +257,12 @@ def test_instance_type_from_cpu_memory(monkeypatch, capfd):
assert 'Standard_D8s_v5' in stdout # Azure, 8 vCPUs, 32 GB memory
assert 'n2-standard-8' in stdout # GCP, 8 vCPUs, 32 GB memory

_test_resources_launch(monkeypatch, memory=32)
enabled_clouds = list(clouds.CLOUD_REGISTRY.values())
# DO has no 32 mem instances
enabled_clouds.remove(clouds.CLOUD_REGISTRY['do'])
asaiacai marked this conversation as resolved.
Show resolved Hide resolved
_test_resources_launch(monkeypatch,
memory=32,
enabled_clouds=enabled_clouds)
stdout, _ = capfd.readouterr()
# Choose memory-optimized instance types, when the memory
# is specified
Expand All @@ -276,7 +288,9 @@ def test_instance_type_from_cpu_memory(monkeypatch, capfd):
assert 'Standard_F4s_v2' in stdout # Azure, 4 vCPUs, 8 GB memory
assert 'cpu_4x_general' in stdout # Lambda, 4 vCPUs, 16 GB memory

_test_resources_launch(monkeypatch, accelerators='T4')
_test_resources_launch(monkeypatch,
accelerators='T4',
enabled_clouds=enabled_clouds)
stdout, _ = capfd.readouterr()
# Choose cheapest T4 instance type
assert 'g4dn.xlarge' in stdout # AWS, 4 vCPUs, 16 GB memory, 1 T4 GPU
Expand All @@ -286,14 +300,18 @@ def test_instance_type_from_cpu_memory(monkeypatch, capfd):
_test_resources_launch(monkeypatch,
cpus='16+',
memory='32+',
accelerators='T4')
accelerators='T4',
enabled_clouds=enabled_clouds)
stdout, _ = capfd.readouterr()
# Choose cheapest T4 instance type that satisfies the requirement
assert 'n1-standard-16' in stdout # GCP, 16 vCPUs, 60 GB memory, 1 T4 GPU
assert 'g4dn.4xlarge' in stdout # AWS, 16 vCPUs, 64 GB memory, 1 T4 GPU
assert 'Standard_NC16as_T4_v3' in stdout # Azure, 16 vCPUs, 110 GB memory, 1 T4 GPU

_test_resources_launch(monkeypatch, memory='200+', accelerators='T4')
_test_resources_launch(monkeypatch,
memory='200+',
accelerators='T4',
enabled_clouds=enabled_clouds)
stdout, _ = capfd.readouterr()
# Choose cheapest T4 instance type that satisfies the requirement
assert 'n1-highmem-32' in stdout # GCP, 32 vCPUs, 208 GB memory, 1 T4 GPU
Expand Down
Loading