Skip to content

Commit d948997

Browse files
Jinyu-WJinyu Wang
and
Jinyu Wang
authored
update docker hub init (#367)
* update docker hub init * replace personal account with maro-team * update hello files for CIM * update docker repository name * update docker file name * fix bugs in notebook, rectify docs * fix doc build issue * remove docs from playground; fix citibike lp example Event issue * update the exampel for vector env * update vector env example * update README due to PR comments * add link to playground above MARO installation in README * fix some typos Co-authored-by: Jinyu Wang <[email protected]>
1 parent dca9280 commit d948997

28 files changed

+369
-385
lines changed

.github/ISSUE_TEMPLATE/---bug-report.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ Steps to reproduce the behavior:
3838
- How you installed MARO (`pip`, `source`):
3939
- OS (`Linux`, `Windows`, `macOS`):
4040
- Python version (`3.6`, `3.7`):
41-
- Docker image (e.g., arthursjiang/maro:cpu[5f36ed]):
41+
- Docker image (e.g., maro2020/maro:latest):
4242
- CPU/GPU:
4343
- Any other relevant information:
4444

.github/workflows/deploy_docker_image.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ jobs:
3030
run: |
3131
pip install -r ./maro/requirements.build.txt
3232
cython ./maro/backends/backend.pyx ./maro/backends/np_backend.pyx ./maro/backends/raw_backend.pyx ./maro/backends/frame.pyx --cplus -3 -E NODES_MEMORY_LAYOUT=ONE_BLOCK -X embedsignature=True
33-
cat ./maro/__misc__.py | grep __version__ | egrep -o [0-9].[0-9].[0-9,a-z]+ | { read version; docker build -f ./docker_files/cpu.play.df . -t ${{ secrets.DOCKER_HUB_USERNAME }}/maro:cpu -t ${{ secrets.DOCKER_HUB_USERNAME }}/maro:latest -t ${{ secrets.DOCKER_HUB_USERNAME }}/maro:cpu-$version; }
33+
cat ./maro/__misc__.py | grep __version__ | egrep -o [0-9].[0-9].[0-9,a-z]+ | { read version; docker build -f ./docker_files/cpu.playground.df . -t ${{ secrets.DOCKER_HUB_USERNAME }}/maro:cpu -t ${{ secrets.DOCKER_HUB_USERNAME }}/maro:latest -t ${{ secrets.DOCKER_HUB_USERNAME }}/maro:cpu-$version; }
3434
3535
- name: Login docker hub
3636
run: |

README.md

+13-14
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,13 @@
22
[![Platform](https://raw.githubusercontent.com/microsoft/maro/master/docs/source/images/badges/platform.svg)](https://pypi.org/project/pymaro/)
33
[![Python Versions](https://img.shields.io/pypi/pyversions/pymaro.svg?logo=python&logoColor=white)](https://pypi.org/project/pymaro/#files)
44
[![Code Size](https://img.shields.io/github/languages/code-size/microsoft/maro)](https://github.com/microsoft/maro)
5-
[![Docker Size](https://img.shields.io/docker/image-size/arthursjiang/maro)](https://hub.docker.com/repository/docker/arthursjiang/maro/tags?page=1)
5+
[![Docker Size](https://img.shields.io/docker/image-size/maro2020/maro)](https://hub.docker.com/repository/docker/maro2020/maro/tags?page=1)
66
[![Issues](https://img.shields.io/github/issues/microsoft/maro)](https://github.com/microsoft/maro/issues)
77
[![Pull Requests](https://img.shields.io/github/issues-pr/microsoft/maro)](https://github.com/microsoft/maro/pulls)
88
[![Dependencies](https://img.shields.io/librariesio/github/microsoft/maro)](https://libraries.io/pypi/pymaro)
99
[![test](https://github.com/microsoft/maro/workflows/test/badge.svg)](https://github.com/microsoft/maro/actions?query=workflow%3Atest)
1010
[![build](https://github.com/microsoft/maro/workflows/build/badge.svg)](https://github.com/microsoft/maro/actions?query=workflow%3Abuild)
11-
[![docker](https://github.com/microsoft/maro/workflows/docker/badge.svg)](https://hub.docker.com/repository/docker/arthursjiang/maro)
11+
[![docker](https://github.com/microsoft/maro/workflows/docker/badge.svg)](https://hub.docker.com/repository/docker/maro2020/maro)
1212
[![docs](https://readthedocs.org/projects/maro/badge/?version=latest)](https://maro.readthedocs.io/)
1313
[![PypI Versions](https://img.shields.io/pypi/v/pymaro)](https://pypi.org/project/pymaro/#files)
1414
[![Wheel](https://img.shields.io/pypi/wheel/pymaro)](https://pypi.org/project/pymaro/#files)
@@ -23,8 +23,8 @@
2323
[![Lint](https://github.com/microsoft/maro/workflows/lint/badge.svg)](https://github.com/microsoft/maro/actions?query=workflow%3Alint)
2424
[![Coverage](https://img.shields.io/codecov/c/github/microsoft/maro)](https://codecov.io/gh/microsoft/maro)
2525
[![Downloads](https://img.shields.io/pypi/dm/pymaro)](https://pypi.org/project/pymaro/#files)
26-
[![Docker Pulls](https://img.shields.io/docker/pulls/arthursjiang/maro)](https://hub.docker.com/repository/docker/arthursjiang/maro)
27-
[![Play with MARO](https://raw.githubusercontent.com/microsoft/maro/master/docs/source/images/badges/play_with_maro.svg)](https://hub.docker.com/r/arthursjiang/maro)
26+
[![Docker Pulls](https://img.shields.io/docker/pulls/maro2020/maro)](https://hub.docker.com/repository/docker/maro2020/maro)
27+
[![Play with MARO](https://raw.githubusercontent.com/microsoft/maro/master/docs/source/images/badges/play_with_maro.svg)](https://hub.docker.com/r/maro2020/maro)
2828

2929
# [![MARO LOGO](./docs/source/images/logo.svg)](https://maro.readthedocs.io/en/latest/)
3030

@@ -58,6 +58,8 @@ of user-defined functions for message auto-handling, cluster provision, and job
5858
| `examples` | Showcase of MARO. |
5959
| `notebooks` | MARO quick-start notebooks. |
6060

61+
*Try [MARO playground](#run-playground) to have a quick experience.*
62+
6163
## Install MARO from [PyPI](https://pypi.org/project/pymaro/#files)
6264

6365
*Notes: The CLI commands (including the visualization tool) are not included in pymaro package. To enable these support, you need to install from source.*
@@ -185,14 +187,13 @@ maro inspector dashboard --source_path ./dump_data/YOUR_SNAPSHOT_DUMP_FOLDER
185187
186188
## Run Playground
187189
188-
- Pull from [Docker Hub](https://hub.docker.com/repository/registry-1.docker.io/arthursjiang/maro/tags?page=1)
190+
- Pull from [Docker Hub](https://hub.docker.com/r/maro2020/playground)
189191
190192
```sh
191193
# Run playground container.
192194
# Redis commander (GUI for redis) -> http://127.0.0.1:40009
193-
# Local host docs -> http://127.0.0.1:40010
194-
# Jupyter lab with maro -> http://127.0.0.1:40011
195-
docker run -p 40009:40009 -p 40010:40010 -p 40011:40011 arthursjiang/maro:cpu
195+
# Jupyter lab with maro -> http://127.0.0.1:40010
196+
docker run -p 40009:40009 -p 40010:40010 maro2020/playground
196197
```
197198
198199
- Build from source
@@ -204,9 +205,8 @@ maro inspector dashboard --source_path ./dump_data/YOUR_SNAPSHOT_DUMP_FOLDER
204205
205206
# Run playground container.
206207
# Redis commander (GUI for redis) -> http://127.0.0.1:40009
207-
# Local host docs -> http://127.0.0.1:40010
208-
# Jupyter lab with maro -> http://127.0.0.1:40011
209-
docker run -p 40009:40009 -p 40010:40010 -p 40011:40011 maro/playground:cpu
208+
# Jupyter lab with maro -> http://127.0.0.1:40010
209+
docker run -p 40009:40009 -p 40010:40010 maro2020/playground
210210
```
211211
212212
- Windows
@@ -217,9 +217,8 @@ maro inspector dashboard --source_path ./dump_data/YOUR_SNAPSHOT_DUMP_FOLDER
217217
218218
# Run playground container.
219219
# Redis commander (GUI for redis) -> http://127.0.0.1:40009
220-
# Local host docs -> http://127.0.0.1:40010
221-
# Jupyter lab with maro -> http://127.0.0.1:40011
222-
docker run -p 40009:40009 -p 40010:40010 -p 40011:40011 maro/playground:cpu
220+
# Jupyter lab with maro -> http://127.0.0.1:40010
221+
docker run -p 40009:40009 -p 40010:40010 maro2020/playground
223222
```
224223
225224
## Contributing
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,8 @@
1-
FROM python:3.6
1+
FROM python:3.7
22

33
WORKDIR /maro_playground
44

5-
# Setup notebook
6-
ADD ./notebooks ./notebooks
7-
RUN /usr/local/bin/python -m pip install --upgrade pip
8-
RUN pip install -r ./notebooks/requirements.nb.txt
9-
RUN jupyter contrib nbextension install --system
10-
RUN jt -t onedork -fs 95 -altp -tfs 11 -nfs 115 -cellw 88% -T
11-
12-
13-
# Install redis
14-
RUN wget http://download.redis.io/releases/redis-6.0.6.tar.gz; tar xzf redis-6.0.6.tar.gz; cd redis-6.0.6; make
15-
RUN rm redis-6.0.6.tar.gz
16-
17-
# Install others
5+
# Install zsh and other packages for the terminal usage
186
RUN apt-get -o Acquire::Check-Valid-Until=false -o Acquire::Check-Date=false update
197
RUN apt-get install -y zsh
208
RUN apt-get install -y htop
@@ -23,42 +11,36 @@ RUN wget https://github.com/robbyrussell/oh-my-zsh/raw/master/tools/install.sh -
2311
RUN chsh -s `which zsh` && wget https://raw.githubusercontent.com/ArthurJiang/config/master/.zshrc -O ~/.zshrc
2412
RUN apt-get install -y npm
2513
RUN rm -rf /var/lib/apt/lists/*
14+
15+
# Install redis
16+
RUN wget http://download.redis.io/releases/redis-6.0.6.tar.gz; tar xzf redis-6.0.6.tar.gz; cd redis-6.0.6; make
17+
RUN rm redis-6.0.6.tar.gz
2618
RUN npm install -g redis-commander
2719

20+
# Setup notebook
21+
ADD ./notebooks ./notebooks
22+
RUN /usr/local/bin/python -m pip install --upgrade pip
23+
RUN pip install -r ./notebooks/requirements.nb.txt
24+
RUN jupyter contrib nbextension install --system
25+
RUN jt -t onedork -fs 95 -altp -tfs 11 -nfs 115 -cellw 88% -T
26+
RUN rm ./notebooks/*.txt
27+
RUN rm ./notebooks/*.sh
28+
2829
# Add examples
2930
ADD ./examples ./examples
31+
ADD ./examples/requirements.ex.txt ./examples/requirements.ex.txt
32+
RUN pip install -r ./examples/requirements.ex.txt
33+
RUN rm ./examples/requirements.ex.txt
3034

31-
# Add local docs
32-
ADD ./docs ./docs
33-
ADD ./maro ./maro
34-
ADD setup.py setup.py
35-
ADD ./scripts ./scripts
36-
RUN bash scripts/install_maro.sh
37-
RUN pip install -U -r ./docs/requirements.docs.txt
38-
RUN cd docs; make html
39-
RUN rm -rf ./maro
40-
RUN rm setup.py
41-
RUN rm -rf ./scripts
35+
# Install MARO
36+
RUN pip install pymaro
37+
ENV PYTHONPATH ./
4238

4339
# Add run cmd
4440
ADD ./scripts/run_playground.sh ./run.sh
4541

46-
# Add readme
42+
# Add README
4743
ADD ./playground.md ./README.md
4844

49-
# Clean
50-
RUN rm ./notebooks/*.txt
51-
RUN rm ./notebooks/*.sh
52-
RUN rm -r ./docs/source
53-
RUN rm ./docs/make.bat
54-
RUN rm ./docs/Makefile
55-
RUN rm ./docs/README.md
56-
RUN rm ./docs/requirements.docs.txt
57-
RUN rm -rf ./build
58-
RUN rm -rf ./pymaro.egg-info
59-
60-
# Install maro
61-
RUN pip install pymaro
62-
6345
# Start service
6446
CMD ["/bin/bash", "./run.sh"]

docs/source/examples/greedy_policy_citi_bike.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -76,4 +76,4 @@ This environment is driven by `real trip history data <https://s3.amazonaws.com/
7676
7777
.. note::
7878

79-
All related code snippets are supported in `maro playground <https://hub.docker.com/r/arthursjiang/maro>`_.
79+
All related code snippets are supported in `maro playground <https://hub.docker.com/r/maro2020/playground>`_.

docs/source/examples/multi_agent_dqn_cim.rst

+5-5
Original file line numberDiff line numberDiff line change
@@ -19,15 +19,15 @@ in the roll-out loop. In this example,
1919
to or unloaded from the vessel) to action objects that can be executed by the environment.
2020
* ``get_offline_reward`` computes the reward of a given action as a linear combination of fulfillment and
2121
shortage within a future time frame.
22-
* ``on_finish`` processes a complete trajectory into data that can be used directly by the learning agents.
22+
* ``on_finish`` processes a complete trajectory into data that can be used directly by the learning agents.
2323

2424

2525
.. code-block:: python
2626
class CIMTrajectoryForDQN(Trajectory):
2727
def __init__(
2828
self, env, *, port_attributes, vessel_attributes, action_space, look_back, max_ports_downstream,
2929
reward_time_window, fulfillment_factor, shortage_factor, time_decay,
30-
finite_vessel_space=True, has_early_discharge=True
30+
finite_vessel_space=True, has_early_discharge=True
3131
):
3232
super().__init__(env)
3333
self.port_attributes = port_attributes
@@ -113,7 +113,7 @@ Agent
113113

114114
The out-of-the-box DQN is used as our agent.
115115

116-
.. code-block:: python
116+
.. code-block:: python
117117
agent_config = {
118118
"model": ...,
119119
"optimization": ...,
@@ -149,7 +149,7 @@ The learner's side requires a concrete learner class that inherits from ``AbsLea
149149
method which contains the main training loop. Here the implementation is similar to the single-threaded version
150150
except that the ``collect`` method is used to obtain roll-out data from the actors (since the roll-out executors
151151
are located on the actors' side). The agents created here are where training occurs and hence always contains the
152-
latest policies.
152+
latest policies.
153153

154154
.. code-block:: python
155155
def cim_dqn_learner():
@@ -165,4 +165,4 @@ latest policies.
165165
166166
.. note::
167167

168-
All related code snippets are supported in `maro playground <https://hub.docker.com/r/arthursjiang/maro>`_.
168+
All related code snippets are supported in `maro playground <https://hub.docker.com/r/maro2020/playground>`_.

docs/source/index.rst

+26-36
Original file line numberDiff line numberDiff line change
@@ -32,45 +32,35 @@ Quick Start
3232
.. code-block:: python
3333
3434
from maro.simulator import Env
35-
from maro.simulator.scenarios.cim.common import Action
35+
from maro.simulator.scenarios.cim.common import Action, ActionType, DecisionEvent
3636
37-
# Initialize an environment with a specific scenario, related topology.
38-
# In Container Inventory Management, 1 tick means 1 day, here durations=100 means a length of 100 days
37+
from random import randint
38+
39+
# Initialize an Env for cim scenario
3940
env = Env(scenario="cim", topology="toy.5p_ssddd_l0.0", start_tick=0, durations=100)
4041
41-
# Query environment summary, which includes business instances, intra-instance attributes, etc.
42-
print(env.summary)
43-
44-
for ep in range(2):
45-
# Gym-like step function.
46-
metrics, decision_event, is_done = env.step(None)
47-
48-
while not is_done:
49-
past_week_ticks = [
50-
x for x in range(decision_event.tick - 7, decision_event.tick)
51-
]
52-
decision_port_idx = decision_event.port_idx
53-
intr_port_infos = ["booking", "empty", "shortage"]
54-
55-
# Query the snapshot list of the environment to get the information of
56-
# the booking, empty container inventory, shortage of the decision port in the past week.
57-
past_week_info = env.snapshot_list["ports"][
58-
past_week_ticks : decision_port_idx : intr_port_infos
59-
]
60-
61-
dummy_action = Action(
62-
vessel_idx=decision_event.vessel_idx,
63-
port_idx=decision_event.port_idx,
64-
quantity=0
65-
)
66-
67-
# Drive environment with dummy action (no repositioning).
68-
metrics, decision_event, is_done = env.step(dummy_action)
69-
70-
# Query environment business metrics at the end of an episode,
71-
# it is your optimized object (usually includes multi-target).
72-
print(f"ep: {ep}, environment metrics: {env.metrics}")
73-
env.reset()
42+
metrics: object = None
43+
decision_event: DecisionEvent = None
44+
is_done: bool = False
45+
action: Action = None
46+
47+
# Start the env with a None Action
48+
metrics, decision_event, is_done = env.step(None)
49+
50+
while not is_done:
51+
# Generate a random Action according to the action_scope in DecisionEvent
52+
action_scope = decision_event.action_scope
53+
to_discharge = action_scope.discharge > 0 and randint(0, 1) > 0
54+
55+
action = Action(
56+
decision_event.vessel_idx,
57+
decision_event.port_idx,
58+
randint(0, action_scope.discharge if to_discharge else action_scope.load),
59+
ActionType.DISCHARGE if to_discharge else ActionType.LOAD
60+
)
61+
62+
# Respond the environment with the generated Action
63+
metrics, decision_event, is_done = env.step(action)
7464
7565
Contents
7666
----------

docs/source/installation/playground.rst

+10-16
Original file line numberDiff line numberDiff line change
@@ -2,16 +2,15 @@
22
Playground Docker Image
33
=======================
44

5-
Pull from `Docker Hub <https://hub.docker.com/repository/registry-1.docker.io/arthursjiang/maro/tags?page=1>`_
5+
Pull from `Docker Hub <https://hub.docker.com/repository/registry-1.docker.io/maro2020/playground/tags?page=1>`_
66
------------------------------------------------------------------------------------------------------------------
77

88
.. code-block:: sh
99
1010
# Run playground container.
1111
# Redis commander (GUI for redis) -> http://127.0.0.1:40009
12-
# Local host docs -> http://127.0.0.1:40010
13-
# Jupyter lab with maro -> http://127.0.0.1:40011
14-
docker run -p 40009:40009 -p 40010:40010 -p 40011:40011 arthursjiang/maro:cpu
12+
# Jupyter lab with maro -> http://127.0.0.1:40010
13+
docker run -p 40009:40009 -p 40010:40010 maro2020/playground
1514
1615
Run from Source
1716
---------------
@@ -25,9 +24,8 @@ Run from Source
2524
2625
# Run playground container.
2726
# Redis commander (GUI for redis) -> http://127.0.0.1:40009
28-
# Local host docs -> http://127.0.0.1:40010
29-
# Jupyter lab with maro -> http://127.0.0.1:40011
30-
docker run -p 40009:40009 -p 40010:40010 -p 40011:40011 maro/playground:cpu
27+
# Jupyter lab with maro -> http://127.0.0.1:40010
28+
docker run -p 40009:40009 -p 40010:40010 maro2020/playground
3129
3230
* Windows
3331

@@ -38,9 +36,8 @@ Run from Source
3836
3937
# Run playground container.
4038
# Redis commander (GUI for redis) -> http://127.0.0.1:40009
41-
# Local host docs -> http://127.0.0.1:40010
42-
# Jupyter lab with maro -> http://127.0.0.1:40011
43-
docker run -p 40009:40009 -p 40010:40010 -p 40011:40011 maro/playground:cpu
39+
# Jupyter lab with maro -> http://127.0.0.1:40010
40+
docker run -p 40009:40009 -p 40010:40010 maro2020/playground
4441
4542
Major Services in Playground
4643
----------------------------
@@ -54,15 +51,12 @@ Major Services in Playground
5451
* - ``Redis Commander``
5552
- Redis web GUI.
5653
- http://127.0.0.1:40009
57-
* - ``Read the Docs``
58-
- Local host docs.
59-
- http://127.0.0.1:40010
6054
* - ``Jupyter Lab``
6155
- Jupyter lab with MARO environment, examples, notebooks.
62-
- http://127.0.0.1:40011
56+
- http://127.0.0.1:40010
6357

6458

65-
*(If you use other port mapping, remember to change the port number.)*
59+
*(Remember to change ports if you use different ports mapping.)*
6660

6761
Major Materials in Root Folder
6862
------------------------------
@@ -78,4 +72,4 @@ Major Materials in Root Folder
7872
- Quick-start tutorial.
7973

8074

81-
*(Those not mentioned in the table can be ignored.)*
75+
*(The ones not mentioned in this table can be ignored.)*

0 commit comments

Comments
 (0)