Skip to content

Commit

Permalink
updated readme
Browse files Browse the repository at this point in the history
  • Loading branch information
gnzsnz committed Aug 26, 2023
1 parent 08f87d8 commit 759c17a
Show file tree
Hide file tree
Showing 2 changed files with 42 additions and 21 deletions.
61 changes: 41 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,22 +13,14 @@ A dockerized Jupyter quant research enviroment.
- The usual suspects are included, numpy, pandas, sci-py, scikit-learn
- Includes tools for quant analysis, statsmodels, pymc, arch, py_vollib, zipline-reloaded, PyPortfolioOpt, etc.
- ib_insync for Interactive Broker connectivity. Works well with [IB Gateway](https://github.com/gnzsnz/ib-gateway-docker)
- sudo, so you can install new packages if needed
- bash and stow, so you can BYODF (bring your own dot files)
- sudo, so you can install new packages if needed.
- bash and stow, so you can BYODF (bring your own dot files). Plus common command line utilities like git, less, nano (tiny), jq, ssh, curl, bash completion and others.
- Support for [apt cache](https://github.com/gnzsnz/apt-cacher-ng). If you have other linux boxes using you can leverage your cache. apt cache support major linux distributions not only debian/ubuntu.
- It does not include a build environment. If you need to install a package that does not provide wheels you can build your wheels, as explained in [common tasks](#common-tasks)

## Volumes

The image is designed to work with 3 volumes:

1. `quant_data` - volume for ~/.local folder. It contains caches and all python packages. This enables to add additional packages through pip.
1. `quant_conf` - volume for ~/.config, all config goes here. This includes jupyter, ipython, matplotlib, etc
1. Bind mount (but you could use a named volume) - volume for all notebooks, under `~/Notebooks`.
## Quick Start

This allows to have ephemeral containers and to keep your notebooks (3), your config (2) and your additional packages (1). Eventually you would need to update the image, in this case your notebooks (3) can move without issues, your config (2) should still work but no warranty, and your packages could still be used. Eventually you would need to refresh (1) and less frecuently (2)

## Sample docker compose
Create a `docker-compose.yml` file with this content

```yml
version: "3.6"
Expand All @@ -51,6 +43,25 @@ volumes:
quant_data:
```
You can use `.env-dist` as your starting point.

```bash
cp .env-dist .env
# verify everything looks good
docker compose config
docker compose up
```

## Volumes

The image is designed to work with 3 volumes:

1. `quant_data` - volume for ~/.local folder. It contains caches and all python packages. This enables to add additional packages through pip.
1. `quant_conf` - volume for ~/.config, all config goes here. This includes jupyter, ipython, matplotlib, etc
1. Bind mount (but you could use a named volume) - volume for all notebooks, under `~/Notebooks`.

This allows to have ephemeral containers and to keep your notebooks (3), your config (2) and your additional packages (1). Eventually you would need to update the image, in this case your notebooks (3) can move without issues, your config (2) should still work but no warranty, and your packages could still be used. Eventually you would need to refresh (1) and less frecuently (2)

## Common tasks

- get running server URL
Expand All @@ -61,33 +72,39 @@ Currently running servers:
http://40798f7a604a:8888/?token=ebf9e870d2aa0ed877590eb83b4d3bbbdfbd55467422a167 :: /home/gordon/Notebooks
```

you will need to change hostname, 40798f7a604a in this case by your docker host ip.
or

```bash
docker logs -t jupyter-quant 2>&1 | grep '127.0.0.1:8888/lab?token='
```

You will need to change hostname (40798f7a604a in this case) or 127.0.0.1 by your docker host ip.

- show jupyter config

```bash
docker exec -it jupyter-quat jupyter-server --show-config
docker exec -it jupyter-quant jupyter-server --show-config
```

- set password

```bash
docker exec -it jupyter-quat jupyter-server password
docker exec -it jupyter-quant jupyter-server password
```

- get help

```bash
docker exec -it jupyter-quat jupyter-server --help
docker exec -it jupyter-quat jupyter-lab --help
docker exec -it jupyter-quant jupyter-server --help
docker exec -it jupyter-quant jupyter-lab --help
```

- get installed packeges
- get installed packages

```bash
docker exec -it jupyter-quat pip list
docker exec -it jupyter-quant pip list
# outdated packages
docker exec -it jupyter-quat pip list -o
docker exec -it jupyter-quant pip list -o
```

- the image's entrypoint supports jupyter-lab parameters, for example
Expand All @@ -112,3 +129,7 @@ pip wheel --no-cache-dir --wheel-dir /wheels numpy
```

This will build wheels for numpy (ot any other package that you need) and save the file in $PWD/wheels. Then you can copy the wheels in your notebooks mount (3 above) and install it within the container. You can even drag and drop into jupyter.

- Install your dotfiles.

`git clone` your dotfiles to `Notebook/etc/dotfiles`, set enviroment variable `BYODF=/home/gordon/Notebook/etc/dotfiles` in your docker compose. When the container starts up stow will create links like `/home/gordon/.bashrc`
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,4 @@ scipy
seaborn
statsmodels
yfinance
zipline-reloaded
zipline-reloaded

0 comments on commit 759c17a

Please sign in to comment.