Merge branch 'neptune-ai-master'

.gitlab-ci.yml

@@ -158,7 +158,6 @@ stages:
   echo ""
   cat cicd-release/values.yaml
   echo ""
-  FILE_NAME=$(ls cicd-release/ | grep neptune-docs)
 
   helm upgrade -i \
     neptune-docs \

docs/index.rst

@@ -123,6 +123,7 @@ Documentation contents
    PyTorchLightning <integrations/pytorch_lightning.rst>
    Catalyst <integrations/catalyst.rst>
    PyTorch Ignite <integrations/pytorch_ignite.rst>
+   Skorch <integrations/skorch.rst>
    Neptune Contrib <integrations/neptune-contrib.rst>
 
 .. External links

docs/integrations/skorch.rst

@@ -0,0 +1,220 @@
+Log Skorch metrics to neptune
+=============================
+.. image:: ../_static/images/others/skorch_neptuneai.png
+   :target: ../_static/images/others/skorch_neptuneai.png
+   :alt: Skorch neptune.ai integration
+
+Prerequisites
+-------------
+Integration with |Skorch| framework is introduced as a part of logging module so just need to have |neptune-client| installed.
+
+.. code-block:: bash
+
+    pip install neptune-client
+
+
+Create an **Experiment**
+------------------------
+
+.. note:: I am using a shared, anonymous token but you can change to your user api token.
+
+.. code-block:: python3
+
+    import neptune
+
+    neptune.init(
+        api_token='eyJhcGlfYWRkcmVzcyI6Imh0dHBzOi8vdWkubmVwdHVuZS5haSIsImFwaV9rZXkiOiJiNzA2YmM4Zi03NmY5LTRjMmUtOTM5ZC00YmEwMzZmOTMyZTQifQ==',
+        project_qualified_name='shared/skorch-integration')
+
+    experiment = neptune.create_experiment(name='skorch-basic-example',
+                                           params={'max_epochs': 20,
+                                                   'lr': 0.1},
+                                           upload_source_files=['skorch_example.py'])
+
+Create **NeptuneLogger** callback
+--------------------------------
+Pass the experiment object as first argument.
+
+.. note:: To be able to log information after the .fit() method finishes remember to pass ``close_after_train=False``
+
+.. code-block:: python3
+
+    from skorch.callbacks.logging import NeptuneLogger
+    neptune_logger = NeptuneLogger(experiment, close_after_train=False)
+
+Pass **neptune_logger** to **NeuralNetClassifier**
+-------------------------------------------------
+.. code-block:: python3
+
+    net = NeuralNetClassifier(
+        ClassifierModule,
+        max_epochs=20,
+        lr=0.1,
+        callbacks=[neptune_logger])
+
+    net.fit(X, y)
+
+Log additional information
+--------------------------
+
+**Log test metrics after training**
+
+.. code-block:: python3
+
+    from sklearn.metrics import roc_auc_score
+
+    y_pred = net.predict_proba(X)
+    auc = roc_auc_score(y, y_pred[:, 1])
+
+    neptune_logger.experiment.log_metric('roc_auc_score', auc)
+
+**Log performance charts**
+
+.. code-block:: python3
+
+    from scikitplot.metrics import plot_roc
+    import matplotlib.pyplot as plt
+
+    fig, ax = plt.subplots(figsize=(16, 12))
+    plot_roc(y, y_pred, ax=ax)
+
+    neptune_logger.experiment.log_image('roc_curve', fig)
+
+**Log trained model**
+
+.. code-block:: python3
+
+    net.save_params(f_params='basic_model.pkl')
+    neptune_logger.experiment.log_artifact('basic_model.pkl')
+
+Monitor your Skorch training in Neptune
+---------------------------------------
+Now you can watch your Skorch model training in neptune!
+
+Check out this |example experiment|.
+
+.. image:: ../_static/images/skorch/skorch_monitoring.gif
+   :target: ../_static/images/skorch/skorch_monitoring.gif
+   :alt: Skorch monitoring in neptune
+
+Close experiment
+----------------
+If you passed ``close_after_train=False`` to ``NeptuneLogger`` you may want to close your experiment when you are done logging.
+
+.. code-block:: python3
+
+    neptune_logger.experiment.stop()
+
+Full Skorch monitoring script
+-----------------------------
+Simply copy and paste it to ``skorch_example.py`` and run.
+Remember to change your credentials in **neptune.init()**:
+
+.. code-block:: python3
+
+    neptune.init(api_token=os.getenv('NEPTUNE_API_TOKEN'),
+                 project_qualified_name='USER_NAME/PROJECT_NAME')
+
+.. code-block:: python3
+
+    import torch
+    from torch import nn
+    import torch.nn.functional as F
+
+    torch.manual_seed(0)
+
+    # create data
+    import numpy as np
+    from sklearn.datasets import make_classification
+
+    X, y = make_classification(1000, 20, n_informative=10, random_state=0)
+    X = X.astype(np.float32)
+
+
+    # create pytorch module
+    class ClassifierModule(nn.Module):
+        def __init__(
+                self,
+                num_units=10,
+                nonlin=F.relu,
+                dropout=0.5,
+        ):
+            super(ClassifierModule, self).__init__()
+            self.num_units = num_units
+            self.nonlin = nonlin
+            self.dropout = dropout
+
+            self.dense0 = nn.Linear(20, num_units)
+            self.nonlin = nonlin
+            self.dropout = nn.Dropout(dropout)
+            self.dense1 = nn.Linear(num_units, 10)
+            self.output = nn.Linear(10, 2)
+
+        def forward(self, X, **kwargs):
+            X = self.nonlin(self.dense0(X))
+            X = self.dropout(X)
+            X = F.relu(self.dense1(X))
+            X = F.softmax(self.output(X), dim=-1)
+            return X
+
+
+    # create neptune logger and pass it to NeuralNetClassifier
+    from skorch import NeuralNetClassifier
+    import neptune
+    from skorch.callbacks.logging import NeptuneLogger
+
+    neptune.init('neptune-ai/skorch-integration')
+    experiment = neptune.create_experiment(
+        name='skorch-basic-example',
+        params={'max_epochs': 20,
+                'lr': 0.1},
+        upload_source_files=['skorch_example.py'])
+    neptune_logger = NeptuneLogger(experiment, close_after_train=False)
+
+    net = NeuralNetClassifier(
+        ClassifierModule,
+        max_epochs=20,
+        lr=0.1,
+        callbacks=[neptune_logger]
+    )
+
+    # run training
+    net.fit(X, y)
+
+    # log score after training
+    from sklearn.metrics import roc_auc_score
+
+    y_pred = net.predict_proba(X)
+    auc = roc_auc_score(y, y_pred[:, 1])
+
+    neptune_logger.experiment.log_metric('roc_auc_score', auc)
+
+    # log charts like ROC curve
+    from scikitplot.metrics import plot_roc
+    import matplotlib.pyplot as plt
+
+    fig, ax = plt.subplots(figsize=(16, 12))
+    plot_roc(y, y_pred, ax=ax)
+    neptune_logger.experiment.log_image('roc_curve', fig)
+
+    # log model after training
+    net.save_params(f_params='basic_model.pkl')
+    neptune_logger.experiment.log_artifact('basic_model.pkl')
+
+    # close experiment
+    neptune_logger.experiment.stop()
+
+
+.. External links
+
+.. |Skorch| raw:: html
+
+    <a href="https://github.com/skorch-dev/skorch" target="_blank">Skorch</a>
+
+.. |example experiment| raw:: html
+
+    <a href="https://ui.neptune.ai/o/neptune-ai/org/skorch-integration/e/SKOR-27/charts" target="_blank">example experiment</a>
+
+.. |neptune-client| raw:: html
+
+    <a href="https://github.com/neptune-ai/neptune-client" target="_blank">neptune-client</a>

docs/notebooks/configuration.rst

@@ -1,7 +1,7 @@
 Connecting the Jupyter Extension to Your Neptune Account
 ========================================================
 
-After you have successfully `installed the Jupyter extension for Neptune <installation.html>`_, 
+After you have successfully `installed the Jupyter extension for Neptune <installation.html>`_,
 you connect it to your Neptune account.
 
 **Procedure**
@@ -13,7 +13,7 @@ you connect it to your Neptune account.
    :alt: image
 
 
-The **Configure your connection to Neptune** dialog appears. 
+The **Configure your connection to Neptune** dialog appears.
 
 .. image:: ../_static/images/notebooks/configure_connect.png
    :target: ../_static/images/notebooks/configure_connect.png
@@ -25,7 +25,7 @@ The **Configure your connection to Neptune** dialog appears.
 3. In the Neptune UI, obtain your API Token and copy it to the clipboard.
 
    a. In the upper right corner, click the avatar, and then click **Get API Token**.
-   
+
     .. image:: ../_static/images/notebooks/get_api_token.png
         :target: ../_static/images/notebooks/get_api_token.png
         :alt: image
@@ -39,6 +39,6 @@ The **Configure your connection to Neptune** dialog appears.
 .. warning:: Your *API Token* is private and unique. Never share it. It's like sharing password.
 
 
-5. To conclude, to see experiments that you will run associated with this Notebook, click **Activate**. 
+5. To conclude, to see experiments that you will run associated with this Notebook, click **Activate**.
 In the dialog that appears, click **Activate**.
 

docs/notebooks/installation.rst

@@ -1,12 +1,15 @@
-Installing neptune-notebooks 
+
+Installing neptune-notebooks
 ============================
-This page provides instructions for installing neptune-notebooks -- the Neptune Jupyter extension. 
+
+This page provides instructions for installing neptune-notebooks -- the Neptune Jupyter extension.
 
 The extension enables the integration of Neptune and Jupyter or JupyterLab.
 
-When the extension is successfully installed, 
-you will be able to upload notebooks to Neptune, check out previously uploaded Notebooks, 
-and track experiments and metrics directly from the Jupyter UI. 
+When the extension is successfully installed,
+you will be able to upload notebooks to Neptune, check out previously uploaded Notebooks,
+and track experiments and metrics directly from the Jupyter UI.
+
 Neptune versions the Notebook automatically once an experiment has started.
 
 About neptune-notebooks
@@ -22,7 +25,7 @@ neptune-notebooks is a Python package hosted `here <https://pypi.org/project/nep
 Installation for Jupyter
 ------------------------
 
-.. tip:: When installing Python packages, it is best practice to work in a `virtual environment <https://virtualenv.pypa.io/en/latest/>`_. 
+.. tip:: When installing Python packages, it is best practice to work in a `virtual environment <https://virtualenv.pypa.io/en/latest/>`_.
 
 Procedure
 
@@ -48,7 +51,8 @@ Procedure
 Installation for JupyterLab
 ---------------------------
 
-JupyterLab is the next-generation web-based UI for Project Jupyter. 
+JupyterLab is the next-generation web-based UI for Project Jupyter.
+
 
 When you install JupyterLab using ``pip install jupyterlab``, the standard Jupyter is also installed, as a dependency.
 
@@ -67,11 +71,12 @@ Details for these methods follow below.
 
 Irrespective of which method you use, the following must be preinstalled on your system before you begin.
 
-- `Node.js <https://nodejs.org/en>`_ 
+
+- `Node.js <https://nodejs.org/en>`_
 - `npm <https://www.npmjs.com/get-npm>`_
 - `JupyterLab <https://jupyterlab.readthedocs.io/en/stable/getting_started/installation.html>`_
 
-.. tip:: When installing Python packages, it is best practice to work in a `virtual environment <https://virtualenv.pypa.io/en/latest/>`_. 
+.. tip:: When installing Python packages, it is best practice to work in a `virtual environment <https://virtualenv.pypa.io/en/latest/>`_.
 
 
 Method 1: Install the full neptune-notebooks package through the command line
@@ -81,13 +86,13 @@ Method 1: Install the full neptune-notebooks package through the command line
 
 This is the most comprehensive type of installation:
 
-- It includes extensions for both standard Jupyter and JupyterLab. 
+- It includes extensions for both standard Jupyter and JupyterLab.
 - It also installs a CLI for uploading Notebook files.
 - The the neptune-client package is a dependency.
 
 **Cons**
 
-- After the installation, you still have to perform some manual actions in JupyterLab. 
+- After the installation, you still have to perform some manual actions in JupyterLab.
 - This method may not be the easiest way of installing extensions in JupyterLab.
 
 **Procedure**
@@ -123,7 +128,8 @@ The neptune-notebooks pip package, neptune-client pip package and extension to t
 
 **Pros**
 
-This method gives you more flexibility. For example, the extension can be running on a 
+
+This method gives you more flexibility. For example, the extension can be running on a
 different machine than your code that is executed from the Notebook.
 
 **Procedure**