Skip to content

Developing deep machine learning models in MaxMSP

Notifications You must be signed in to change notification settings

esling/maxmsp_ai

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 

Repository files navigation


ACIDS
Deep machine learning in MaxMSP and Ableton Live

Deep machine Learning course for MaxMSP with tutorials in JAX, PyTorch and Numpy.

Course given at the MaxMSP summer school (Tokyo university of the arts, Japan)
Professor: Philippe Esling

This repository contains the courses in creative machine learning applied to music and other creative mediums and how to develop and use these models inside MaxMSP and Ableton Live. This course is given at the MaxMSP summer school ((Tokyo university of the arts, Japan)). The courses slides along with a set of MaxMSP patches, Max4Live devices and interactive Jupyter Notebooks will be updated along the course to provide all the examples. This course is proudly provided by the ACIDS group, part of the Analysis / Synthesis team at IRCAM. This course can be followed entirely online through the set of Google slides and Colab notebooks links provided openly along each lesson. However, we do recommend to fork the entire environment and follow the interactive notebooks through Jupyter lab to develop your own coding environment.

This course is meant to be only an introduction to all of these complex ideas. We highly recommend to follow the companion "Creative Machine Learning" course to truly acheive mastery of deep models development.

Table of Contents
  1. Lessons
  2. Detailed lessons
  3. Contribution
  4. About

Lessons

Quick explanation. For each of the following lessons, you will find a set of badges containing links to different parts of the course, which allows you to follow either the online or offline versions.

  • Online: Slides Colab
  • Offline: Powerpoint Notebook

Simply click on the corresponding badge to follow the lesson. Note that if the badge is displayed in red color as follows Slides it means that the content is not available yet and will be uploaded later.


Slides Powerpoint

This course provides a brief history of the development of artificial intelligence and introduces the general concepts of machine learning through a series of recent applications in the creative fields. This course also presents the pre-requisites, course specificities, toolboxes and tutorials that will be covered and how to setup the overall environment. Finally, we introduce the formal notions required to understand machine learning along with classic problems of linear models for regression and classification. We discuss the mathematical derivation for optimization and various problems of overfitting, cross-validation and model properties and complexity that are still quintessential in modern machine learning. We finish with a quick roundup of how to use existing deep models on GitHub.


Slides Powerpoint

This course provides a brief history of the development of neural networks along with all mathematical and implementation details. We discuss geometric perspectives on neurons and gradient descent and how these interpretation naturally extend to the case of multi-layer perceptrons. We further introduce more advanced types of neural networks such as convolutional and recurrent architectures, along with more advanced models (LSTM, GRU) and recent developments such as residual architectures. We further discuss issues of regularization and initialization in networks. Finally, we finish this course by discussing the recent attention mechanism and transformer architectures and provide a set of modern applications.


Slides Powerpoint

In this course, we will discuss how to embed existing deep models into MaxMSP. In order to capitalize on existing body of work in ML research, we will seek to run the model from Python (as most research is being done in this language). Hence, we will first discuss OSC communication and how to setup an OSC protocol and exchange between MaxMSP and Python. We briefly introduce the basics of Max4Live devices and show how to use these through the case study of the FlowSynth AI-based synthesizer control.


Slides Powerpoint

In this course, we go further in how we can potentially embed existing deep models into MaxMSP. One of the key problem of the previous method (OSC and Python) resides in the lack of computational efficiency. Hence, we seek here to learn how to develop our own externals. We provide an introduction to the Max SDK in order to develop our own externals in C. We further discuss the Torchscript and Tensorflow Lite interfaces that allow to embed Python models into efficient C codes. We show how to use these skills through the case study of the RAVE real-time deep audio generation.


Setup

Along the tutorials, we provide a reference code for each section. This code contains helper functions that will alleviate you from the burden of data import and other sideline implementations. You will find designated spaces in each file to develop your solutions. The code is in Python (notebooks impending) and relies on the concept of code sections, which allows you to evaluate only part of the code (to avoid running long import tasks multiple times and concentrate on the question at hand.

Please refer to the setup notebook to check if your configuration is correct

Dependencies

Python installation

In order to get the baseline scripts and notebooks to work, you need to have a working distribution of Python 3.7 as a minimum (we also recommend to update your version to Python 3.9). We will also be using a large set of libraries, with the following ones being the most prohiminent

We highly recommend that you install Pip or Anaconda that will manage the automatic installation of those Python libraries (along with their dependencies). If you are using Pip, you can use the following commands

pip install -r requirements.txt

If you prefer to install all the libraries by hand to check their version, you can use individual commands

pip install numpy
pip install scikit-learn
pip install torch
pip install jax
pip install librosa
pip install matplotlib

For those of you who have never coded in Python, here are a few interesting resources to get started.

Jupyter notebooks and lab

In order to ease following the exercises along with the course, we will be relying on Jupyter Notebooks. If you have never used a notebook before, we recommend that you look at their website to understand the concept. Here we also provide the instructions to install Jupyter Lab which is a more integrative version of notebooks. You can install it on your computer as follows (if you use pip)

pip install jupyterlab

Then, once installed, you can go to the folder where you cloned this repository, and type in

jupyter lab

Contribution

Please take a look at our contributing guidelines if you're interested in helping!

About

Code and documentation copyright 2012-2023 by all members of ACIDS.

Code released under the CC-BY-NC-SA 4.0 licence.

About

Developing deep machine learning models in MaxMSP

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published