readme update

01_machine_learning_for_trading/README.md

@@ -16,9 +16,7 @@ Historically, algorithmic trading used to be more narrowly defined as the automa
 
 This chapter looks at industry trends that have led to the emergence of ML as a source of competitive advantage in the investment industry. We will also look at where ML fits into the investment process to enable algorithmic trading strategies. 
 
-More specifically, we will be covering the following topics:
-
-#### Table of contents
+## Content
 
 1. [The rise of ML in the investment industry](#the-rise-of-ml-in-the-investment-industry)
     * [From electronic to high-frequency trading](#from-electronic-to-high-frequency-trading)

02_market_and_fundamental_data/README.md

@@ -12,8 +12,6 @@ This chapter introduces market and fundamental data sources and explains how the
 
 ## Content
 
-#### Table of contents
-
 1. [Market data reflects the trading environment](#market-data-reflects-the-trading-environment)
     * [Market microstructure: The nuts and bolts of trading](#market-microstructure-the-nuts-and-bolts-of-trading)
 2. [Working with high-frequency market data](#working-with-high-frequency-market-data)

03_alternative_data/README.md

@@ -12,6 +12,19 @@ It demonstrates the workflow, from acquisition to preprocessing and storage usin
 - How to work with alternative data in Python, such as by scraping the internet
 - Important categories and providers of alternative data
 
+## Content
+
+1. [The Alternative Data Revolution](#the-alternative-data-revolution)
+    * [Resources](#resources)
+2. [Sources of alternative data](#sources-of-alternative-data)
+3. [Criteria for evaluating alternative datasets](#criteria-for-evaluating-alternative-datasets)
+    * [Resources](#resources-2)
+4. [The Market for Alternative Data](#the-market-for-alternative-data)
+5. [Working with Alternative Data](#working-with-alternative-data)
+    * [Code Example: Open Table Web Scraping](#code-example-open-table-web-scraping)
+    * [Code Example: SeekingAlpha Earnings Transcripts](#code-example-seekingalpha-earnings-transcripts)
+    * [Python Libraries & Documentation](#python-libraries--documentation)
+
 ## The Alternative Data Revolution
 
 For algorithmic trading, new data sources offer an informational advantage if they provide access to information unavailable from traditional sources, or provide access sooner. Following global trends, the investment industry is rapidly expanding beyond market and fundamental data to alternative sources to reap alpha through an informational edge. Annual spending on data, technological capabilities, and related talent are expected to increase from the current $3 billion by 12.8% annually through 2020.
@@ -27,7 +40,18 @@ Today, investors can access macro or company-specific data in real-time that his
 - [Big data: The next frontier for innovation, competition, and productivity](https://www.mckinsey.com/business-functions/digital-mckinsey/our-insights/big-data-the-next-frontier-for-innovation), McKinsey 2011
 - [McKinsey on Artificial Intelligence](https://www.mckinsey.com/featured-insights/artificial-intelligence)
 
-## Evaluating Alternative Datasets
+## Sources of alternative data
+
+Alternative datasets are generated by many sources but can be classified at a high level as predominantly produced by:
+- Individuals who post on social media, review products, or use search engines
+- Businesses that record commercial transactions, in particular, credit card payments, or capture supply-chain activity as intermediaries
+- Sensors that, among many other things, capture economic activity through images such as satellites or security cameras, or through movement patterns such as cell phone towers
+
+The nature of alternative data continues to evolve rapidly as new data sources become available and sources previously labeled “alternative” become part of the mainstream. The Baltic Dry Index (BDI), for instance, assembles data from several hundred shipping companies to approximate the supply/demand of dry bulk carriers and is now available on the Bloomberg Terminal.
+
+Alternative data sources differ in crucial respects that determine their value or signal content for algorithmic trading strategies.
+
+## Criteria for evaluating alternative datasets
 
 The ultimate objective of alternative data is to provide an informational advantage in the competitive search for trading signals that produce alpha, namely positive, uncorrelated investment returns. In practice, the signals extracted from alternative datasets can be used on a standalone basis or combined with other signals as part of a quantitative strategy.
 
@@ -48,17 +72,17 @@ This section illustrates the acquisition of alternative data using web scraping,
 - [Quantifying Trading Behavior in Financial Markets Using Google Trends](https://www.nature.com/articles/srep01684), Preis, Moat and Stanley, Nature, 2013
 - [Quantifying StockTwits semantic terms’ trading behavior in financial markets: An effective application of decision tree algorithms](https://www.sciencedirect.com/science/article/pii/S0957417415005473), Al Nasseri et al, Expert Systems with Applications, 2015
 
-#### Open Table Web Scraping
+### Code Example: Open Table Web Scraping
 
 This subfolder [01_opentable](01_opentable) contains the script [opentable_selenium](01_opentable/opentable_selenium.py) to scrape OpenTable data using Scrapy and Selenium.
 
 - [How to View the Source Code of a Web Page in Every Browser](https://www.lifewire.com/view-web-source-code-4151702)
 
-#### SeekingAlpha Earnings Transcripts
+### Code Example: SeekingAlpha Earnings Transcripts
 
 The subfolder [02_earnings_calls](02_earnings_calls) contains the script [sa_selenium](02_earnings_calls/sa_selenium.py) to scrape earnings call transcripts from the [SeekingAlpha](www.seekingalpha.com) website.
 
-#### Python Libraries & Documentation
+## Python Libraries & Documentation
 - requests [docs](http://docs.python-requests.org/en/master/)
 - beautifulsoup [docs](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)
 - Selenium [docs](https://www.seleniumhq.org/)

04_alpha_factor_research/README.md

@@ -17,16 +17,35 @@ In particular, this chapter will address the following topics:
 
 Please see the [Appendix - Alpha Factor Library](../24_alpha_factor_library) for additional material on this topic, including numerous code examples that compute a broad range of alpha factors.
 
+## Content
+
+1. [Alpha Factors in practice: from data to signals](#alpha-factors-in-practice-from-data-to-signals)
+2. [Building on Decades of Factor Research](#building-on-decades-of-factor-research)
+    * [References](#references)
+3. [Engineering alpha factors that predict returns](#engineering-alpha-factors-that-predict-returns)
+    * [Code Example: How to engineer factors using pandas and NumPy](#code-example-how-to-engineer-factors-using-pandas-and-numpy)
+    * [Code Example: How to use TA-Lib to create technical alpha factors](#code-example-how-to-use-ta-lib-to-create-technical-alpha-factors)
+    * [Code Example: How to denoise your Alpha Factors with the Kalman Filter](#code-example-how-to-denoise-your-alpha-factors-with-the-kalman-filter)
+    * [Code Example: How to preprocess your noisy signals using Wavelets](#code-example-how-to-preprocess-your-noisy-signals-using-wavelets)
+    * [Resources](#resources)
+4. [From signals to trades: backtesting with `Zipline`](#from-signals-to-trades-backtesting-with-zipline)
+    * [Code Example: How to use Zipline to backtest a single-factor strategy](#code-example-how-to-use-zipline-to-backtest-a-single-factor-strategy)
+    * [Code Example: Combining factors from diverse data sources on the Quantopian platform](#code-example-combining-factors-from-diverse-data-sources-on-the-quantopian-platform)
+    * [Code Example: Separating signal and noise – how to use alphalens](#code-example-separating-signal-and-noise--how-to-use-alphalens)
+5. [Alternative Algorithmic Trading Libraries and Platforms](#alternative-algorithmic-trading-libraries-and-platforms)
+
 ## Alpha Factors in practice: from data to signals
 
 Alpha factors are transformations of market, fundamental, and alternative data that contain predictive signals. They are designed to capture risks that drive asset returns. One set of factors describes fundamental, economy-wide variables such as growth, inflation, volatility, productivity, and demographic risk. Another set consists of tradeable investment styles such as the market portfolio, value-growth investing, and momentum investing.
 
 There are also factors that explain price movements based on the economics or institutional setting of financial markets, or investor behavior, including known biases of this behavior. The economic theory behind factors can be rational, where the factors have high returns over the long run to compensate for their low returns during bad times, or behavioral, where factor risk premiums result from the possibly biased, or not entirely rational behavior of agents that is not arbitraged away.
 
-### On the shoulders of giants: meet the factor establishment
+## Building on Decades of Factor Research
 
 In an idealized world, categories of risk factors should be independent of each other (orthogonal), yield positive risk premia, and form a complete set that spans all dimensions of risk and explains the systematic risks for assets in a given class. In practice, these requirements will hold only approximately.
 
+### References
+
 - [Dissecting Anomalies](http://schwert.ssb.rochester.edu/f532/ff_JF08.pdf) by Eugene Fama and Ken French (2008)
 - [Explaining Stock Returns: A Literature Review](https://www.ifa.com/pdfs/explainingstockreturns.pdf) by James L. Davis (2001)
 - [Market Efficiency, Long-Term Returns, and Behavioral Finance](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=15108) by Eugene Fama (1997)
@@ -43,18 +62,6 @@ Based on a conceptual understanding of key factor categories, their rationale an
 
 The notebook [feature_engineering.ipynb](00_data/feature_engineering.ipynb) in the [data](00_data) directory illustrates how to engineer basic factors.
 
-#### Additional Resources
-
-- [Fama French](https://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html) Data Library
-- [numpy](https://numpy.org/) website
-    - [Quickstart Tutorial](https://numpy.org/devdocs/user/quickstart.html)
-- [pandas](https://pandas.pydata.org/) website
-    - [User Guide](https://pandas.pydata.org/docs/user_guide/index.html)
-    - [10 minutes to pandas](https://pandas.pydata.org/pandas-docs/stable/getting_started/10min.html)
-    - [Python Pandas Tutorial: A Complete Introduction for Beginners](https://www.learndatasci.com/tutorials/python-pandas-tutorial-complete-introduction-for-beginners/)
-- [alphatools](https://github.com/marketneutral/alphatools) - Quantitative finance research tools in Python
-- [mlfinlab](https://github.com/hudson-and-thames/mlfinlab) - Package based on the work of Dr Marcos Lopez de Prado regarding his research with respect to Advances in Financial Machine Learning
-
 ### Code Example: How to use TA-Lib to create technical alpha factors
 
 The notebook [how_to_use_talib](02_how_to_use_talib.ipynb) illustrates the usage of TA-Lib, which includes a broad range of common technical indicators. These indicators have in common that they only use market data, i.e., price and volume information.
@@ -65,41 +72,40 @@ The notebook [common_alpha_factors](../24_alpha_factor_library/02_common_alpha_f
 
 The notebook [kalman_filter_and_wavelets](03_kalman_filter_and_wavelets.ipynb) demonstrates the use of the Kalman filter using the `PyKalman` package for smoothing; we will also use it in [Chapter 9](../09_time_series_models) when we develop a pairs trading strategy.
 
-#### Additional Resources 
-
-- [PyKalman](https://pykalman.github.io/) documentation
-- [Tutorial: The Kalman Filter](http://web.mit.edu/kirtley/kirtley/binlustuff/literature/control/Kalman%20filter.pdf)
-- [Understanding and Applying Kalman Filtering](http://biorobotics.ri.cmu.edu/papers/sbp_papers/integrated3/kleeman_kalman_basics.pdf)
-- [How a Kalman filter works, in pictures](https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/)
-
 ### Code Example: How to preprocess your noisy signals using Wavelets
 
 The notebook [kalman_filter_and_wavelets](03_kalman_filter_and_wavelets.ipynb) also demonstrates how to work with wavelets using the `PyWavelets` package.
 
-#### Additional Resources 
+### Resources
 
+- [Fama French](https://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html) Data Library
+- [numpy](https://numpy.org/) website
+    - [Quickstart Tutorial](https://numpy.org/devdocs/user/quickstart.html)
+- [pandas](https://pandas.pydata.org/) website
+    - [User Guide](https://pandas.pydata.org/docs/user_guide/index.html)
+    - [10 minutes to pandas](https://pandas.pydata.org/pandas-docs/stable/getting_started/10min.html)
+    - [Python Pandas Tutorial: A Complete Introduction for Beginners](https://www.learndatasci.com/tutorials/python-pandas-tutorial-complete-introduction-for-beginners/)
+- [alphatools](https://github.com/marketneutral/alphatools) - Quantitative finance research tools in Python
+- [mlfinlab](https://github.com/hudson-and-thames/mlfinlab) - Package based on the work of Dr Marcos Lopez de Prado regarding his research with respect to Advances in Financial Machine Learning
+- [PyKalman](https://pykalman.github.io/) documentation
+- [Tutorial: The Kalman Filter](http://web.mit.edu/kirtley/kirtley/binlustuff/literature/control/Kalman%20filter.pdf)
+- [Understanding and Applying Kalman Filtering](http://biorobotics.ri.cmu.edu/papers/sbp_papers/integrated3/kleeman_kalman_basics.pdf)
+- [How a Kalman filter works, in pictures](https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/)
 - [PyWavelets](https://pywavelets.readthedocs.io/en/latest/) - Wavelet Transforms in Python
 - [An Introduction to Wavelets](https://www.eecis.udel.edu/~amer/CISC651/IEEEwavelet.pdf) 
 - [The Wavelet Tutorial](http://web.iitd.ac.in/~sumeet/WaveletTutorial.pdf)
 - [Wavelets for Kids](http://www.gtwavelet.bme.gatech.edu/wp/kidsA.pdf)
-
-### References
-
 - [The Barra Equity Risk Model Handbook](https://www.alacra.com/alacra/help/barra_handbook_GEM.pdf)
 - [Active Portfolio Management: A Quantitative Approach for Producing Superior Returns and Controlling Risk](https://www.amazon.com/Active-Portfolio-Management-Quantitative-Controlling/dp/0070248826) by Richard Grinold and Ronald Kahn, 1999
 - [Modern Investment Management: An Equilibrium Approach](https://www.amazon.com/Modern-Investment-Management-Equilibrium-Approach/dp/0471124109) by Bob Litterman, 2003
 - [Quantitative Equity Portfolio Management: Modern Techniques and Applications](https://www.crcpress.com/Quantitative-Equity-Portfolio-Management-Modern-Techniques-and-Applications/Qian-Hua-Sorensen/p/book/9781584885580) by Edward Qian, Ronald Hua, and Eric Sorensen
 - [Spearman Rank Correlation](https://statistics.laerd.com/statistical-guides/spearmans-rank-order-correlation-statistical-guide.php)
 
-## From signals to trades: backtesting with `zipline`
+## From signals to trades: backtesting with `Zipline`
 
 The open source [zipline](http://www.zipline.io/index.html) library is an event-driven backtesting system maintained and used in production by the crowd-sourced quantitative investment fund [Quantopian](https://www.quantopian.com/) to facilitate algorithm-development and live-trading. It automates the algorithm's reaction to trade events and provides it with current and historical point-in-time data that avoids look-ahead bias.
 
-[Chapter 8](../08_ml4t_workflow) contains a more comprehensive introduction to Zipline.
-
-### Installation
-- The current release 1.3 has a few shortcomings such as the [dependency on benchmark data from the IEX exchange](https://github.com/quantopian/zipline/issues/2480) and limitations for importing features beyond the basic OHLCV data points.
-- To enable the use of `zipline`, I've provided a [patched version](https://github.com/stefan-jansen/zipline) that works for the purposes of this book.
+- [Chapter 8](../08_ml4t_workflow) contains a more comprehensive introduction to Zipline.
 - Please follow the [instructions](../installation) in the `installation` folder.
 
 ### Code Example: How to use Zipline to backtest a single-factor strategy
@@ -127,10 +133,11 @@ The analysis can be conducted using `tearsheets` or individual computations and
 
 - See [here](https://github.com/quantopian/alphalens/blob/master/alphalens/examples/alphalens_tutorial_on_quantopian.ipynb) for a detailed `alphalens` tutorial by Quantopian
 
-### Alternative Algorithmic Trading Libraries
+## Alternative Algorithmic Trading Libraries and Platforms
 
 - [QuantConnect](https://www.quantconnect.com/)
 - [Alpha Trading Labs](https://www.alphalabshft.com/)
+    - Alpha Trading Labs is no longer active
 - [WorldQuant](https://www.worldquantvrc.com/en/cms/wqc/home/)
 - Python Algorithmic Trading Library [PyAlgoTrade](http://gbeced.github.io/pyalgotrade/)
 - [pybacktest](https://github.com/ematvey/pybacktest)