Skip to content

Commit

Permalink
Merge pull request torch#348 from nicholas-leonard/readthedocs
Browse files Browse the repository at this point in the history 
nn.readthedocs.org
  • Loading branch information
@soumith
soumith committed Aug 12, 2015
2 parents 0f5c1cc + 0e05ac9 commit 14599d4
Show file tree
Hide file tree
Showing 12 changed files with 308 additions and 259 deletions.
2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[![Build Status](https://travis-ci.org/torch/nn.svg?branch=master)](https://travis-ci.org/torch/nn)
<a name="nn.dok"/>
<a name="nn.dok"></a>
# Neural Network Package #

This package provides an easy and modular way to build and train simple or complex neural networks using [Torch](https://github.com/torch/torch7/blob/master/README.md):
Expand Down
35 changes: 18 additions & 17 deletions doc/containers.md
View file
Original file line number Diff line number Diff line change
@@ -1,35 +1,36 @@
<a name="nn.Containers"/>
<a name="nn.Containers"></a>
# Containers #
Complex neural networks are easily built using container classes:
* [Container](#nn.Container) : abstract class inherited by containers ;
* [Sequential](#nn.Sequential) : plugs layers in a feed-forward fully connected manner ;
* [Parallel](#nn.Parallel) : applies its `ith` child module to the `ith` slice of the input Tensor ;
* [Concat](#nn.Concat) : concatenates in one layer several modules along dimension `dim` ;
* [DepthConcat](#nn.DepthConcat) : like Concat, but adds zero-padding when non-`dim` sizes don't match;

* [Container](#nn.Container) : abstract class inherited by containers ;
* [Sequential](#nn.Sequential) : plugs layers in a feed-forward fully connected manner ;
* [Parallel](#nn.Parallel) : applies its `ith` child module to the `ith` slice of the input Tensor ;
* [Concat](#nn.Concat) : concatenates in one layer several modules along dimension `dim` ;
* [DepthConcat](#nn.DepthConcat) : like Concat, but adds zero-padding when non-`dim` sizes don't match;

See also the [Table Containers](#nn.TableContainers) for manipulating tables of [Tensors](https://github.com/torch/torch7/blob/master/doc/tensor.md).

<a name="nn.Container"/>
<a name="nn.Container"></a>
## Container ##

This is an abstract [Module](module.md#nn.Module) class which declares methods defined in all containers.
It reimplements many of the Module methods such that calls are propagated to the
contained modules. For example, a call to [zeroGradParameters](module.md#nn.Module.zeroGradParameters)
will be propagated to all contained modules.

<a name="nn.Container.add"/>
<a name="nn.Container.add"></a>
### add(module) ###
Adds the given `module` to the container. The order is important

<a name="nn.Container.get"/>
<a name="nn.Container.get"></a>
### get(index) ###
Returns the contained modules at index `index`.

<a name="nn.Container.size"/>
<a name="nn.Container.size"></a>
### size() ###
Returns the number of contained modules.

<a name="nn.Sequential"/>
<a name="nn.Sequential"></a>
## Sequential ##

Sequential provides a means to plug layers together
Expand All @@ -51,7 +52,7 @@ which gives the output:
[torch.Tensor of dimension 1]
```

<a name="nn.Sequential.remove"/>
<a name="nn.Sequential.remove"></a>
### remove([index]) ###

Remove the module at the given `index`. If `index` is not specified, remove the last layer.
Expand All @@ -71,7 +72,7 @@ nn.Sequential {
```


<a name="nn.Sequential.insert"/>
<a name="nn.Sequential.insert"></a>
### insert(module, [index]) ###

Inserts the given `module` at the given `index`. If `index` is not specified, the incremented length of the sequence is used and so this is equivalent to use `add(module)`.
Expand All @@ -92,7 +93,7 @@ nn.Sequential {



<a name="nn.Parallel"/>
<a name="nn.Parallel"></a>
## Parallel ##

`module` = `Parallel(inputDimension,outputDimension)`
Expand Down Expand Up @@ -149,7 +150,7 @@ end
```


<a name="nn.Concat"/>
<a name="nn.Concat"></a>
## Concat ##

```lua
Expand Down Expand Up @@ -179,7 +180,7 @@ which gives the output:
[torch.Tensor of dimension 10]
```

<a name="nn.DepthConcat"/>
<a name="nn.DepthConcat"></a>
## DepthConcat ##

```lua
Expand Down Expand Up @@ -273,7 +274,7 @@ module output tensors non-`dim` sizes aren't all odd or even.
Such that in order to keep the mappings aligned, one need
only ensure that these be all odd (or even).

<a name="nn.TableContainers"/>
<a name="nn.TableContainers"></a>
## Table Containers ##
While the above containers are used for manipulating input [Tensors](https://github.com/torch/torch7/blob/master/doc/tensor.md), table containers are used for manipulating tables :
* [ConcatTable](table.md#nn.ConcatTable)
Expand Down
91 changes: 46 additions & 45 deletions doc/convolution.md
View file
Original file line number Diff line number Diff line change
@@ -1,31 +1,32 @@
<a name="nn.convlayers.dok"/>
<a name="nn.convlayers.dok"></a>
# Convolutional layers #

A convolution is an integral that expresses the amount of overlap of one function `g` as it is shifted over another function `f`. It therefore "blends" one function with another. The neural network package supports convolution, pooling, subsampling and other relevant facilities. These are divided base on the dimensionality of the input and output [Tensors](https://github.com/torch/torch7/blob/master/doc/tensor.md#tensor):
* [Temporal Modules](#nn.TemporalModules) apply to sequences with a one-dimensional relationship

* [Temporal Modules](#nn.TemporalModules) apply to sequences with a one-dimensional relationship
(e.g. sequences of words, phonemes and letters. Strings of some kind).
* [TemporalConvolution](#nn.TemporalConvolution) : a 1D convolution over an input sequence ;
* [TemporalSubSampling](#nn.TemporalSubSampling) : a 1D sub-sampling over an input sequence ;
* [TemporalMaxPooling](#nn.TemporalMaxPooling) : a 1D max-pooling operation over an input sequence ;
* [LookupTable](#nn.LookupTable) : a convolution of width `1`, commonly used for word embeddings ;
* [Spatial Modules](#nn.SpatialModules) apply to inputs with two-dimensional relationships (e.g. images):
* [SpatialConvolution](#nn.SpatialConvolution) : a 2D convolution over an input image ;
* [SpatialSubSampling](#nn.SpatialSubSampling) : a 2D sub-sampling over an input image ;
* [SpatialMaxPooling](#nn.SpatialMaxPooling) : a 2D max-pooling operation over an input image ;
* [SpatialAveragePooling](#nn.SpatialAveragePooling) : a 2D average-pooling operation over an input image ;
* [SpatialAdaptiveMaxPooling](#nn.SpatialAdaptiveMaxPooling) : a 2D max-pooling operation which adapts its parameters dynamically such that the output is of fixed size ;
* [SpatialLPPooling](#nn.SpatialLPPooling) : computes the `p` norm in a convolutional manner on a set of input images ;
* [SpatialConvolutionMap](#nn.SpatialConvolutionMap) : a 2D convolution that uses a generic connection table ;
* [SpatialZeroPadding](#nn.SpatialZeroPadding) : padds a feature map with specified number of zeros ;
* [SpatialSubtractiveNormalization](#nn.SpatialSubtractiveNormalization) : a spatial subtraction operation on a series of 2D inputs using
* [SpatialBatchNormalization](#nn.SpatialBatchNormalization): mean/std normalization over the mini-batch inputs and pixels, with an optional affine transform that follows
* [TemporalConvolution](#nn.TemporalConvolution) : a 1D convolution over an input sequence ;
* [TemporalSubSampling](#nn.TemporalSubSampling) : a 1D sub-sampling over an input sequence ;
* [TemporalMaxPooling](#nn.TemporalMaxPooling) : a 1D max-pooling operation over an input sequence ;
* [LookupTable](#nn.LookupTable) : a convolution of width `1`, commonly used for word embeddings ;
* [Spatial Modules](#nn.SpatialModules) apply to inputs with two-dimensional relationships (e.g. images):
* [SpatialConvolution](#nn.SpatialConvolution) : a 2D convolution over an input image ;
* [SpatialSubSampling](#nn.SpatialSubSampling) : a 2D sub-sampling over an input image ;
* [SpatialMaxPooling](#nn.SpatialMaxPooling) : a 2D max-pooling operation over an input image ;
* [SpatialAveragePooling](#nn.SpatialAveragePooling) : a 2D average-pooling operation over an input image ;
* [SpatialAdaptiveMaxPooling](#nn.SpatialAdaptiveMaxPooling) : a 2D max-pooling operation which adapts its parameters dynamically such that the output is of fixed size ;
* [SpatialLPPooling](#nn.SpatialLPPooling) : computes the `p` norm in a convolutional manner on a set of input images ;
* [SpatialConvolutionMap](#nn.SpatialConvolutionMap) : a 2D convolution that uses a generic connection table ;
* [SpatialZeroPadding](#nn.SpatialZeroPadding) : padds a feature map with specified number of zeros ;
* [SpatialSubtractiveNormalization](#nn.SpatialSubtractiveNormalization) : a spatial subtraction operation on a series of 2D inputs using
* [SpatialBatchNormalization](#nn.SpatialBatchNormalization): mean/std normalization over the mini-batch inputs and pixels, with an optional affine transform that follows
a kernel for computing the weighted average in a neighborhood ;
* [Volumetric Modules](#nn.VolumetricModules) apply to inputs with three-dimensional relationships (e.g. videos) :
* [VolumetricConvolution](#nn.VolumetricConvolution) : a 3D convolution over an input video (a sequence of images) ;
* [VolumetricMaxPooling](#nn.VolumetricMaxPooling) : a 3D max-pooling operation over an input video.
* [VolumetricAveragePooling](#nn.VolumetricAveragePooling) : a 3D average-pooling operation over an input video.
* [Volumetric Modules](#nn.VolumetricModules) apply to inputs with three-dimensional relationships (e.g. videos) :
* [VolumetricConvolution](#nn.VolumetricConvolution) : a 3D convolution over an input video (a sequence of images) ;
* [VolumetricMaxPooling](#nn.VolumetricMaxPooling) : a 3D max-pooling operation over an input video.
* [VolumetricAveragePooling](#nn.VolumetricAveragePooling) : a 3D average-pooling operation over an input video.

<a name="nn.TemporalModules"/>
<a name="nn.TemporalModules"></a>
## Temporal Modules ##
Excluding an optional first batch dimension, temporal layers expect a 2D Tensor as input. The
first dimension is the number of frames in the sequence (e.g. `nInputFrame`), the last dimenstion
Expand All @@ -35,7 +36,7 @@ of dimensions, although the size of each dimension may change. These are commonl
Note: The [LookupTable](#nn.LookupTable) is special in that while it does output a temporal Tensor of size `nOutputFrame x outputFrameSize`,
its input is a 1D Tensor of indices of size `nIndices`. Again, this is excluding the option first batch dimension.

<a name="nn.TemporalConvolution"/>
<a name="nn.TemporalConvolution"></a>
## TemporalConvolution ##

```lua
Expand Down Expand Up @@ -121,7 +122,7 @@ which gives:
-0.63871422284166
```

<a name="nn.TemporalMaxPooling"/>
<a name="nn.TemporalMaxPooling"></a>
## TemporalMaxPooling ##

```lua
Expand All @@ -139,7 +140,7 @@ If the input sequence is a 2D tensor of dimension `nInputFrame x inputFrameSize`
nOutputFrame = (nInputFrame - kW) / dW + 1
```

<a name="nn.TemporalSubSampling"/>
<a name="nn.TemporalSubSampling"></a>
## TemporalSubSampling ##

```lua
Expand Down Expand Up @@ -175,7 +176,7 @@ The output value of the layer can be precisely described as:
output[i][t] = bias[i] + weight[i] * sum_{k=1}^kW input[i][dW*(t-1)+k)]
```

<a name="nn.LookupTable"/>
<a name="nn.LookupTable"></a>
## LookupTable ##

```lua
Expand Down Expand Up @@ -253,13 +254,13 @@ Outputs something like:
[torch.DoubleTensor of dimension 2x4x3]
```

<a name="nn.SpatialModules"/>
<a name="nn.SpatialModules"></a>
## Spatial Modules ##
Excluding and optional batch dimension, spatial layers expect a 3D Tensor as input. The
first dimension is the number of features (e.g. `frameSize`), the last two dimenstions
are spatial (e.g. `height x width`). These are commonly used for processing images.

<a name="nn.SpatialConvolution"/>
<a name="nn.SpatialConvolution"></a>
### SpatialConvolution ###

```lua
Expand Down Expand Up @@ -303,7 +304,7 @@ output[i][j][k] = bias[k]
```


<a name="nn.SpatialConvolutionMap"/>
<a name="nn.SpatialConvolutionMap"></a>
### SpatialConvolutionMap ###

```lua
Expand All @@ -317,7 +318,7 @@ connection table between input and output features. The
using a [full connection table](#nn.tables.full). One can specify
different types of connection tables.

<a name="nn.tables.full"/>
<a name="nn.tables.full"></a>
#### Full Connection Table ####

```lua
Expand All @@ -327,7 +328,7 @@ table = nn.tables.full(nin,nout)
This is a precomputed table that specifies connections between every
input and output node.

<a name="nn.tables.onetoone"/>
<a name="nn.tables.onetoone"></a>
#### One to One Connection Table ####

```lua
Expand All @@ -337,7 +338,7 @@ table = nn.tables.oneToOne(n)
This is a precomputed table that specifies a single connection to each
output node from corresponding input node.

<a name="nn.tables.random"/>
<a name="nn.tables.random"></a>
#### Random Connection Table ####

```lua
Expand All @@ -348,7 +349,7 @@ This table is randomly populated such that each output unit has
`nto` incoming connections. The algorihtm tries to assign uniform
number of outgoing connections to each input node if possible.

<a name="nn.SpatialLPPooling"/>
<a name="nn.SpatialLPPooling"></a>
### SpatialLPPooling ###

```lua
Expand All @@ -357,7 +358,7 @@ module = nn.SpatialLPPooling(nInputPlane, pnorm, kW, kH, [dW], [dH])

Computes the `p` norm in a convolutional manner on a set of 2D input planes.

<a name="nn.SpatialMaxPooling"/>
<a name="nn.SpatialMaxPooling"></a>
### SpatialMaxPooling ###

```lua
Expand All @@ -379,7 +380,7 @@ oheight = op((height + 2*padH - kH) / dH + 1)
`op` is a rounding operator. By default, it is `floor`. It can be changed
by calling `:ceil()` or `:floor()` methods.

<a name="nn.SpatialAveragePooling"/>
<a name="nn.SpatialAveragePooling"></a>
### SpatialAveragePooling ###

```lua
Expand All @@ -390,7 +391,7 @@ Applies 2D average-pooling operation in `kWxkH` regions by step size
`dWxdH` steps. The number of output features is equal to the number of
input planes.

<a name="nn.SpatialAdaptiveMaxPooling"/>
<a name="nn.SpatialAdaptiveMaxPooling"></a>
### SpatialAdaptiveMaxPooling ###

```lua
Expand All @@ -413,7 +414,7 @@ y_i_start = floor((i /oheight) * iheight)
y_i_end = ceil(((i+1)/oheight) * iheight)
```

<a name="nn.SpatialSubSampling"/>
<a name="nn.SpatialSubSampling"></a>
### SpatialSubSampling ###

```lua
Expand Down Expand Up @@ -454,7 +455,7 @@ output[i][j][k] = bias[k]
+ weight[k] sum_{s=1}^kW sum_{t=1}^kH input[dW*(i-1)+s)][dH*(j-1)+t][k]
```

<a name="nn.SpatialUpSamplingNearest"/>
<a name="nn.SpatialUpSamplingNearest"></a>
### SpatialUpSamplingNearest ###

```lua
Expand All @@ -475,7 +476,7 @@ output(u,v) = input(floor((u-1)/scale)+1, floor((v-1)/scale)+1)

Where `u` and `v` are index from 1 (as per lua convention). There are no learnable parameters.

<a name="nn.SpatialZeroPadding"/>
<a name="nn.SpatialZeroPadding"></a>
### SpatialZeroPadding ###

```lua
Expand All @@ -485,7 +486,7 @@ module = nn.SpatialZeroPadding(padLeft, padRight, padTop, padBottom)
Each feature map of a given input is padded with specified number of
zeros. If padding values are negative, then input is cropped.

<a name="nn.SpatialSubtractiveNormalization"/>
<a name="nn.SpatialSubtractiveNormalization"></a>
### SpatialSubtractiveNormalization ###

```lua
Expand Down Expand Up @@ -522,7 +523,7 @@ w2=image.display(processed)
```
![](image/lena.jpg)![](image/lenap.jpg)

<a name="nn.SpatialBatchNormalization"/>
<a name="nn.SpatialBatchNormalization"></a>
## SpatialBatchNormalization ##

`module` = `nn.SpatialBatchNormalization(N [,eps] [, momentum] [,affine])`
Expand Down Expand Up @@ -565,13 +566,13 @@ A = torch.randn(b, m, h, w)
C = model.forward(A) -- C will be of size `b x m x h x w`
```

<a name="nn.VolumetricModules"/>
<a name="nn.VolumetricModules"></a>
## Volumetric Modules ##
Excluding and optional batch dimension, volumetric layers expect a 4D Tensor as input. The
first dimension is the number of features (e.g. `frameSize`), the second is sequential (e.g. `time`) and the
last two dimenstions are spatial (e.g. `height x width`). These are commonly used for processing videos (sequences of images).

<a name="nn.VolumetricConvolution"/>
<a name="nn.VolumetricConvolution"></a>
### VolumetricConvolution ###

```lua
Expand Down Expand Up @@ -608,7 +609,7 @@ size `nOutputPlane x nInputPlane x kT x kH x kW`) and `self.bias` (Tensor of
size `nOutputPlane`). The corresponding gradients can be found in
`self.gradWeight` and `self.gradBias`.

<a name="nn.VolumetricMaxPooling"/>
<a name="nn.VolumetricMaxPooling"></a>
### VolumetricMaxPooling ###

```lua
Expand All @@ -619,7 +620,7 @@ Applies 3D max-pooling operation in `kTxkWxkH` regions by step size
`dTxdWxdH` steps. The number of output features is equal to the number of
input planes / dT.

<a name="nn.VolumetricAveragePooling"/>
<a name="nn.VolumetricAveragePooling"></a>
### VolumetricAveragePooling ###

```lua
Expand Down
Loading

0 comments on commit 14599d4

Please sign in to comment.