diff --git a/morea/neural-networks/#experience-auto.md# b/morea/neural-networks/#experience-auto.md# new file mode 100644 index 0000000..f17d10a --- /dev/null +++ b/morea/neural-networks/#experience-auto.md# @@ -0,0 +1,40 @@ +--- +title: "Autoencoders" +published: true +morea_id: experience-auto +morea_type: experience +morea_summary: "Autoencoders and SVD" +morea_start_date: "2021-07-15T23:00" +morea_labels: +--- + +First review the section about eigenvalues and eigenspaces +[here]. Recall that \({\mathbb R}^p\) represents the linear space of +all vectors with \(p\) real coordinates. For a matrix \(n\times p\) +matrix \(X\), one can use spectral decomposition of \(X^TX\) +(respectively \(XX^T\)) to find an orthonormal basis for \({\mathbb +R}^p\) (respectively \({\mathbb R}^n\)) using eigenvectors of \(X^TX\) +(respectively \(XX^T\)), and therefore for the rows (respectively +columns) of \(X\). Assume that \(n \ge p\), and let the eigenvalues +of \(X^TX\) be \(\lambda_1\ge \lambda_2 \cdots \ge \lambda_p\), then +the highest \(p\) eigenvalues of \(XX^T\) are also +\(\lambda_1, \lambda_2 \upto \lambda_p\), while the remaining \(n-p\) +eigenvalues are all 0. + + +Let \(V\) (respectively \(U\)) be the matrix formed +by placing as columns the orthonormal basis obtained by the +eigendecomposition of \(X^TX\) (respectively \(XX^T\)). +Let \(\Sigma\) +be the \(n\times p\) matrix formed with the positive square roots of +the eigenvalues of \(X^TX\) in all the diagonal locations. +The singular value decomposition observes that +$$ X = U \Sigma V^T. $$ + +The following +[notebook](https://uhm-descartes.github.io/ee445/morea/neural-networks/autoencoder.ipynb) +shows you how to build a neural network that recovers the singular +value decomposition, the autoencoder. In the assessment, you will +reuse the code, but with non-linear activations to project the MNIST +dataset into a low dimensional manifold. + diff --git a/morea/neural-networks/.#experience-auto.md b/morea/neural-networks/.#experience-auto.md new file mode 100644 index 0000000..92b91a3 --- /dev/null +++ b/morea/neural-networks/.#experience-auto.md @@ -0,0 +1 @@ +nsant@Dell-XPS15.10720:1683013381 \ No newline at end of file diff --git a/morea/neural-networks/assessment-nn.md b/morea/neural-networks/assessment-nn.md index 60a85a2..18cccee 100644 --- a/morea/neural-networks/assessment-nn.md +++ b/morea/neural-networks/assessment-nn.md @@ -1,14 +1,28 @@ --- -title: "CHANGE ME" -published: false -morea_id: assessment-CHANGE-ME -morea_summary: "CHANGE ME" +title: "Assessment" +published: true +morea_id: assessment-nn +morea_summary: "Neural Networks" morea_outcomes_assessed: # - outcome-CHANGE-ME morea_type: assessment morea_start_date: "2021-07-16T09:00" morea_labels: --- -# CHANGE ME +# Problems -TBD +* Use a feedforward architecture to train and predict on the CIFAR-10 + and Fashion-MNIST dataset. Here, you may need to use dropout to + train better and reduce overfitting---find out how to implement this + technique. We discussed dropout very briefly in class, so you may want + to look up Dropout techniques online for more background. + +* Project the MNIST dataset into as small a manifold as + possible. Meaning, you should come up with two transformations (we + will call them encoder and decoder). The encoder should represent + each \(28\times 28\) test image into a small vector (you can have + this vector have less than 10 coordinates, but it is ok if your + output is slightly larger too), but that doesn't lose + information---namely the decoder can reconstruct the original image + (with negligible loss) from the small vector output from the + encoder. diff --git a/morea/neural-networks/assessment-nn.md~ b/morea/neural-networks/assessment-nn.md~ new file mode 100644 index 0000000..60a85a2 --- /dev/null +++ b/morea/neural-networks/assessment-nn.md~ @@ -0,0 +1,14 @@ +--- +title: "CHANGE ME" +published: false +morea_id: assessment-CHANGE-ME +morea_summary: "CHANGE ME" +morea_outcomes_assessed: + # - outcome-CHANGE-ME +morea_type: assessment +morea_start_date: "2021-07-16T09:00" +morea_labels: +--- +# CHANGE ME + +TBD diff --git a/morea/neural-networks/module-nn.md b/morea/neural-networks/module-nn.md index 29434f6..d365bd7 100644 --- a/morea/neural-networks/module-nn.md +++ b/morea/neural-networks/module-nn.md @@ -18,7 +18,7 @@ morea_icon_url: /morea/CHANGE-ME/CHANGE-ME.png morea_start_date: "2021-07-12" morea_end_date: "2021-07-16" morea_labels: -morea_sort_order: 21 +morea_sort_order: 81 --- CHANGE ME