Skip to content

Classification , Regression , Neural Network , CNN , Transfer Learning , NLP with Tensorflow

Notifications You must be signed in to change notification settings

86lekwenshiung/Neural-Network-with-Tensorflow

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

89 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Tensorflow Developer Certification Learning Journey

Sources and Credits


  1. Zero to Mastery Deep Learning with TensorFlow course
  2. https://github.com/mrdbourke/tensorflow-deep-learning
  3. CNN Explainer
  4. Neural Network Playground
  5. Tensorflow Hub

Content Table

No. Notebook Dir Key Summary Data
01 TF Regression Typical Acrhitecture for Regression -
02 TF Classification Typical Acrhitecture for Classification -
03 TF CNN Typical Acrhitecture for CNN 10_food_classes_all_data
04 TF Transfer Learning : Feature Extraction Transfer Learning Feature Extraction
05 TF Transfer Learning : Fine Tuning Transfer Learning Fine Tuning 10_food_classes_all_data
06 TF Transfer Learning : Scaling Up Transfer Learning Scaling Up 101_food_classes_all_data
07 Natural Language Processing Natural Language Processing Techniques

1.0 Tensorflow Regression Basic Architecture

(back to top)

Hyperparameter Typical value
Input layer shape Same shape as number of features (e.g. 3 for # bedrooms, # bathrooms, # car spaces in housing price prediction)
Hidden layer(s) Problem specific, minimum = 1, maximum = unlimited
Neurons per hidden layer Problem specific, generally 10 to 100
Output layer shape Same shape as desired prediction shape (e.g. 1 for house price)
Hidden activation Usually ReLU (rectified linear unit)
Output activation None, ReLU, logistic/tanh
Loss function MSE (mean square error) or MAE (mean absolute error)/Huber (combination of MAE/MSE) if outliers
Optimizer SGD (stochastic gradient descent), Adam

Table 1: Typical architecture of a regression network. Source: Adapted from page 293 of Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow Book by Aurélien Géron


2.0 Tensorflow Classification Basic Architecture

(back to top)

Hyperparameter Binary Classification Multiclass classification
Input layer shape Same as number of features (e.g. 5 for age, sex, height, weight, smoking status in heart disease prediction) Same as binary classification
Hidden layer(s) Problem specific, minimum = 1, maximum = unlimited Same as binary classification
Neurons per hidden layer Problem specific, generally 10 to 100 Same as binary classification
Output layer shape 1 (one class or the other) 1 per class (e.g. 3 for food, person or dog photo)
Hidden activation Usually ReLU (rectified linear unit) Same as binary classification
Output activation Sigmoid Softmax
Loss function Cross entropy (tf.keras.losses.BinaryCrossentropy in TensorFlow) Cross entropy (tf.keras.losses.CategoricalCrossentropy in TensorFlow)
Optimizer SGD (stochastic gradient descent), Adam Same as binary classification

Table 1: Typical architecture of a classification network. Source: Adapted from page 295 of Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow Book by Aurélien Géron


3.0 Tensorflow CNN Basic Architecture

(back to top)

Hyperparameter/Layer type What does it do? Typical values
Input image(s) Target images you'd like to discover patterns in Whatever you can take a photo (or video) of
Input layer Takes in target images and preprocesses them for further layers input_shape = [batch_size, image_height, image_width, color_channels]
Convolution layer Extracts/learns the most important features from target images Multiple, can create with tf.keras.layers.ConvXD (X can be multiple values)
Hidden activation Adds non-linearity to learned features (non-straight lines) Usually ReLU (tf.keras.activations.relu)
Pooling layer Reduces the dimensionality of learned image features Average (tf.keras.layers.AvgPool2D) or Max (tf.keras.layers.MaxPool2D)
Fully connected layer Further refines learned features from convolution layers tf.keras.layers.Dense
Output layer Takes learned features and outputs them in shape of target labels output_shape = [number_of_classes] (e.g. 3 for pizza, steak or sushi)
Output activation Adds non-linearities to output layer tf.keras.activations.sigmoid (binary classification) or tf.keras.activations.softmax
Hyperparameter Name Description Typical Values
Filter How many filters should pass over an input tensors 10,32,64,128 (higher value , more complex
Kernel Size(filter size) Shape of the filter over the output 3,5,7, lower value = smaller features. Higher value = larger features
Padding Pad the target sensor with 0s at the border(if 'same') to preserve input shape. Or leaves in the target sensor(if 'valid') , lowering output shape 'same or 'valid'
Strides No. of steps a filter takes across an image at a time(if stride = 1 , a filter moves across an image 1 pixel at a time 1(default) ,2

4.0 Tensorflow Feature Extraction Overview

(back to top)

  1. Transfer learning is when you take a pretrained model as it is and apply it to your task without any changes.

    • For example, many computer vision models are pretrained on the ImageNet dataset which contains 1000 different classes of images. This means passing a single image to this model will produce 1000 different prediction probability values (1 for each class).
    • This is helpful if you have 1000 classes of image you'd like to classify and they're all the same as the ImageNet classes, however, it's not helpful if you want to classify only a small subset of classes (such as 10 different kinds of food). Model's with "/classification" in their name on TensorFlow Hub provide this kind of functionality.
  2. Feature extraction transfer learning is when you take the underlying patterns (also called weights) a pretrained model has learned and adjust its outputs to be more suited to your problem.

    • For example, say the pretrained model you were using had 236 different layers (EfficientNetB0 has 236 layers), but the top layer outputs 1000 classes because it was pretrained on ImageNet. To adjust this to your own problem, you might remove the original activation layer and replace it with your own but with the right number of output classes.
    • The important part here is that only the top few layers become trainable, the rest remain frozen. This way all the underlying patterns remain in the rest of the layers and you can utilise them for your own problem. This kind of transfer learning is very helpful when your data is similar to the data a model has been pretrained on.
  3. Fine-tuning transfer learning is when you take the underlying patterns (also called weights) of a pretrained model and adjust (fine-tune) them to your own problem.

    • This usually means training some, many or all of the layers in the pretrained model. This is useful when you've got a large dataset (e.g. 100+ images per class) where your data is slightly different to the data the original model was trained on.
    • A common workflow is to "freeze" all of the learned patterns in the bottom layers of a pretrained model so they're untrainable. And then train the top 2-3 layers of so the pretrained model can adjust its outputs to your custom data (feature extraction).
    • After you've trained the top 2-3 layers, you can then gradually "unfreeze" more and more layers and run the training process on your own data to further fine-tune the pretrained model.

___

7.0 Natural Language Processing's Techniques

(back to top)

About

Classification , Regression , Neural Network , CNN , Transfer Learning , NLP with Tensorflow

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published