Skip to content

numpy implementation of net 2 net from the paper Net2Net: Accelerating Learning via Knowledge Transfer http://arxiv.org/abs/1511.05641

License

Notifications You must be signed in to change notification settings

DanielSlater/Net2Net

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Net2Net

numpy implementation of net 2 net from the paper Net2Net: Accelerating Learning via Knowledge Transfer http://arxiv.org/abs/1511.05641

Requirements

  • numpy

Usage

Here is how you would use it to create a wider version of an existing layer

import numpy as np

weights = np.matrix([[1.0, 0.1, 0.5], [1.0, 0.1, 0.5]])
bias = np.array([0.0, 0.0, 0.0])
weights_next_layer = np.matrix([[1.0], [0.2], [0.5]])

weights, bias, weights_next_layer = net_2_wider_net(weights, bias,
                                                  weights_next_layer,
                                                  new_layer_size=5)

Then simply use the new variables from then on.

Here is creating the weights and biases for a new layer using net 2 deeper net

import numpy as np

bias = np.array([0.0, 0.0, 0.0])

next_layer_weights, next_layer_bias = net_2_deeper_net(bias)

There are complete examples of using this for grid searching the number of hidden nodes in examples/tensorflow_grid_search.py

About

numpy implementation of net 2 net from the paper Net2Net: Accelerating Learning via Knowledge Transfer http://arxiv.org/abs/1511.05641

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages