Skip to content

Commit

Permalink
Updated Documentation for 1.1.0 Release
Browse files Browse the repository at this point in the history
  • Loading branch information
@coreylammie
coreylammie committed Feb 7, 2021
1 parent a0d82ce commit 5c20fa2
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 22 deletions.
39 changes: 18 additions & 21 deletions docs/Tutorial.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -97,13 +97,10 @@
"\n",
" return 100. * float(correct) / float(len(test_loader.dataset))\n",
"\n",
"device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
"device = torch.device('cpu' if 'cpu' in memtorch.__version__ else 'cuda')\n",
"epochs = 50\n",
"train_loader, validation_loader, test_loader = LoadCIFAR10(batch_size=256, validation=False)\n",
"model = Net().to(device)\n",
"if device == 'cuda':\n",
" model = torch.nn.DataParallel(model)\n",
"\n",
"criterion = nn.CrossEntropyLoss()\n",
"learning_rate = 1e-2\n",
"optimizer = optim.Adam(model.parameters(), lr=learning_rate)\n",
Expand Down Expand Up @@ -179,7 +176,7 @@
"source": [
"*memtorch.bh.map.Parameter.naive_map* is used to convert the weights within all *torch.nn.Linear* layers to equivalent conductance values, to be programmed to the two memristive devices used to represent each weight (positive and negative) using Eq. (13). \n",
"\n",
"*transistor* is *True*, so a 1T1R arrangement is simulated. *programming_routine* is set to *None* to skip device-level simulation of the programming routine. We note if *transistor* is *False* *programming_routine* must not be *None*. In which case, device-level simulation is performed for each device using *memtorch.bh.crossbar.gen_programming_signal* and *memtorch.bh.memristor.Memristor.simulate*, which uses finite differences to model internal device dynamics. As *scheme* is not defined, a double-column parameter representation scheme is adopted.\n",
"*tile_shape* is (128, 128), so modular crossbar tiles are used to represent weights. *ADC_resolution* sets the bit width of all emulated Analogue to Digital Converters (ADC); in this particular instance an 8-bit ADC resolution is used. *ADC_overflow* sets the initial overflow rate of each ADC. *quant_method* sets the quantization method used. *transistor* is *True*, so a 1T1R arrangement is simulated. *programming_routine* is set to *None* to skip device-level simulation of the programming routine. We note if *transistor* is *False* *programming_routine* must not be *None*. In which case, device-level simulation is performed for each device using *memtorch.bh.crossbar.gen_programming_signal* and *memtorch.bh.memristor.Memristor.simulate*, which uses finite differences to model internal device dynamics. As *scheme* is not defined, a double-column parameter representation scheme is adopted. Finally, *max_input_voltage* is 1.0, so inputs to each layer are encoded between -1.0V and +1.0V.\n",
"\n",
"All patched *torch.nn.Linear* layers are tuned using linear regression. A randomly generated tensor of size (8, *self.in_features*) is propagated through each memristive layer and each legacy layer (accessible using *layer.forward_legacy*). *sklearn.linear_model.LinearRegression* is used to determine the coefficient and intercept between the linear relationship of each set of outputs, which is used to define the *transform_output* lamdba function, that maps the output of each layer to their equivalent representations.\n"
]
Expand All @@ -200,18 +197,20 @@
"from memtorch.bh.crossbar.Program import naive_program\n",
"\n",
"\n",
"model = Net()\n",
"if torch.cuda.is_available():\n",
" model = torch.nn.DataParallel(model)\n",
"\n",
"model = Net().to(device)\n",
"model.load_state_dict(torch.load('trained_model.pt'), strict=False)\n",
"patched_model = patch_model(copy.deepcopy(model),\n",
" memristor_model=reference_memristor,\n",
" memristor_model_params=reference_memristor_params,\n",
" module_parameters_to_patch=[torch.nn.Linear],\n",
" mapping_routine=naive_map,\n",
" transistor=True,\n",
" programming_routine=None)\n"
" programming_routine=None,\n",
" tile_shape=(128, 128),\n",
" max_input_voltage=1.0,\n",
" ADC_resolution=8,\n",
" ADC_overflow_rate=0.,\n",
" quant_method='linear')"
]
},
{
Expand Down Expand Up @@ -257,7 +256,7 @@
"id": "R4H9f4d248V7"
},
"source": [
"We use a simple prototype model to demonstrate modeling non-ideal device characteristics."
"We use a simple prototype model to demonstrate modeling non-ideal device characteristics. For sake of simplicity, from here-on-in, modular crossbar tiles are not used, and quantization noise is ignored."
]
},
{
Expand Down Expand Up @@ -291,18 +290,16 @@
" return self.linear_layer(x)\n",
" \n",
"torch.manual_seed(0)\n",
"model = Model()\n",
"if torch.cuda.is_available():\n",
" model = torch.nn.DataParallel(model)\n",
"\n",
"model = Model().to(device)\n",
"reference_memristor_params = {'time_series_resolution': 1e-10, 'r_off': 200, 'r_on': 100}\n",
"patched_model = patch_model(copy.deepcopy(model),\n",
" memristor_model=reference_memristor,\n",
" memristor_model_params=reference_memristor_params,\n",
" module_parameters_to_patch=[torch.nn.Linear, torch.nn.Conv2d],\n",
" mapping_routine=naive_map,\n",
" transistor=True,\n",
" programming_routine=None)"
" programming_routine=None,\n",
" max_input_voltage=1.0)"
]
},
{
Expand Down Expand Up @@ -337,22 +334,22 @@
"from mpl_toolkits.axes_grid1 import make_axes_locatable\n",
"\n",
"reference_memristor_params = {'time_series_resolution': 1e-10, \n",
" 'r_off': memtorch.bh.StochasticParameter(200, std=20, min=2),\n",
" 'r_on': memtorch.bh.StochasticParameter(100, std=10, min=1)}\n",
" 'r_off': memtorch.bh.StochasticParameter(loc=200, scale=20, min=2),\n",
" 'r_on': memtorch.bh.StochasticParameter(loc=100, scale=10, min=1)}\n",
"\n",
"patched_model_ = patch_model(copy.deepcopy(model),\n",
" memristor_model=reference_memristor,\n",
" memristor_model_params=reference_memristor_params,\n",
" module_parameters_to_patch=[torch.nn.Linear, torch.nn.Conv2d],\n",
" mapping_routine=naive_map,\n",
" transistor=True,\n",
" programming_routine=None)\n",
" programming_routine=None,\n",
" max_input_voltage=1.0)\n",
"\n",
"A = torch.Tensor(np.vectorize(lambda x: x.r_off)(patched_model_.linear_layer.crossbars[0].devices))\n",
"B = torch.Tensor(np.vectorize(lambda x: x.r_on)(patched_model_.linear_layer.crossbars[0].devices))\n",
"C = torch.cat((A, B), 0)\n",
"\n",
"\n",
"plt.subplot(2, 1, 1)\n",
"plt.imshow(A.transpose(0, 1), interpolation='nearest', aspect=1, vmin=C.min(), vmax=C.max(), cmap='seismic')\n",
"plt.xticks([])\n",
Expand Down Expand Up @@ -554,7 +551,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.8"
"version": "3.8.5"
}
},
"nbformat": 4,
Expand Down
2 changes: 1 addition & 1 deletion docs/conf.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
author = 'Corey Lammie'

# The full version, including alpha/beta/rc tags
release = '1.0.9'
release = '1.1.0'
autodoc_inherit_docstrings = False

# -- General configuration ---------------------------------------------------
Expand Down

0 comments on commit 5c20fa2

Please sign in to comment.