A clean and scalable template to kickstart your AutoML project 🚀⚡🔥
Currently uses dev version of Hydra.
Suggestions are always welcome!
You can refer to Wiki for more details.
Install
- install via
pip
pip install hyperbox
- install via
github
git clone https://github.com/marsggbo/hyperbox
cd hyperbox
pip install -r requirements.txt
python setup.py develop
Quick Start
python -m hyperbox.run experiment=example_random_nas +trainer.fast_dev_run=True
Hyperbox Mutables (Searchable `nn.Module`)
- Code implementation for Figure (left)
import torch.nn as nn
from hyperbox . mutables . spaces import OperationSpace
op1 = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=3, stride =1, padding=1)
op2 = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=5, stride =1, padding=2)
op3 = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=7, stride =1, padding=3)
ops = OperationSpace(candidates=[op1, op2, op3], key=’conv_op’, mask=[1, 0, 0])Code implementation for Figure (middle)
import torch
from hyperbox.mutables.spaces import InputSpace
in1 = torch.rand(2, 64)
in2 = torch.rand(2, 32)
in3 = torch.rand(2, 16)
inputs = [in1, in2, in3]
skipconnect = InputSpace(n_candidates=3, n_chosen=1, key=’sc’, mask=[0, 1, 0])
out = skipconnect([ in1 , in2 , in3 ])
assert out is in2
>>> True- Code implementation for Figure (right)
from hyperbox.mutables.ops import Conv2d, Linear
from hyperbox.mutables.spaces import ValueSpace
# convolution
ks = ValueSpace([3, 5, 7], key=’kernelSize’, mask=[0, 1, 0])
cout = ValueSpace([16, 32, 64], key=’channelOut’, mask=[0, 1, 0])
conv = Conv2d(3 , cout , ks , stride =1, padding=2, bias=False )
print([x.shape for x in conv.parameters()])
>>> [torch.Size([32, 3, 5, 5])]
# linear
cout = ValueSpace([10, 100], key=’channelOut1’)
isBias = ValueSpace([0, 1], key=’bias’) # 0: False, 1: True
linear = Linear(10, cout , bias=isBias)
print([x.shape for x in linear.parameters()])
>>> [ torch . Size ([100 , 10]) , torch . Size ([100]) ]Hyperbox Mutator (Search Algorithms)
- Random Search Algorithm
RandomMutator
from hyperbox.mutator import RandomMutator
from hyperbox.networks.ofa import OFAMobileNetV3
net = OFAMobileNetV3()
rm = RandomMutator(net)
rm.reset() # search a subnet
arch: dict = rm._cache # arch of a subnet
print(arch)
subnet = OFAMobileNetV3(mask=arch) # initialize a subnet, which has smaller parameters than `net`the model arch is saved in rm._cache
