Skip to content

Commit

Permalink
Deploying to gh-pages from @ ad8b773 🚀
Browse files Browse the repository at this point in the history
  • Loading branch information
mlipasti committed Nov 7, 2023
1 parent 18cfc9d commit a8f2494
Show file tree
Hide file tree
Showing 8 changed files with 11 additions and 11 deletions.
2 changes: 1 addition & 1 deletion assets/tutorials/bitstream/code/output/ex10.res
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.0030000000000000027
0.007000000000000006
2 changes: 1 addition & 1 deletion assets/tutorials/bitstream/code/output/ex4.res
Original file line number Diff line number Diff line change
@@ -1 +1 @@
SBit(pos = true, neg = false)
SBit(pos = false, neg = false)
2 changes: 1 addition & 1 deletion assets/tutorials/bitstream/code/output/ex7.res
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.0033932135728543256
0.006586826347305397
2 changes: 1 addition & 1 deletion assets/tutorials/pruning/code/output/ex7.out
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Propagated MobileNet Mults 5093506 Adds 4984905
Propagated MobileNet Mults 4977772 Adds 4875654
2 changes: 1 addition & 1 deletion assets/tutorials/pruning/code/output/ex8.out
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Resized MobileNet Mults 3779404 Adds 3622566
Resized MobileNet Mults 3645231 Adds 3494586
2 changes: 1 addition & 1 deletion index.html
Original file line number Diff line number Diff line change
@@ -1 +1 @@
<!doctype html> <html lang=en > <meta charset=UTF-8 > <meta name=viewport content="width=device-width, initial-scale=1"> <link rel=stylesheet href="/bitstream-hackathon/css/jtd.css"> <link rel=icon href="/bitstream-hackathon/assets/favicon.ico"> <title>UW-Madison Bitstream Computing Hackathon</title> <div class=page-wrap > <div class=side-bar > <!-- <div class=header > <a href="/bitstream-hackathon/" class=title > <img src="/bitstream-hackathon/assets/pharm_homepage.jpg"> </a> </div> --> <label for=show-menu class=show-menu >MENU</label> <input type=checkbox id=show-menu role=button > <div class=menu id=side-menu > <ul class=menu-list > <li class="menu-list-item active"><a href="/bitstream-hackathon/" class="menu-list-link active">Home</a> <li class=menu-list-item >Tutorials <ul class=menu-list-child-list style="display: block;"> <li class="menu-list-item "><a href="/bitstream-hackathon/tutorials/overview/" class="menu-list-link ">Overview</a> <li class="menu-list-item "><a href="/bitstream-hackathon/tutorials/bitstream/" class="menu-list-link ">Bitstreams 101</a> <li class="menu-list-item "><a href="/bitstream-hackathon/tutorials/bitstreamlining/" class="menu-list-link ">Bitstreamlining</a> <li class="menu-list-item "><a href="/bitstream-hackathon/tutorials/pruning/" class="menu-list-link ">Pruning</a> <li class="menu-list-item "><a href="/bitstream-hackathon/tutorials/mobilenet/" class="menu-list-link ">Simulating MobileNet</a> <li class="menu-list-item "><a href="/bitstream-hackathon/tutorials/submission/" class="menu-list-link ">Submission guide</a> </ul> </ul> </div> <div class=footer > This is <em>Just the docs</em>, adapted from the <a href="https://github.com/pmarsceill/just-the-docs" target=_blank >Jekyll theme</a>. </div> </div> <div class=main-content-wrap > <div class=main-content > <div class=main-header > <span style="padding-right: 250px;"> <img src="/bitstream-hackathon/assets/red-flush-UWlogo.jpg" width=150px > <img src="/bitstream-hackathon/assets/pharm_homepage.jpg" width=175px style="padding-bottom: 10px;"> </span> <a id=github href="/bitstream-hackathon//github.com/UW-PHARM/BitSAD.jl">BitSAD.jl on GitHub</a> </div> <div class=franklin-content ><h1 id=bitstream_computing_hackathon_at_uw-madison ><a href="#bitstream_computing_hackathon_at_uw-madison" class=header-anchor >Bitstream Computing Hackathon at UW-Madison</a></h1> <div class=franklin-toc ><ol><li><a href="#welcome_to_bchuw">Welcome to <strong>BCH@UW&#33;</strong></a><li><a href="#participation_rules">Participation rules</a><li><a href="#information_and_suppport">Information and suppport</a></ol></div> <h2 id=welcome_to_bchuw ><a href="#welcome_to_bchuw" class=header-anchor >Welcome to <strong>BCH@UW&#33;</strong></a></h2> <p>Watch the <a href="https://youtu.be/ZT_1PJJm-O8">Introduction video</a> or browse the <a href="assets/hackathon_introduction.pptx">PPTX slides</a> at your convenience.</p> <p>This hackathon will give you a chance to learn about ultra low-power neural networks, how they are designed, how they are programmed or trained, and how they are used to process sensory data from the real world.</p> <p>When you participate, you will learn:</p> <ul> <li><p>a new programming language &#40;Julia&#41;</p> <li><p>about low-cost, ultra-low computing using Bitstreams as a data type</p> <li><p>how to train a powerful neural network called MobileNet</p> <li><p>how to prune/tune/quantize this network to make it energy efficient</p> </ul> <p>The <strong>BCH@UW</strong> Hackathon kicks off on <em>Sun 11/12/2023 at 1pm in EH2261</em>. In the meantime, you can work through the tutorials on this website to get started with the tools and optimization flows.</p> <h2 id=participation_rules ><a href="#participation_rules" class=header-anchor >Participation rules</a></h2> <p><strong>Goal:</strong> Prune a pre-trained MobileNetv1 model to optimize for energy efficiency without compromising the accuracy.</p> <p>Your entry will be evaluated on three categories:</p> <ul> <li><p><em>Accuracy:</em> how well can your model classify input images?</p> <li><p><em>Area:</em> how big is the hardware circuit implementation of your model?</p> <li><p><em>Energy:</em> how energy efficient is your hardware?</p> </ul> <p>Follow the instructions in the <a href="/bitstream-hackathon/tutorials/submission">submission guide</a> to evaluate your result.</p> <h2 id=information_and_suppport ><a href="#information_and_suppport" class=header-anchor >Information and suppport</a></h2> <p>Join the mailing list for updates/questions/support:</p> <ul> <li><p>Send a blank email to <a href="mailto:bitstream-hackathon&#43;[email protected]">bitstream-hackathon&#43;[email protected]</a></p> <li><p>Or navigate to <a href="https://groups.google.com/u/1/g/bitstream-hackathon">https://groups.google.com/u/1/g/bitstream-hackathon</a> and select &quot;Join&quot;</p> </ul> <p><strong>NOTE:</strong> You need a gmail/google account to join the mailing list. You can always create one with your existing email and delete it once you are done with it by following the instructions found here: <a href="https://support.google.com/accounts/answer/27441">https://support.google.com/accounts/answer/27441</a></p> <p><a href="assets/flyer.pdf">Hackathon flyer</a></p> <h1></h1> <p>🚧 Site under construction 🚧</p> <div class=page-foot > <hr> <a href="http://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a> UW-Madison PHARM Group. Last modified: November 07, 2023. Website built with <a href="https://github.com/tlienart/Franklin.jl">Franklin.jl</a> and the <a href="https://julialang.org">Julia programming language</a>. </div> </div> </div> </div> </div> <!-- end of class page-wrap-->
<!doctype html> <html lang=en > <meta charset=UTF-8 > <meta name=viewport content="width=device-width, initial-scale=1"> <link rel=stylesheet href="/bitstream-hackathon/css/jtd.css"> <link rel=icon href="/bitstream-hackathon/assets/favicon.ico"> <title>UW-Madison Bitstream Computing Hackathon</title> <div class=page-wrap > <div class=side-bar > <!-- <div class=header > <a href="/bitstream-hackathon/" class=title > <img src="/bitstream-hackathon/assets/pharm_homepage.jpg"> </a> </div> --> <label for=show-menu class=show-menu >MENU</label> <input type=checkbox id=show-menu role=button > <div class=menu id=side-menu > <ul class=menu-list > <li class="menu-list-item active"><a href="/bitstream-hackathon/" class="menu-list-link active">Home</a> <li class=menu-list-item >Tutorials <ul class=menu-list-child-list style="display: block;"> <li class="menu-list-item "><a href="/bitstream-hackathon/tutorials/overview/" class="menu-list-link ">Overview</a> <li class="menu-list-item "><a href="/bitstream-hackathon/tutorials/bitstream/" class="menu-list-link ">Bitstreams 101</a> <li class="menu-list-item "><a href="/bitstream-hackathon/tutorials/bitstreamlining/" class="menu-list-link ">Bitstreamlining</a> <li class="menu-list-item "><a href="/bitstream-hackathon/tutorials/pruning/" class="menu-list-link ">Pruning</a> <li class="menu-list-item "><a href="/bitstream-hackathon/tutorials/mobilenet/" class="menu-list-link ">Simulating MobileNet</a> <li class="menu-list-item "><a href="/bitstream-hackathon/tutorials/submission/" class="menu-list-link ">Submission guide</a> </ul> </ul> </div> <div class=footer > This is <em>Just the docs</em>, adapted from the <a href="https://github.com/pmarsceill/just-the-docs" target=_blank >Jekyll theme</a>. </div> </div> <div class=main-content-wrap > <div class=main-content > <div class=main-header > <span style="padding-right: 250px;"> <img src="/bitstream-hackathon/assets/red-flush-UWlogo.jpg" width=150px > <img src="/bitstream-hackathon/assets/pharm_homepage.jpg" width=175px style="padding-bottom: 10px;"> </span> <a id=github href="/bitstream-hackathon//github.com/UW-PHARM/BitSAD.jl">BitSAD.jl on GitHub</a> </div> <div class=franklin-content ><h1 id=bitstream_computing_hackathon_at_uw-madison ><a href="#bitstream_computing_hackathon_at_uw-madison" class=header-anchor >Bitstream Computing Hackathon at UW-Madison</a></h1> <div class=franklin-toc ><ol><li><a href="#welcome_to_bchuw">Welcome to <strong>BCH@UW&#33;</strong></a><li><a href="#participation_rules">Participation rules</a><li><a href="#information_and_suppport">Information and suppport</a></ol></div> <h2 id=welcome_to_bchuw ><a href="#welcome_to_bchuw" class=header-anchor >Welcome to <strong>BCH@UW&#33;</strong></a></h2> <p>Watch the <a href="https://youtu.be/ZT_1PJJm-O8">Introduction video</a> <a href="https://youtu.be/DQgAueqjj4I">improved audio</a>&#41; or browse the <a href="assets/hackathon_introduction.pptx">PPTX slides</a> at your convenience.</p> <p>This hackathon will give you a chance to learn about ultra low-power neural networks, how they are designed, how they are programmed or trained, and how they are used to process sensory data from the real world.</p> <p>When you participate, you will learn:</p> <ul> <li><p>a new programming language &#40;Julia&#41;</p> <li><p>about low-cost, ultra-low computing using Bitstreams as a data type</p> <li><p>how to train a powerful neural network called MobileNet</p> <li><p>how to prune/tune/quantize this network to make it energy efficient</p> </ul> <p>The <strong>BCH@UW</strong> Hackathon kicks off on <em>Sun 11/12/2023 at 1pm in EH2261</em>. In the meantime, you can work through the tutorials on this website to get started with the tools and optimization flows.</p> <h2 id=participation_rules ><a href="#participation_rules" class=header-anchor >Participation rules</a></h2> <p><strong>Goal:</strong> Prune a pre-trained MobileNetv1 model to optimize for energy efficiency without compromising the accuracy.</p> <p>Your entry will be evaluated on three categories:</p> <ul> <li><p><em>Accuracy:</em> how well can your model classify input images?</p> <li><p><em>Area:</em> how big is the hardware circuit implementation of your model?</p> <li><p><em>Energy:</em> how energy efficient is your hardware?</p> </ul> <p>Follow the instructions in the <a href="/bitstream-hackathon/tutorials/submission">submission guide</a> to evaluate your result.</p> <h2 id=information_and_suppport ><a href="#information_and_suppport" class=header-anchor >Information and suppport</a></h2> <p>Join the mailing list for updates/questions/support:</p> <ul> <li><p>Send a blank email to <a href="mailto:bitstream-hackathon&#43;[email protected]">bitstream-hackathon&#43;[email protected]</a></p> <li><p>Or navigate to <a href="https://groups.google.com/u/1/g/bitstream-hackathon">https://groups.google.com/u/1/g/bitstream-hackathon</a> and select &quot;Join&quot;</p> </ul> <p><strong>NOTE:</strong> You need a gmail/google account to join the mailing list. You can always create one with your existing email and delete it once you are done with it by following the instructions found here: <a href="https://support.google.com/accounts/answer/27441">https://support.google.com/accounts/answer/27441</a></p> <p><a href="assets/flyer.pdf">Hackathon flyer</a></p> <h1></h1> <p>🚧 Site under construction 🚧</p> <div class=page-foot > <hr> <a href="http://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a> UW-Madison PHARM Group. Last modified: November 07, 2023. Website built with <a href="https://github.com/tlienart/Franklin.jl">Franklin.jl</a> and the <a href="https://julialang.org">Julia programming language</a>. </div> </div> </div> </div> </div> <!-- end of class page-wrap-->
6 changes: 3 additions & 3 deletions tutorials/bitstream/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
x = SBitstream(<span class=hljs-number >0.3</span>)</code></pre><pre><code class="plaintext hljs">SBitstream{Float64}(value = 0.3)
with 0 bits.</code></pre> <p>Here, we created a <code>SBitstream</code> &#40;the type in BitSAD for stochastic bitstreams&#41; encoding the real value 0.3. <code>SBitstream</code> will keep track of the mean of the Bernoulli distribution, which we can recover with <code>float</code>.</p> <pre><code class="julia hljs">float(x)</code></pre><pre><code class="plaintext hljs">0.3</code></pre>
<p>You&#39;ll also notice that there were &quot;0 bits enqueue&quot; in <code>x</code>. This refers to the fact that the bitstream, <code>x</code>, is a sequence of samples. Currently, we have not drawn any samples from <code>x</code>. We can try that now:</p>
<pre><code class="julia hljs">xt = pop!(x)</code></pre><pre><code class="plaintext hljs">SBit(pos = true, neg = false)</code></pre>
<pre><code class="julia hljs">xt = pop!(x)</code></pre><pre><code class="plaintext hljs">SBit(pos = false, neg = false)</code></pre>
<p>Now, we have a single sample, <code>xt</code>, which is of type <code>SBit</code>. An <code>SBit</code> is a &quot;stochastic bit&quot; which is just a convenient alias for a <a href="https://docs.julialang.org/en/v1.6/manual/functions/#Named-Tuples"><code>NamedTuple</code></a> with two parts –- the positive part &#40;<code>pos</code>&#41; and the negative part &#40;<code>neg</code>&#41;.</p>
<blockquote>
<p>Wait, I thought stochastic bitstreams were a single bit sequence? <br /> –- You &#40;probably&#41;</p>
Expand All @@ -25,7 +25,7 @@
SBitstream{Float64}(value = 0.3)
with 1002 bits.</code></pre>
<p>Finally, we can see that the empirical average over the <code>SBit</code>s in queue matches the encoded value quite closely.</p>
<pre><code class="julia hljs">abs(estimate(x) - float(x))</code></pre><pre><code class="plaintext hljs">0.0033932135728543256</code></pre>
<pre><code class="julia hljs">abs(estimate(x) - float(x))</code></pre><pre><code class="plaintext hljs">0.006586826347305397</code></pre>
<h2 id=operations_on_sbitstreams ><a href="#operations_on_sbitstreams" class=header-anchor >Operations on <code>SBitstream</code>s</a></h2>
<p>So far, we have not computed any meaningful results with BitSAD. Let&#39;s go back to the multiplication example and try to multiply two <code>SBitstream</code>s.</p>
<pre><code class="julia hljs">y = SBitstream(<span class=hljs-number >0.5</span>)
Expand All @@ -51,7 +51,7 @@ <h2 id=operations_on_sbitstreams ><a href="#operations_on_sbitstreams" class=hea
push!(z, zbit)
<span class=hljs-keyword >end</span>

abs(estimate(z) - float(z))</code></pre><pre><code class="plaintext hljs">0.0030000000000000027</code></pre>
abs(estimate(z) - float(z))</code></pre><pre><code class="plaintext hljs">0.007000000000000006</code></pre>
<p>We used a helper function, <code>multiply_sbit</code> to multiply the positive and negative channel of each <code>SBit</code> separately. This resulted in a new <code>SBit</code>, <code>zbit</code>, which we pushed onto <code>z</code>. When we take the empirical average of all these <code>zbit</code>s, we see that it is close to the true mean of <code>z</code>.</p>
<p>Hopefully, you can now see why stochastic computing can be so resource efficient. Each channel of <code>multiply_sbit</code> only needed to multiply two 1-bit numbers. This can be done with a single AND gate.</p>

Expand Down
4 changes: 2 additions & 2 deletions tutorials/pruning/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,13 @@ <h3 id=propagating_the_pruning ><a href="#propagating_the_pruning" class=header-
<pre><code class="julia hljs">m_pruned = keepprune(m_ch_pruned)
m_prop = prune_propagate(m_pruned)
mults, adds, output_size = compute_dot_prods(m_prop, (<span class=hljs-number >96</span>, <span class=hljs-number >96</span>, <span class=hljs-number >3</span>, <span class=hljs-number >1</span>))
println(<span class=hljs-string >&quot;Propagated MobileNet Mults &quot;</span>, mults, <span class=hljs-string >&quot; Adds &quot;</span>, adds)</code></pre><pre><code class="plaintext hljs">Propagated MobileNet Mults 5093506 Adds 4984905
println(<span class=hljs-string >&quot;Propagated MobileNet Mults &quot;</span>, mults, <span class=hljs-string >&quot; Adds &quot;</span>, adds)</code></pre><pre><code class="plaintext hljs">Propagated MobileNet Mults 4977772 Adds 4875654
</code></pre>
<h3 id=resizing_the_propagated_model ><a href="#resizing_the_propagated_model" class=header-anchor >Resizing the propagated model</a></h3>
<p>If enough nodes get pruned out, there would be slices in the model which accomplish nothing, computationally. Instead of wasting resources on passing these kernels full of zeros around, they can be eliminated from the structure of our model.</p>
<pre><code class="julia hljs">m_resized = resize(m_prop)
mults, adds, output_size = compute_dot_prods(m_resized, (<span class=hljs-number >96</span>, <span class=hljs-number >96</span>, <span class=hljs-number >3</span>, <span class=hljs-number >1</span>))
println(<span class=hljs-string >&quot;Resized MobileNet Mults &quot;</span>, mults, <span class=hljs-string >&quot; Adds &quot;</span>, adds)</code></pre><pre><code class="plaintext hljs">Resized MobileNet Mults 3779404 Adds 3622566
println(<span class=hljs-string >&quot;Resized MobileNet Mults &quot;</span>, mults, <span class=hljs-string >&quot; Adds &quot;</span>, adds)</code></pre><pre><code class="plaintext hljs">Resized MobileNet Mults 3645231 Adds 3494586
</code></pre>
<h3 id=pruning_and_finetuning_pipeline ><a href="#pruning_and_finetuning_pipeline" class=header-anchor >Pruning and Finetuning pipeline</a></h3>
<p>Now that we seen how to prune our model, let&#39;s try to finetune it to recover some of the accuracy we lost. A basic template setup for training the model is provided by trainer function, and can be used as a starting point for your own training methodology.</p>
Expand Down

0 comments on commit a8f2494

Please sign in to comment.