Skip to content

Commit

Permalink
update
Browse files Browse the repository at this point in the history
  • Loading branch information
@shi-yan
shi-yan committed Aug 2, 2024
1 parent 677b32b commit c9978d0
Show file tree
Hide file tree
Showing 8 changed files with 93 additions and 55 deletions.
8 changes: 4 additions & 4 deletions 2D_Techniques/billboarding.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -143,12 +143,12 @@

<div id="article-container">
<article>
<h2 >2.5 Billboarding</h2><p>We will look at a technique called billboarding this tutorial. billboarding is a technique of rendering, such that a flat object always faces the viewer no matter viewed from which direction. it has application in games to show meta information, such as the health bar, or in visualizations to show labels. or in games to show grass and vegetation.</p><a href="https://shi-yan.github.io/WebGPUUnleashed/code/code.html#2_05_billboard" target="_blank" class="comment"><svg style="margin-right:10px;vertical-align: middle;" xmlns="http://www.w3.org/2000/svg" height="32"
width="32" fill="#dadadb" viewBox="0 -960 960 960"><path d="M189-160q-60 0-102.5-43T42-307q0-9 1-18t3-18l84-336q14-54 57-87.5t98-33.5h390q55 0 98 33.5t57 87.5l84 336q2 9 3.5 18.5T919-306q0 61-43.5 103.5T771-160q-42 0-78-22t-54-60l-28-58q-5-10-15-15t-21-5H385q-11 0-21 5t-15 15l-28 58q-18 38-54 60t-78 22Zm3-80q19 0 34.5-10t23.5-27l28-57q15-31 44-48.5t63-17.5h190q34 0 63 18t45 48l28 57q8 17 23.5 27t34.5 10q28 0 48-18.5t21-46.5q0 1-2-19l-84-335q-7-27-28-44t-49-17H285q-28 0-49.5 17T208-659l-84 335q-2 6-2 18 0 28 20.5 47t49.5 19Zm348-280q17 0 28.5-11.5T580-560q0-17-11.5-28.5T540-600q-17 0-28.5 11.5T500-560q0 17 11.5 28.5T540-520Zm80-80q17 0 28.5-11.5T660-640q0-17-11.5-28.5T620-680q-17 0-28.5 11.5T580-640q0 17 11.5 28.5T620-600Zm0 160q17 0 28.5-11.5T660-480q0-17-11.5-28.5T620-520q-17 0-28.5 11.5T580-480q0 17 11.5 28.5T620-440Zm80-80q17 0 28.5-11.5T740-560q0-17-11.5-28.5T700-600q-17 0-28.5 11.5T660-560q0 17 11.5 28.5T700-520Zm-360 60q13 0 21.5-8.5T370-490v-40h40q13 0 21.5-8.5T440-560q0-13-8.5-21.5T410-590h-40v-40q0-13-8.5-21.5T340-660q-13 0-21.5 8.5T310-630v40h-40q-13 0-21.5 8.5T240-560q0 13 8.5 21.5T270-530h40v40q0 13 8.5 21.5T340-460Zm140-20Z"/></svg>Launch Playground - 2_05_billboard</a><p>in the first two use cases, billboard is used to show 2D meta information, we want the information they show be always visible regardless of scene rotation. hence we want them to always face the viewer. in the last use case, rendering geometry accurate grass and vegetation is almost impossible due to complexity, hence we want to fake it by rendering screen facing 2D grass images. but as we rotate the scene, we don't want the viewer to discover that the grass is actually a 2D plane, hence we need to adjust the 2D plane's rotation to let it face the view at all time.</p><p>in this tutorial, we will render a rotating scene by rotating the camera. with a texture rendered in the background that doesn't rotate with the scene. The background is a billboard. The key aspect of this tutorial is to learn how to keep the background face the camera. This is an opportunity to test our understanding of the transformation and projection matrices.</p><p>the key idea of this demo is simple. we render everything else other than the billboard using a normal model view matrix, and then we modify the model view matrix, keep the translation part, and remove the rotation and scaling part.</p><p>the key line of the code is the following, let's explain.</p><div class="code-fragments"><pre><code class="language-javascript code-block" startNumber=509>let modelViewMatrixBillboard = glMatrix.mat4.fromValues(1, 0, 0, modelViewMatrix[3],
<h2 >2.5 Billboarding</h2><p>In this tutorial, we'll explore billboarding, a powerful rendering technique that ensures flat objects always face the viewer, regardless of the viewing angle. This method has diverse applications, from displaying meta-information in games (such as health bars) and visualization labels to simulating complex vegetation in 3D environments.</p><a href="https://shi-yan.github.io/WebGPUUnleashed/code/code.html#2_05_billboard" target="_blank" class="comment"><svg style="margin-right:10px;vertical-align: middle;" xmlns="http://www.w3.org/2000/svg" height="32"
width="32" fill="#dadadb" viewBox="0 -960 960 960"><path d="M189-160q-60 0-102.5-43T42-307q0-9 1-18t3-18l84-336q14-54 57-87.5t98-33.5h390q55 0 98 33.5t57 87.5l84 336q2 9 3.5 18.5T919-306q0 61-43.5 103.5T771-160q-42 0-78-22t-54-60l-28-58q-5-10-15-15t-21-5H385q-11 0-21 5t-15 15l-28 58q-18 38-54 60t-78 22Zm3-80q19 0 34.5-10t23.5-27l28-57q15-31 44-48.5t63-17.5h190q34 0 63 18t45 48l28 57q8 17 23.5 27t34.5 10q28 0 48-18.5t21-46.5q0 1-2-19l-84-335q-7-27-28-44t-49-17H285q-28 0-49.5 17T208-659l-84 335q-2 6-2 18 0 28 20.5 47t49.5 19Zm348-280q17 0 28.5-11.5T580-560q0-17-11.5-28.5T540-600q-17 0-28.5 11.5T500-560q0 17 11.5 28.5T540-520Zm80-80q17 0 28.5-11.5T660-640q0-17-11.5-28.5T620-680q-17 0-28.5 11.5T580-640q0 17 11.5 28.5T620-600Zm0 160q17 0 28.5-11.5T660-480q0-17-11.5-28.5T620-520q-17 0-28.5 11.5T580-480q0 17 11.5 28.5T620-440Zm80-80q17 0 28.5-11.5T740-560q0-17-11.5-28.5T700-600q-17 0-28.5 11.5T660-560q0 17 11.5 28.5T700-520Zm-360 60q13 0 21.5-8.5T370-490v-40h40q13 0 21.5-8.5T440-560q0-13-8.5-21.5T410-590h-40v-40q0-13-8.5-21.5T340-660q-13 0-21.5 8.5T310-630v40h-40q-13 0-21.5 8.5T240-560q0 13 8.5 21.5T270-530h40v40q0 13 8.5 21.5T340-460Zm140-20Z"/></svg>Launch Playground - 2_05_billboard</a><p>In the first two use cases, billboarding is employed to display 2D meta-information that should remain visible regardless of scene rotation. For the vegetation scenario, rendering geometrically accurate grass and foliage is often computationally prohibitive due to its complexity. Instead, we can create the illusion of 3D vegetation by rendering screen-facing 2D grass images. As the scene rotates, we dynamically adjust the 2D plane's orientation to maintain its face-forward appearance, preventing the viewer from perceiving its true flat nature.</p><p>Our tutorial will showcase a rotating scene achieved by rotating the camera, with a non-rotating, billboarded texture in the background. The primary focus is on learning how to keep the background consistently facing the camera, providing an excellent opportunity to deepen our understanding of transformation and projection matrices.</p><p>The core concept of this demonstration is straightforward: we render all non-billboard elements using a standard model view matrix. For the billboard, we modify the model view matrix, preserving its translation component while eliminating rotation and scaling.</p><p>The key line of code that achieves this effect is as follows. Let's break it down:</p><div class="code-fragments"><pre><code class="language-javascript code-block" startNumber=509>let modelViewMatrixBillboard = glMatrix.mat4.fromValues(1, 0, 0, modelViewMatrix[3],
0, 1, 0, modelViewMatrix[7],
0, 0, 1, modelViewMatrix[11],
modelViewMatrix[12], modelViewMatrix[13], modelViewMatrix[14], modelViewMatrix[15]);
</pre></code><div class="code-fragments-caption"><a target="_blank" href="https://shi-yan.github.io/WebGPUUnleashed/code/code.html?highlight=508:511#2_05_billboard">2_05_billboard/index.html:509-512 Modify the Transportation Matrix for the Billboard by Removing the Rotation and Scaling Components</a></div></div><p>recall that the upper left 3x3 sub matrix of a model view matrix defines the rotation and scaling. by setting it to an identity matrix, we essentially removed any rotation. but we want to keep the last column of the model view matrix, because the last column defines the translation.</p><p>when the rotation is removed, we want the billboard to always face the camera. As we know that anything in the world coordinate system applied with the model view matrix will become in the camera coordinate system. in the camera coordinate system, the origin is at the camera position, and the direction that the camera is point at is the -Z axis. hence to make the billboard face the camera, we need to define a plane facing the +Z direction.</p><p><div class="img-container"><img class="img" onclick="openImage(this)" src="thumb_viewcoordinatesrighthanded.png" original_src="viewcoordinatesrighthanded.png" alt="Right-Hand Camera Coordinates" sources='[]' /><div class="img-title">Right-Hand Camera Coordinates</div></div></p><p>hence here is how do we define the vertices of the billboard:</p><div class="code-fragments"><pre><code class="language-javascript code-block" startNumber=170>const positions = new Float32Array([
</pre></code><div class="code-fragments-caption"><a target="_blank" href="https://shi-yan.github.io/WebGPUUnleashed/code/code.html?highlight=508:511#2_05_billboard">2_05_billboard/index.html:509-512 Modify the Transportation Matrix for the Billboard by Removing the Rotation and Scaling Components</a></div></div><p>Recall that the upper-left 3x3 sub-matrix of a modelview matrix defines rotation and scaling. By setting this to an identity matrix, we effectively eliminate any rotation and scaling. However, we retain the last column of the model view matrix, as it defines the translation.</p><p>With rotation removed, our goal is to ensure the billboard consistently faces the camera. We know that applying the model view matrix to anything in the world coordinate system transforms it into the camera coordinate system. In this system, the origin is at the camera position, and the camera's viewing direction aligns with the -Z axis. Therefore, to make the billboard face the camera, we must define a plane facing the +Z direction.</p><p><div class="img-container"><img class="img" onclick="openImage(this)" src="thumb_viewcoordinatesrighthanded.png" original_src="viewcoordinatesrighthanded.png" alt="Right-Hand Camera Coordinates" sources='[]' /><div class="img-title">Right-Hand Camera Coordinates</div></div></p><p>Now, let's examine how we define the vertices of the billboard:</p><div class="code-fragments"><pre><code class="language-javascript code-block" startNumber=170>const positions = new Float32Array([
10.0, -10.0, 0.0,
-10.0, -10.0, 0.0,
10.0, 10.0, 0.0,
Expand Down Expand Up @@ -188,7 +188,7 @@ <h2 >2.5 Billboarding</h2><p>We will look at a technique called billboarding thi
format: 'depth24plus-stencil8'
}
};
</pre></code><div class="code-fragments-caption"><a target="_blank" href="https://shi-yan.github.io/WebGPUUnleashed/code/code.html?highlight=169:185,217:239#2_05_billboard">2_05_billboard/index.html:170-240 Setting the Billboard Geometry</a></div></div><p>We are using a triangle strip with clockwise order as the front (facing the +Z axis).</p>
</pre></code><div class="code-fragments-caption"><a target="_blank" href="https://shi-yan.github.io/WebGPUUnleashed/code/code.html?highlight=169:185,217:239#2_05_billboard">2_05_billboard/index.html:170-240 Setting the Billboard Geometry</a></div></div><p>We utilize a triangle strip with clockwise order as the front face (oriented towards the +Z axis). This configuration ensures that the billboard will always present its front face to the camera, maintaining visibility regardless of the viewing angle.</p>
</article>

<div class="older_newer_link_section">
Expand Down
6 changes: 3 additions & 3 deletions 2D_Techniques/implementing_fake_3d.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -143,15 +143,15 @@

<div id="article-container">
<article>
<h2 >2.6 Implementing Fake 3D</h2><p>in this example<a class="footnote-ref" href="#footnote_1">[1]</a>, we will look at a fake 3d effect. for a real 3d scene, if we move our camera position in the screen plane, we will notice the 3d effect called parallax, because objects further away move slower than objects nearby.</p><p><div class="img-container"><img class="img" onclick="openImage(this)" src="parallax.gif" alt="Train Window Presents a Real Life Parallax Effect" sources='["https://www.artstation.com/artwork/Z5vndm"]' /><div class="img-title">Train Window Presents a Real Life Parallax Effect<a class="img-source" target="_blank" href="https://www.artstation.com/artwork/Z5vndm">[SOURCE]</a></div></div></p><p>given a 2d picture, if we have a pairing depth map, we can fake this parallax effect to produce a 3D effect. the idea is simple, given a movement offset, we offset the texture coordinates. The amount of the offset is determined by the movement as well as the depth map, such that if a pixel is far away, we do not offset the texture coordinate as much as that of a nearby pixel.</p><p>To generate a depth map for your image, we can use the <a class="link" href="pytorch.org/hub/intelisl_midas_v2" target="_blank">MiDaS model</a>. please refer to the link for the details about how to create a depth map. In this example, we have a sample image with a paired depth image.</p><p><div class="img-container"><img class="img" onclick="openImage(this)" src="thumb_portraitanddepth.png" original_src="portraitanddepth.png" alt="The Picture Used in This Sample and Its Generated Depth" sources='[]' /><div class="img-title">The Picture Used in This Sample and Its Generated Depth</div></div></p><p>to make our demo more interesting, we want to use mouse to control the movement vector. The vector is calculated by subtract the current mouse position and the canvas center, such that the further our mouse is alway from the center of the image, we will offset the image more in the opposite direction.</p><div class="code-fragments"><pre><code class="language-javascript code-block" startNumber=242>canvas.addEventListener('mousemove', function (event) {
<h2 >2.6 Implementing Fake 3D</h2><p>In this example<a class="footnote-ref" href="#footnote_1">[1]</a>, we'll explore a technique for creating a convincing fake 3D effect. In a genuine 3D scene, moving the camera position within the screen plane produces a phenomenon known as parallax. This effect occurs because distant objects appear to move more slowly than those in the foreground.</p><p><div class="img-container"><img class="img" onclick="openImage(this)" src="parallax.gif" alt="Train Window Presents a Real Life Parallax Effect" sources='["https://www.artstation.com/artwork/Z5vndm"]' /><div class="img-title">Train Window Presents a Real Life Parallax Effect<a class="img-source" target="_blank" href="https://www.artstation.com/artwork/Z5vndm">[SOURCE]</a></div></div></p><p>We can simulate this parallax effect on a 2D image by utilizing a corresponding depth map, resulting in a compelling 3D illusion. The concept is straightforward: we adjust the texture coordinates based on a movement offset. The magnitude of this adjustment is determined by both the movement and the depth map, ensuring that pixels representing distant objects are offset less than those representing nearby objects.</p><p>To generate a depth map for your image, consider using the <a class="link" href="pytorch.org/hub/intelisl_midas_v2" target="_blank">MiDaS model</a>. For detailed instructions on creating a depth map, please consult the provided link. In this example, we'll work with a sample image and its paired depth map.</p><p><div class="img-container"><img class="img" onclick="openImage(this)" src="thumb_portraitanddepth.png" original_src="portraitanddepth.png" alt="The Picture Used in This Sample and Its Generated Depth" sources='[]' /><div class="img-title">The Picture Used in This Sample and Its Generated Depth</div></div></p><p>To enhance our demonstration, we'll implement mouse control for the movement vector. This vector is calculated by subtracting the current mouse position from the canvas center. As a result, the further the mouse moves from the image's center, the more the image will be offset in the opposite direction, creating a dynamic and interactive experience.</p><div class="code-fragments"><pre><code class="language-javascript code-block" startNumber=242>canvas.addEventListener('mousemove', function (event) {
let mousePos = getMousePos(canvas, event);
mousePos = { x: mousePos.x * 2.0 / canvas.width - 1.0, y: mousePos.y * 2.0 / canvas.height - 1.0 };
//console.log('mousemove', mousePos);
requestAnimationFrame(() =&gt; {
render([mousePos.x * 0.01, mousePos.y* 0.01]);
});
});
</pre></code><div class="code-fragments-caption"><a target="_blank" href="https://shi-yan.github.io/WebGPUUnleashed/code/code.html?highlight=241:248#2_06_fake_3d">2_06_fake_3d/index.html:242-249 Getting the Offset Vector From Mouse</a></div></div><p>the magic happens in the fragment shader:</p><div class="code-fragments"><pre><code class="language-javascript code-block" startNumber=31>@group(0) @binding(0)
</pre></code><div class="code-fragments-caption"><a target="_blank" href="https://shi-yan.github.io/WebGPUUnleashed/code/code.html?highlight=241:248#2_06_fake_3d">2_06_fake_3d/index.html:242-249 Getting the Offset Vector From Mouse</a></div></div><p>The magic happens in the fragment shader:</p><div class="code-fragments"><pre><code class="language-javascript code-block" startNumber=31>@group(0) @binding(0)
var t_diffuse: texture_2d&lt;f32&gt;;
@group(0) @binding(1)
var t_depth: texture_2d&lt;f32&gt;;
Expand All @@ -167,7 +167,7 @@ <h2 >2.6 Implementing Fake 3D</h2><p>in this example<a class="footnote-ref" href

return color;
}
</pre></code><div class="code-fragments-caption"><a target="_blank" href="https://shi-yan.github.io/WebGPUUnleashed/code/code.html?highlight=30:45#2_06_fake_3d">2_06_fake_3d/index.html:31-46 Offset the Texture Coordinates Based on Depth and the Offset Vector</a></div></div><p>in the shader, we do two texture sampling. the first time is against the depth map to obtain the depth. The second time is from the actual image, but we will adjust the texture coordinates. the adjustment is the mouse offset scaled by the depth.</p><p>This is a very simple program, but the end result is very interesting. Check it out for your self, try to replace the image with your own.</p><a href="https://shi-yan.github.io/WebGPUUnleashed/code/code.html#2_06_fake_3d" target="_blank" class="comment"><svg style="margin-right:10px;vertical-align: middle;" xmlns="http://www.w3.org/2000/svg" height="32"
</pre></code><div class="code-fragments-caption"><a target="_blank" href="https://shi-yan.github.io/WebGPUUnleashed/code/code.html?highlight=30:45#2_06_fake_3d">2_06_fake_3d/index.html:31-46 Offset the Texture Coordinates Based on Depth and the Offset Vector</a></div></div><p>In the shader, we perform two texture sampling operations. The first samples the depth map to obtain the depth value for each pixel. The second samples the actual image, but with adjusted texture coordinates. This adjustment is calculated by scaling the mouse offset by the depth value, creating the illusion of depth and movement.</p><p>Despite its simplicity, this program yields remarkably engaging results. We encourage you to experiment with the demonstration and try substituting the provided image with your own to see how the effect translates to different scenes and subjects.</p><a href="https://shi-yan.github.io/WebGPUUnleashed/code/code.html#2_06_fake_3d" target="_blank" class="comment"><svg style="margin-right:10px;vertical-align: middle;" xmlns="http://www.w3.org/2000/svg" height="32"
width="32" fill="#dadadb" viewBox="0 -960 960 960"><path d="M189-160q-60 0-102.5-43T42-307q0-9 1-18t3-18l84-336q14-54 57-87.5t98-33.5h390q55 0 98 33.5t57 87.5l84 336q2 9 3.5 18.5T919-306q0 61-43.5 103.5T771-160q-42 0-78-22t-54-60l-28-58q-5-10-15-15t-21-5H385q-11 0-21 5t-15 15l-28 58q-18 38-54 60t-78 22Zm3-80q19 0 34.5-10t23.5-27l28-57q15-31 44-48.5t63-17.5h190q34 0 63 18t45 48l28 57q8 17 23.5 27t34.5 10q28 0 48-18.5t21-46.5q0 1-2-19l-84-335q-7-27-28-44t-49-17H285q-28 0-49.5 17T208-659l-84 335q-2 6-2 18 0 28 20.5 47t49.5 19Zm348-280q17 0 28.5-11.5T580-560q0-17-11.5-28.5T540-600q-17 0-28.5 11.5T500-560q0 17 11.5 28.5T540-520Zm80-80q17 0 28.5-11.5T660-640q0-17-11.5-28.5T620-680q-17 0-28.5 11.5T580-640q0 17 11.5 28.5T620-600Zm0 160q17 0 28.5-11.5T660-480q0-17-11.5-28.5T620-520q-17 0-28.5 11.5T580-480q0 17 11.5 28.5T620-440Zm80-80q17 0 28.5-11.5T740-560q0-17-11.5-28.5T700-600q-17 0-28.5 11.5T660-560q0 17 11.5 28.5T700-520Zm-360 60q13 0 21.5-8.5T370-490v-40h40q13 0 21.5-8.5T440-560q0-13-8.5-21.5T410-590h-40v-40q0-13-8.5-21.5T340-660q-13 0-21.5 8.5T310-630v40h-40q-13 0-21.5 8.5T240-560q0 13 8.5 21.5T270-530h40v40q0 13 8.5 21.5T340-460Zm140-20Z"/></svg>Launch Playground - 2_06_fake_3d</a><table class="footnote-def"><tr class="footnote-row" id="footnote_1"><td>[1]: </td><td><p>This tutorial is inspired by this <a class="link" href="https://tympanus.net/codrops/2019/02/20/how-to-create-a-fake-3d-image-effect-with-webgl/" target="_blank">WebGL tutorial</a>.</p></td></tr></table>
</article>

Expand Down
Loading

0 comments on commit c9978d0

Please sign in to comment.