Skip to content

Commit b880fbc

Browse files
mvaligurskyMartin Valigursky
andauthored
Manual page for Picker class (to be merged when engine 2.14 is released) (#850)
* Manual page for Picker class (to be merged when engine 2.14 is released) * lint --------- Co-authored-by: Martin Valigursky <[email protected]>
1 parent 5d8dd67 commit b880fbc

File tree

2 files changed

+96
-0
lines changed

2 files changed

+96
-0
lines changed
Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
---
2+
title: Scene Picker
3+
---
4+
5+
The Picker class provides a way to select mesh instances from screen coordinates by clicking or touching the screen. It works by rendering the scene from a camera's viewpoint to an offscreen buffer with unique ID colors, allowing efficient identification of clicked objects. The picker supports both regular meshes and Gaussian Splats, and works across WebGL2 and WebGPU backends.
6+
7+
## Basic Usage
8+
9+
To create a picker, instantiate it with your application, desired resolution, and optionally enable depth picking:
10+
11+
```javascript
12+
const picker = new pc.Picker(app, width, height, depth);
13+
```
14+
15+
The basic workflow involves three steps:
16+
17+
1. **Prepare** - Render the pick buffer by calling [`prepare(camera, scene, layers)`][1] once per frame (or only when the camera or scene changes)
18+
2. **Resize** - Adjust picker resolution if needed using [`resize(width, height)`][2]
19+
3. **Query** - Get picked mesh instances asynchronously using [`getSelectionAsync(x, y, width, height)`][3]
20+
21+
The picker uses an asynchronous API to read pixel data without blocking the rendering thread, ensuring smooth frame rates even when picking. When you're done with the picker, call [`destroy()`][5] to clean up GPU resources.
22+
23+
For complete API documentation, see the [Picker API reference][0].
24+
25+
## Depth Support
26+
27+
By default, the picker only captures mesh instance IDs. However, you can enable depth picking by passing `true` as the fourth constructor parameter:
28+
29+
```javascript
30+
const picker = new pc.Picker(app, width, height, true);
31+
```
32+
33+
When depth picking is enabled, the picker captures depth values along with mesh IDs. This additional information enables calculating the exact 3D world position of clicked points on object surfaces, which is useful for placing objects, measuring distances, or creating editor tools.
34+
35+
## World Position Picking
36+
37+
When depth picking is enabled, you can use [`getWorldPointAsync(x, y)`][4] to get the 3D world position at screen coordinates:
38+
39+
```javascript
40+
picker.getWorldPointAsync(x, y).then((worldPoint) => {
41+
if (worldPoint) {
42+
// worldPoint is a Vec3 in world space
43+
console.log('Clicked at:', worldPoint);
44+
} else {
45+
// No object was clicked (background)
46+
console.log('Clicked on empty space');
47+
}
48+
});
49+
```
50+
51+
The method returns a promise that resolves to a `Vec3` containing the world position, or `null` if no object was clicked. This works correctly with both perspective and orthographic cameras.
52+
53+
## Performance Considerations
54+
55+
The picker's performance can be optimized in several ways:
56+
57+
**Lower Resolution**: Rendering the pick buffer at a fraction of the screen resolution significantly improves performance. For example, using 0.25x screen resolution:
58+
59+
```javascript
60+
const pickerScale = 0.25;
61+
const picker = new pc.Picker(
62+
app,
63+
canvas.width * pickerScale,
64+
canvas.height * pickerScale,
65+
true
66+
);
67+
```
68+
69+
The trade-off is reduced precision - very small objects may be missed at lower resolutions.
70+
71+
**Asynchronous Reads**: The picker's async API prevents blocking the main thread while reading pixel data from the GPU, maintaining smooth frame rates.
72+
73+
**Selective Updates**: Call `prepare()` only when needed. If your camera and objects are static, you can reuse the previously rendered pick buffer without calling `prepare()` again.
74+
75+
## Gaussian Splatting Support
76+
77+
The picker fully supports Gaussian Splat instances with the same API as regular meshes. You can pick splat instances by their mesh instance ID and, with depth enabled, determine exact 3D positions on splat surfaces.
78+
79+
This enables interactive applications like placing markers on splats, measuring distances, or selecting individual splat entities in complex scenes. See the [Gaussian Splatting Picking example][6] for a complete demonstration.
80+
81+
## Examples
82+
83+
These engine examples demonstrate the picker in action:
84+
85+
- [**Area Picker**][7] - Shows how to pick mesh instances in rectangular screen regions with visual feedback
86+
- [**Gaussian Splatting Picking**][6] - Demonstrates picking splat instances and using world position picking to place markers on splat surfaces
87+
88+
[0]: https://api.playcanvas.com/engine/classes/Picker.html
89+
[1]: https://api.playcanvas.com/engine/classes/Picker.html#prepare
90+
[2]: https://api.playcanvas.com/engine/classes/Picker.html#resize
91+
[3]: https://api.playcanvas.com/engine/classes/Picker.html#getSelectionAsync
92+
[4]: https://api.playcanvas.com/engine/classes/Picker.html#getWorldPointAsync
93+
[5]: https://api.playcanvas.com/engine/classes/Picker.html#destroy
94+
[6]: https://playcanvas.github.io/#gaussian-splatting/picking
95+
[7]: https://playcanvas.github.io/#/graphics/area-picker

sidebars.js

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -604,6 +604,7 @@ const sidebars = {
604604
},
605605
items: [
606606
'user-manual/graphics/cameras/depth-layer',
607+
'user-manual/graphics/cameras/scene-picker',
607608
],
608609
},
609610
{

0 commit comments

Comments
 (0)