You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/docs/02-benchmarks/inference-time.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -117,7 +117,7 @@ Benchmark times for text embeddings are highly dependent on the sentence length.
117
117
Image embedding benchmark times are measured using 224×224 pixel images, as required by the model. All input images, whether larger or smaller, are resized to 224×224 before processing. Resizing is typically fast for small images but may be noticeably slower for very large images, which can increase total inference time.
118
118
:::
119
119
120
-
## Image Segmentation
120
+
## Semantic Segmentation
121
121
122
122
:::warning
123
123
Times presented in the tables are measured as consecutive runs of the model. Initial run times may be up to 2x longer due to model loading and initialization.
Data presented in the following sections is based on inference with non-resized output. When resize is enabled, expect higher memory usage and inference time with higher resolutions.
Copy file name to clipboardExpand all lines: docs/docs/03-hooks/02-computer-vision/useSemanticSegmentation.md
+19-19Lines changed: 19 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,27 +1,27 @@
1
1
---
2
-
title: useImageSegmentation
2
+
title: useSemanticSegmentation
3
3
---
4
4
5
-
Semantic image segmentation, akin to image classification, tries to assign the content of the image to one of the predefined classes. However, in case of segmentation this classification is done on a per-pixel basis, so as the result the model provides an image-sized array of scores for each of the classes. You can then use this information to detect objects on a per-pixel basis. React Native ExecuTorch offers a dedicated hook `useImageSegmentation` for this task.
5
+
Semantic semantic segmentation, akin to image classification, tries to assign the content of the image to one of the predefined classes. However, in case of segmentation this classification is done on a per-pixel basis, so as the result the model provides an image-sized array of scores for each of the classes. You can then use this information to detect objects on a per-pixel basis. React Native ExecuTorch offers a dedicated hook `useSemanticSegmentation` for this task.
6
6
7
7
:::warning
8
-
It is recommended to use models provided by us which are available at our [Hugging Face repository](https://huggingface.co/collections/software-mansion/image-segmentation-68d5291bdf4a30bee0220f4f), you can also use [constants](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts) shipped with our library.
8
+
It is recommended to use models provided by us which are available at our [Hugging Face repository](https://huggingface.co/collections/software-mansion/semantic-segmentation-68d5291bdf4a30bee0220f4f), you can also use [constants](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts) shipped with our library.
9
9
:::
10
10
11
11
## API Reference
12
12
13
-
- For detailed API Reference for `useImageSegmentation` see: [`useImageSegmentation` API Reference](../../06-api-reference/functions/useImageSegmentation.md).
14
-
- For all image segmentation models available out-of-the-box in React Native ExecuTorch see: [Image Segmentation Models](../../06-api-reference/index.md#models---image-segmentation).
13
+
- For detailed API Reference for `useSemanticSegmentation` see: [`useSemanticSegmentation` API Reference](../../06-api-reference/functions/useSemanticSegmentation.md).
14
+
- For all semantic segmentation models available out-of-the-box in React Native ExecuTorch see: [Semantic Segmentation Models](../../06-api-reference/index.md#models---semantic-segmentation).
15
15
16
16
## High Level Overview
17
17
18
18
```typescript
19
19
import {
20
-
useImageSegmentation,
20
+
useSemanticSegmentation,
21
21
DEEPLAB_V3_RESNET50,
22
22
} from'react-native-executorch';
23
23
24
-
const model =useImageSegmentation({
24
+
const model =useSemanticSegmentation({
25
25
model: DEEPLAB_V3_RESNET50,
26
26
});
27
27
@@ -37,24 +37,24 @@ try {
37
37
38
38
### Arguments
39
39
40
-
`useImageSegmentation` takes [`ImageSegmentationProps`](../../06-api-reference/interfaces/ImageSegmentationProps.md) that consists of:
40
+
`useSemanticSegmentation` takes [`SemanticSegmentationProps`](../../06-api-reference/interfaces/SemanticSegmentationProps.md) that consists of:
41
41
42
42
-`model` - An object containing:
43
-
-`modelName` - The name of a built-in model. See [`ModelSources`](../../06-api-reference/type-aliases/ModelSources.md) for the list of supported models.
43
+
-`modelName` - The name of a built-in model. See [`SemanticSegmentationModelSources`](../../06-api-reference/type-aliases/SemanticSegmentationModelSources.md) for the list of supported models.
44
44
-`modelSource` - The location of the model binary (a URL or a bundled resource).
45
-
- An optional flag [`preventLoad`](../../06-api-reference/interfaces/ImageSegmentationProps.md#preventload) which prevents auto-loading of the model.
45
+
- An optional flag [`preventLoad`](../../06-api-reference/interfaces/SemanticSegmentationProps.md#preventload) which prevents auto-loading of the model.
46
46
47
47
The hook is generic over the model config — TypeScript automatically infers the correct label type based on the `modelName` you provide. No explicit generic parameter is needed.
48
48
49
49
You need more details? Check the following resources:
50
50
51
-
- For detailed information about `useImageSegmentation` arguments check this section: [`useImageSegmentation` arguments](../../06-api-reference/functions/useImageSegmentation.md#parameters).
52
-
- For all image segmentation models available out-of-the-box in React Native ExecuTorch see: [Image Segmentation Models](../../06-api-reference/index.md#models---image-segmentation).
51
+
- For detailed information about `useSemanticSegmentation` arguments check this section: [`useSemanticSegmentation` arguments](../../06-api-reference/functions/useSemanticSegmentation.md#parameters).
52
+
- For all semantic segmentation models available out-of-the-box in React Native ExecuTorch see: [Semantic Segmentation Models](../../06-api-reference/index.md#models---semantic-segmentation).
53
53
- For more information on loading resources, take a look at [loading models](../../01-fundamentals/02-loading-models.md) page.
54
54
55
55
### Returns
56
56
57
-
`useImageSegmentation` returns an [`ImageSegmentationType`](../../06-api-reference/interfaces/ImageSegmentationType.md) object containing:
57
+
`useSemanticSegmentation` returns an [`SemanticSegmentationType`](../../06-api-reference/interfaces/SemanticSegmentationType.md) object containing:
58
58
59
59
-`isReady` - Whether the model is loaded and ready to process images.
60
60
-`isGenerating` - Whether the model is currently processing an image.
@@ -64,11 +64,11 @@ You need more details? Check the following resources:
64
64
65
65
## Running the model
66
66
67
-
To run the model, use the [`forward`](../../06-api-reference/interfaces/ImageSegmentationType.md#forward) method. It accepts three arguments:
67
+
To run the model, use the [`forward`](../../06-api-reference/interfaces/SemanticSegmentationType.md#forward) method. It accepts three arguments:
68
68
69
-
-[`imageSource`](../../06-api-reference/interfaces/ImageSegmentationType.md#forward) (required) - The image to segment. Can be a remote URL, a local file URI, or a base64-encoded image (whole URI or only raw base64).
70
-
-[`classesOfInterest`](../../06-api-reference/interfaces/ImageSegmentationType.md#forward) (optional) - An array of label keys indicating which per-class probability masks to include in the output. Defaults to `[]` (no class masks). The `ARGMAX` map is always returned regardless of this parameter.
71
-
-[`resizeToInput`](../../06-api-reference/interfaces/ImageSegmentationType.md#forward) (optional) - Whether to resize the output masks to the original input image dimensions. Defaults to `true`. If `false`, returns the raw model output dimensions (e.g. 224x224 for `DEEPLAB_V3_RESNET50`).
69
+
-[`imageSource`](../../06-api-reference/interfaces/SemanticSegmentationType.md#forward) (required) - The image to segment. Can be a remote URL, a local file URI, or a base64-encoded image (whole URI or only raw base64).
70
+
-[`classesOfInterest`](../../06-api-reference/interfaces/SemanticSegmentationType.md#forward) (optional) - An array of label keys indicating which per-class probability masks to include in the output. Defaults to `[]` (no class masks). The `ARGMAX` map is always returned regardless of this parameter.
71
+
-[`resizeToInput`](../../06-api-reference/interfaces/SemanticSegmentationType.md#forward) (optional) - Whether to resize the output masks to the original input image dimensions. Defaults to `true`. If `false`, returns the raw model output dimensions (e.g. 224x224 for `DEEPLAB_V3_RESNET50`).
72
72
73
73
:::warning
74
74
Setting `resizeToInput` to `false` will make `forward` faster.
@@ -85,13 +85,13 @@ The return type is fully typed — TypeScript narrows it based on the labels you
Copy file name to clipboardExpand all lines: docs/docs/04-typescript-api/02-computer-vision/SemanticSegmentationModule.md
+17-17Lines changed: 17 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,26 +1,26 @@
1
1
---
2
-
title: ImageSegmentationModule
2
+
title: SemanticSegmentationModule
3
3
---
4
4
5
-
TypeScript API implementation of the [useImageSegmentation](../../03-hooks/02-computer-vision/useImageSegmentation.md) hook.
5
+
TypeScript API implementation of the [useSemanticSegmentation](../../03-hooks/02-computer-vision/useSemanticSegmentation.md) hook.
6
6
7
7
## API Reference
8
8
9
-
- For detailed API Reference for `ImageSegmentationModule` see: [`ImageSegmentationModule` API Reference](../../06-api-reference/classes/ImageSegmentationModule.md).
10
-
- For all image segmentation models available out-of-the-box in React Native ExecuTorch see: [Image Segmentation Models](../../06-api-reference/index.md#models---image-segmentation).
9
+
- For detailed API Reference for `SemanticSegmentationModule` see: [`SemanticSegmentationModule` API Reference](../../06-api-reference/classes/SemanticSegmentationModule.md).
10
+
- For all semantic segmentation models available out-of-the-box in React Native ExecuTorch see: [Semantic Segmentation Models](../../06-api-reference/index.md#models---semantic-segmentation).
@@ -32,18 +32,18 @@ const result = await segmentation.forward(imageUri);
32
32
33
33
### Methods
34
34
35
-
All methods of `ImageSegmentationModule` are explained in details here: [`ImageSegmentationModule` API Reference](../../06-api-reference/classes/ImageSegmentationModule.md)
35
+
All methods of `SemanticSegmentationModule` are explained in details here: [`SemanticSegmentationModule` API Reference](../../06-api-reference/classes/SemanticSegmentationModule.md)
36
36
37
37
## Loading the model
38
38
39
-
`ImageSegmentationModule` uses static factory methods instead of `new()` + `load()`. There are two ways to create an instance:
39
+
`SemanticSegmentationModule` uses static factory methods instead of `new()` + `load()`. There are two ways to create an instance:
40
40
41
41
### Built-in models — `fromModelName`
42
42
43
-
Use [`fromModelName`](../../06-api-reference/classes/ImageSegmentationModule.md#frommodelname) for models that ship with built-in label maps and preprocessing configs:
43
+
Use [`fromModelName`](../../06-api-reference/classes/SemanticSegmentationModule.md#frommodelname) for models that ship with built-in label maps and preprocessing configs:
@@ -53,12 +53,12 @@ The `config` parameter is a discriminated union — TypeScript ensures you provi
53
53
54
54
### Custom models — `fromCustomConfig`
55
55
56
-
Use [`fromCustomConfig`](../../06-api-reference/classes/ImageSegmentationModule.md#fromcustomconfig) for custom-exported segmentation models with your own label map:
56
+
Use [`fromCustomConfig`](../../06-api-reference/classes/SemanticSegmentationModule.md#fromcustomconfig) for custom-exported segmentation models with your own label map:
@@ -76,11 +76,11 @@ For more information on loading resources, take a look at [loading models](../..
76
76
77
77
## Running the model
78
78
79
-
To run the model, use the [`forward`](../../06-api-reference/classes/ImageSegmentationModule.md#forward) method. It accepts three arguments:
79
+
To run the model, use the [`forward`](../../06-api-reference/classes/SemanticSegmentationModule.md#forward) method. It accepts three arguments:
80
80
81
-
-[`imageSource`](../../06-api-reference/classes/ImageSegmentationModule.md#forward) (required) - The image to segment. Can be a remote URL, a local file URI, or a base64-encoded image (whole URI or only raw base64).
82
-
-[`classesOfInterest`](../../06-api-reference/classes/ImageSegmentationModule.md#forward) (optional) - An array of label keys indicating which per-class probability masks to include in the output. Defaults to `[]`. The `ARGMAX` map is always returned regardless.
83
-
-[`resizeToInput`](../../06-api-reference/classes/ImageSegmentationModule.md#forward) (optional) - Whether to resize the output masks to the original input image dimensions. Defaults to `true`. If `false`, returns the raw model output dimensions.
81
+
-[`imageSource`](../../06-api-reference/classes/SemanticSegmentationModule.md#forward) (required) - The image to segment. Can be a remote URL, a local file URI, or a base64-encoded image (whole URI or only raw base64).
82
+
-[`classesOfInterest`](../../06-api-reference/classes/SemanticSegmentationModule.md#forward) (optional) - An array of label keys indicating which per-class probability masks to include in the output. Defaults to `[]`. The `ARGMAX` map is always returned regardless.
83
+
-[`resizeToInput`](../../06-api-reference/classes/SemanticSegmentationModule.md#forward) (optional) - Whether to resize the output masks to the original input image dimensions. Defaults to `true`. If `false`, returns the raw model output dimensions.
84
84
85
85
:::warning
86
86
Setting `resizeToInput` to `false` will make `forward` faster.
@@ -107,4 +107,4 @@ result.DOG; // Float32Array
107
107
108
108
## Managing memory
109
109
110
-
The module is a regular JavaScript object, and as such its lifespan will be managed by the garbage collector. In most cases this should be enough, and you should not worry about freeing the memory of the module yourself, but in some cases you may want to release the memory occupied by the module before the garbage collector steps in. In this case use the method [`delete`](../../06-api-reference/classes/ImageSegmentationModule.md#delete) on the module object you will no longer use, and want to remove from the memory. Note that you cannot use [`forward`](../../06-api-reference/classes/ImageSegmentationModule.md#forward) after [`delete`](../../06-api-reference/classes/ImageSegmentationModule.md#delete) unless you create a new instance.
110
+
The module is a regular JavaScript object, and as such its lifespan will be managed by the garbage collector. In most cases this should be enough, and you should not worry about freeing the memory of the module yourself, but in some cases you may want to release the memory occupied by the module before the garbage collector steps in. In this case use the method [`delete`](../../06-api-reference/classes/SemanticSegmentationModule.md#delete) on the module object you will no longer use, and want to remove from the memory. Note that you cannot use [`forward`](../../06-api-reference/classes/SemanticSegmentationModule.md#forward) after [`delete`](../../06-api-reference/classes/SemanticSegmentationModule.md#delete) unless you create a new instance.
0 commit comments