Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 20 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,49 +59,42 @@ This breakthrough is driven by an innovative data engine that has automatically

### Prerequisites

- [uv](https://github.com/astral-sh/uv) installed
- Python 3.12 or higher
- PyTorch 2.7 or higher
- CUDA-compatible GPU with CUDA 12.6 or higher

1. **Create a new Conda environment:**
1. **Clone the repository:**

```bash
conda create -n sam3 python=3.12
conda deactivate
conda activate sam3
git clone https://github.com/facebookresearch/sam3.git
cd sam3
```

2. **Install PyTorch with CUDA support:**
2. **Sync the environment:**

```bash
pip install torch==2.7.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126
```

3. **Clone the repository and install the package:**
This will create a virtual environment, install Python 3.12, PyTorch 2.7.0 (with CUDA 12.6), and all project dependencies.

```bash
git clone https://github.com/facebookresearch/sam3.git
cd sam3
pip install -e .
uv sync --extra notebooks --extra train --extra dev
```

4. **Install additional dependencies for example notebooks or development:**
3. **Verify installation:**

```bash
# For running example notebooks
pip install -e ".[notebooks]"

# For development
pip install -e ".[train,dev]"
uv run python -c "import torch; import sam3; print('Installation successful!')"
```

## Getting Started

⚠️ Before using SAM 3, please request access to the checkpoints on the SAM 3
Hugging Face [repo](https://huggingface.co/facebook/sam3). Once accepted, you
need to be authenticated to download the checkpoints. You can do this by running
the following [steps](https://huggingface.co/docs/huggingface_hub/en/quick-start#authentication)
(e.g. `hf auth login` after generating an access token.)
need to be authenticated to download the checkpoints. You can do this by running:

```bash
uv run huggingface-cli login
```

and pasting your access token from [your settings](https://huggingface.co/settings/tokens).

### Basic Usage

Expand Down Expand Up @@ -174,11 +167,8 @@ dataset.
To run the Jupyter notebook examples:

```bash
# Make sure you have the notebooks dependencies installed
pip install -e ".[notebooks]"

# Start Jupyter notebook
jupyter notebook examples/sam3_image_predictor_example.ipynb
# Start Jupyter notebook (notebooks dependencies are installed via uv sync)
uv run jupyter notebook examples/sam3_image_predictor_example.ipynb
```

## Model
Expand Down Expand Up @@ -347,13 +337,13 @@ We release 2 image benchmarks, [SA-Co/Gold](scripts/eval/gold/README.md) and
To set up the development environment:

```bash
pip install -e ".[dev,train]"
uv sync --extra dev --extra train
```

To format the code:

```bash
ufmt format .
uv run ufmt format .
```

## Contributing
Expand Down
2 changes: 1 addition & 1 deletion README_TRAIN.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This repository supports finetuning SAM3 models on custom datasets in multi-node

```bash
cd sam3
pip install -e ".[train]"
uv sync --extra train
```

### Training Script Usage
Expand Down
9 changes: 3 additions & 6 deletions examples/sam3_image_predictor_example.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -108,18 +108,15 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {
"output": {
"id": 785101161160169,
"loadingStatus": "loaded"
}
},
"outputs": [],
"source": [
"bpe_path = f\"{sam3_root}/assets/bpe_simple_vocab_16e6.txt.gz\"\n",
"model = build_sam3_image_model(bpe_path=bpe_path)"
]
"source": "model = build_sam3_image_model()"
},
{
"cell_type": "code",
Expand Down Expand Up @@ -371,4 +368,4 @@
},
"nbformat": 4,
"nbformat_minor": 4
}
}
59 changes: 59 additions & 0 deletions examples/sample_run.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@

import os
import torch
import numpy as np
from PIL import Image
from sam3.model_builder import build_sam3_image_model
from sam3.model.sam3_image_processor import Sam3Processor

def run_sample():
print("Setting up SAM3...")

# Check if assets exist
# Updated to look one directory up for assets since this is in examples/
image_path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "assets/images/test_image.jpg")

if not os.path.exists(image_path):
print(f"Warning: {image_path} not found. Please ensure the assets directory exists.")
return

try:
# Load the model
print("Loading SAM3 model (this may trigger checkpoint download)...")
model = build_sam3_image_model()
# Lower confidence threshold from default 0.5 to detect more objects
processor = Sam3Processor(model, confidence_threshold=0.1)

# Load an image
print(f"Loading image from {image_path}...")
image = Image.open(image_path)

# Run inference
print("Running inference...")
inference_state = processor.set_image(image)

# Prompt the model
prompt_text = "kid wearing a red bib"
print(f"Prompting with: '{prompt_text}'")
output = processor.set_text_prompt(state=inference_state, prompt=prompt_text)

# Get results
masks = output["masks"]
boxes = output["boxes"]
scores = output["scores"]

print("\nSuccess!")
print(f"Found {len(masks)} masks.")
print(f"Scores: {scores[:5]} (showing top 5)")
print(f"Masks shape: {masks.shape}")

except Exception as e:
print("\nError occurred:")
print(str(e))
if "huggingface" in str(e).lower() or "unauthorized" in str(e).lower() or "login" in str(e).lower():
print("\nIt looks like an authentication error.")
print("Please run: uv run huggingface-cli login")
print("And paste your token from https://huggingface.co/settings/tokens")

if __name__ == "__main__":
run_sample()
8 changes: 7 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ name = "sam3"
dynamic = ["version"]
description = "SAM3 (Segment Anything Model 3) implementation"
readme = "README.md"
requires-python = ">=3.8"
requires-python = ">=3.9"
license = {file = "LICENSE"}
authors = [
{name = "Meta AI Research"}
Expand All @@ -33,6 +33,9 @@ dependencies = [
"iopath>=0.1.10",
"typing_extensions",
"huggingface_hub",
"torch==2.7.0",
"torchvision>=0.22.0",
"torchaudio>=2.7.0",
]

[project.optional-dependencies]
Expand Down Expand Up @@ -133,3 +136,6 @@ testpaths = ["tests"]
python_files = "test_*.py"
python_classes = "Test*"
python_functions = "test_*"

[[tool.uv.index]]
url = "https://download.pytorch.org/whl/cu126"
Loading