Skip to content

Commit d256885

Browse files
authoredNov 2, 2024
docs(README): expand with recording, training, and deployment instructions
1 parent 7ee6e2e commit d256885

File tree

1 file changed

+66
-3
lines changed

1 file changed

+66
-3
lines changed
 

‎README.md

+66-3
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ OpenAdapter simplifies deploying advanced screenshot parsing and action models o
1010
- **Cost-Efficiency**: Deploy high-performance instances on demand, with intelligent caching and resource pause/stop features to reduce costs.
1111
- **Container & API Compatibility**: Supports Dockerized models like OmniParser and Set-of-Mark, with future support for Anthropic and OpenAI APIs.
1212
- **CI/CD with GitHub Actions**: Automated integration and deployment ensure consistent updates.
13+
- **Dataset Preparation and Fine-Tuning**: Collect and fine-tune your models with OpenAdapter’s tools for recording, preparing, and training models using user interaction data, such as screenshots and actions, captured directly in your application.
1314

1415
### Prerequisites
1516
- **Python 3.10+**
@@ -41,9 +42,41 @@ OpenAdapter simplifies deploying advanced screenshot parsing and action models o
4142
```
4243

4344
## 💡 Usage
44-
Straightforward commands to deploy and manage model instances.
45+
OpenAdapter provides commands to deploy and manage model instances and capture user interactions for fine-tuning. You can record, train, and tune models with your custom dataset, all managed within the OpenAdapter environment.
4546

46-
### Example Deployment Script (OmniParser)
47+
### Recording User Interactions
48+
To capture user interactions such as screenshots and actions, use OpenAdapter’s `record` command:
49+
```bash
50+
python -m openadapter.record "doing taxes"
51+
```
52+
This command will save the actions in a database file:
53+
```plaintext
54+
Actions saved to ~/openadapter/recording.db
55+
```
56+
57+
### Preparing and Fine-Tuning the Model
58+
Use the recorded data to prepare a dataset and fine-tune your model:
59+
60+
1. **Prepare Dataset**:
61+
```bash
62+
python -m openadapter.train.prepare ~/openadapter/recording.db
63+
```
64+
Example output:
65+
```plaintext
66+
Preparing dataset from: ~/openadapter/recording.db
67+
Dataset prepared at ~/openadapter/prepared_data
68+
```
69+
70+
2. **Fine-Tune Model**:
71+
After preparing the dataset, specify the paths to the prepared data for fine-tuning:
72+
```bash
73+
python -m openadapter.train.tune --caption_model_path=~/openadapter/prepared_data/caption_model --som_model_path=~/openadapter/prepared_data/som_model
74+
```
75+
76+
This flow enables OpenAdapter to use custom datasets created with OpenAdapt for more accurate action detection and screenshot parsing. Adjust paths based on your local setup.
77+
78+
### Deployment Example (OmniParser)
79+
Deploy OmniParser using an AWS GPU instance with OpenAdapter:
4780
```python
4881
from openadapter.server import OpenAdapterConfig, Deploy
4982
import fire
@@ -95,7 +128,37 @@ python -m oa.deploy ssh
95128
- **Planned**: Hugging Face, Anthropic, OpenAI; future support for GCP and Azure.
96129

97130
## Integrations
98-
Works with OpenAdapt or as a standalone solution for automated model deployment.
131+
OpenAdapter works seamlessly with OpenAdapt to build datasets and automate models. It can also function as a standalone solution for deploying and managing models in automated environments.
132+
133+
## Requirements
134+
135+
### Core Requirements
136+
- **Python 3.10 or higher**
137+
138+
### Optional Components
139+
Install specific dependencies based on the use case:
140+
141+
1. **Recording**: Required for capturing user interactions.
142+
```bash
143+
pip install openadapter[record]
144+
```
145+
146+
2. **Training**: Includes dependencies for preparing datasets and fine-tuning models.
147+
```bash
148+
pip install openadapter[train]
149+
```
150+
151+
3. **Deployment**: Necessary for deploying models to production.
152+
```bash
153+
pip install openadapter[deploy]
154+
```
155+
156+
4. **Full Installation**: Installs all dependencies for full-featured use.
157+
```bash
158+
pip install openadapter[full]
159+
```
160+
161+
> **Note**: GPU support is recommended for training and fine-tuning tasks, especially when working with large models like YOLO and BLIP2.
99162
100163
## 🛠️ Roadmap
101164
- **AWS CDK Automation**: Streamline Infrastructure as Code.

0 commit comments

Comments
 (0)