You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: README.md
+66-3
Original file line number
Diff line number
Diff line change
@@ -10,6 +10,7 @@ OpenAdapter simplifies deploying advanced screenshot parsing and action models o
10
10
-**Cost-Efficiency**: Deploy high-performance instances on demand, with intelligent caching and resource pause/stop features to reduce costs.
11
11
-**Container & API Compatibility**: Supports Dockerized models like OmniParser and Set-of-Mark, with future support for Anthropic and OpenAI APIs.
12
12
-**CI/CD with GitHub Actions**: Automated integration and deployment ensure consistent updates.
13
+
-**Dataset Preparation and Fine-Tuning**: Collect and fine-tune your models with OpenAdapter’s tools for recording, preparing, and training models using user interaction data, such as screenshots and actions, captured directly in your application.
13
14
14
15
### Prerequisites
15
16
-**Python 3.10+**
@@ -41,9 +42,41 @@ OpenAdapter simplifies deploying advanced screenshot parsing and action models o
41
42
```
42
43
43
44
## 💡 Usage
44
-
Straightforward commands to deploy and manage model instances.
45
+
OpenAdapter provides commands to deploy and manage model instances and capture user interactions for fine-tuning. You can record, train, and tune models with your custom dataset, all managed within the OpenAdapter environment.
45
46
46
-
### Example Deployment Script (OmniParser)
47
+
### Recording User Interactions
48
+
To capture user interactions such as screenshots and actions, use OpenAdapter’s `record` command:
49
+
```bash
50
+
python -m openadapter.record "doing taxes"
51
+
```
52
+
This command will save the actions in a database file:
53
+
```plaintext
54
+
Actions saved to ~/openadapter/recording.db
55
+
```
56
+
57
+
### Preparing and Fine-Tuning the Model
58
+
Use the recorded data to prepare a dataset and fine-tune your model:
This flow enables OpenAdapter to use custom datasets created with OpenAdapt for more accurate action detection and screenshot parsing. Adjust paths based on your local setup.
77
+
78
+
### Deployment Example (OmniParser)
79
+
Deploy OmniParser using an AWS GPU instance with OpenAdapter:
47
80
```python
48
81
from openadapter.server import OpenAdapterConfig, Deploy
49
82
import fire
@@ -95,7 +128,37 @@ python -m oa.deploy ssh
95
128
-**Planned**: Hugging Face, Anthropic, OpenAI; future support for GCP and Azure.
96
129
97
130
## Integrations
98
-
Works with OpenAdapt or as a standalone solution for automated model deployment.
131
+
OpenAdapter works seamlessly with OpenAdapt to build datasets and automate models. It can also function as a standalone solution for deploying and managing models in automated environments.
132
+
133
+
## Requirements
134
+
135
+
### Core Requirements
136
+
-**Python 3.10 or higher**
137
+
138
+
### Optional Components
139
+
Install specific dependencies based on the use case:
140
+
141
+
1.**Recording**: Required for capturing user interactions.
142
+
```bash
143
+
pip install openadapter[record]
144
+
```
145
+
146
+
2.**Training**: Includes dependencies for preparing datasets and fine-tuning models.
147
+
```bash
148
+
pip install openadapter[train]
149
+
```
150
+
151
+
3.**Deployment**: Necessary for deploying models to production.
152
+
```bash
153
+
pip install openadapter[deploy]
154
+
```
155
+
156
+
4.**Full Installation**: Installs all dependencies for full-featured use.
157
+
```bash
158
+
pip install openadapter[full]
159
+
```
160
+
161
+
> **Note**: GPU support is recommended for training and fine-tuning tasks, especially when working with large models like YOLO and BLIP2.
99
162
100
163
## 🛠️ Roadmap
101
164
-**AWS CDK Automation**: Streamline Infrastructure as Code.
0 commit comments