You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/Advanced-Usage.md
+8-9Lines changed: 8 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -73,12 +73,12 @@ table, th, td {
73
73
74
74
75
75
To enable all the optional analyzers you can add the option `--all_analyzers` when starting the project. Example:
76
-
```
76
+
```bash
77
77
python3 start.py prod --all_analyzers up
78
78
```
79
79
80
80
Otherwise you can enable just one of the cited integration by using the related option. Example:
81
-
```
81
+
```bash
82
82
python3 start.py prod --qiling up
83
83
```
84
84
@@ -136,19 +136,18 @@ List of some of the analyzers with optional configuration:
136
136
* Every Chrome-Extension has a unique alpha=numerc identifier. That's the only Input necessary. Eg: `Norton Safe Search Enhanced`'s identifier is `eoigllimhcllmhedfbmahegmoakcdakd`.
137
137
138
138
139
-
140
139
There are two ways to do this:
141
140
142
-
#### from the GUI
141
+
#####from the GUI
143
142
You can click on "**Custom analyzer configuration**" button and add the runtime configuration in the form of a dictionary.
144
143
Example:
145
-
```
144
+
```javascript
146
145
"VirusTotal_v3_Get_File": {
147
146
"force_active_scan_if_old":true
148
147
}
149
148
```
150
149
151
-
#### from [Pyintelowl](https://github.com/intelowlproject/pyintelowl)
While using `send_observable_analysis_request` or `send_file_analysis_request` endpoints, you can pass the parameter `runtime_configuration` with the optional values.
153
152
Example:
154
153
```python
@@ -231,17 +230,17 @@ Refer to the following blog post for an example on how to deploy IntelOwl on Goo
231
230
[Deploying Intel-Owl on GKE](https://mostwanted002.cf/post/intel-owl-gke/) by [Mayank Malik](https://twitter.com/_mostwanted002_).
232
231
233
232
## Multi Queue
234
-
IntelOwl provides an additional `docker-compose` file, [multi-queue.override.yaml](https://github.com/intelowlproject/IntelOwl/blob/master/docker/multi-queue.override.yml) file, allowing IntelOwl users to better scale with the performance of their own architecture.
233
+
IntelOwl provides an additional [multi-queue.override.yml](https://github.com/intelowlproject/IntelOwl/blob/master/docker/multi-queue.override.yml)compose file allowing IntelOwl users to better scale with the performance of their own architecture.
235
234
236
235
If you want to leverage it, you should add the option `--multi-queue` when starting the project. Example:
237
-
```
236
+
```bash
238
237
python3 start.py prod --multi-queue up
239
238
```
240
239
241
240
This functionality is not enabled by default because this deployment would start 2 more containers so the resource consumption is higher. We suggest to use this option only when leveraging IntelOwl massively.
242
241
243
242
#### Queue Customization
244
-
It is possible to define new celery workers: each requires the addition of a new container in the docker-compose file, as shown in the `multi-queue.override.yaml`.
243
+
It is possible to define new celery workers: each requires the addition of a new container in the docker-compose file, as shown in the `multi-queue.override.yml`.
245
244
246
245
Moreover IntelOwl requires that the name of the workers are provided in the `docker-compose` file. This is done through the environment variable `CELERY_QUEUES` inside the `uwsgi` container. Each queue must be separated using the character `,`, as shown in the [example](https://github.com/intelowlproject/IntelOwl/blob/master/docker/multi-queue.override.yml#L6).
Copy file name to clipboardExpand all lines: docs/source/Contribute.md
+12-15Lines changed: 12 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,7 +13,7 @@ Please create a new branch based on the **develop** branch that contains the mos
13
13
`git checkout -b myfeature develop`
14
14
15
15
Then we strongly suggest to configure [pre-commit](https://github.com/pre-commit/pre-commit) to force linters on every commits you perform:
16
-
```
16
+
```bash
17
17
# create virtualenv to host pre-commit installation
18
18
python3 -m venv intel_owl_test_env
19
19
source intel_owl_test_env/bin/activate
@@ -29,11 +29,11 @@ You may want to look at a few existing examples to start to build a new one, suc
29
29
-[peframe.py](https://github.com/intelowlproject/IntelOwl/blob/develop/api_app/script_analyzers/file_analyzers/peframe.py), if you are creating a [docker based analyzer](#integrating-a-docker-based-analyzer)
30
30
31
31
After having written the new python module, you have to remember to:
32
-
* Put the module in the `file_analyzers` or `observable_analyzers` directory based on what it can analyze
33
-
* Add a new entry in the [analyzer configuration](https://github.com/intelowlproject/IntelOwl/blob/master/configuration/analyzer_config.json) following alphabetical order:
32
+
1. Put the module in the `file_analyzers` or `observable_analyzers` directory based on what it can analyze
33
+
2. Add a new entry in the [analyzer configuration](https://github.com/intelowlproject/IntelOwl/blob/master/configuration/analyzer_config.json) following alphabetical order:
34
34
35
35
Example:
36
-
```
36
+
```javascript
37
37
"Analyzer_Name": {
38
38
"type":"file",
39
39
"external_service":true,
@@ -64,31 +64,28 @@ After having written the new python module, you have to remember to:
64
64
Please see [Analyzers customization section](https://intelowl.readthedocs.io/en/latest/Usage.html#analyzers-customization) to get the explanation of the other available keys.
65
65
66
66
67
-
* Add required unit tests in the [tests](https://github.com/intelowlproject/IntelOwl/blob/master/tests) folder.
68
-
69
-
Then follow the [Test](./Tests.md) guide to start testing.
67
+
3. Add required unit tests in the [tests](https://github.com/intelowlproject/IntelOwl/blob/master/tests) folder. Then follow the [Test](./Tests.md) guide to start testing.
70
68
71
-
* Add the new analyzer/s in the lists in the docs: [Usage](./Usage.md). Also, if the analyzer provides additional optional configuration, add the available options here: [Advanced-Usage](./Advanced-Usage.md)
69
+
4. Add the new analyzer in the lists in the docs: [Usage](./Usage.md). Also, if the analyzer provides additional optional configuration, add the available options here: [Advanced-Usage](./Advanced-Usage.md)
72
70
73
-
* Ultimately, add the required secrets in the files [env_file_app_template](https://github.com/intelowlproject/IntelOwl/blob/master/env_file_app_template), [env_file_app_ci](https://github.com/certego/IntelOwl/blob/master/env_file_app_travis) and in the docs: [Installation](./Installation.md)
71
+
5. Ultimately, add the required secrets in the files `docker/env_file_app_template`, `docker/env_file_app_ci` and in the `docs/Installation.md`.
74
72
75
-
* In the Pull Request remember to provide some real world examples (screenshots and raw JSON results) of some successful executions of the analyzer to let us understand how it would work.
73
+
5. In the Pull Request remember to provide some real world examples (screenshots and raw JSON results) of some successful executions of the analyzer to let us understand how it would work.
76
74
77
75
### Integrating a docker based analyzer
78
76
If the analyzer you wish to integrate doesn't exist as a callable API online or python package, it should be integrated with its own docker image
79
77
which can be queried from the main Django API.
80
78
81
79
* It should follow the same design principle as the [Box-Js integration](https://github.com/intelowlproject/IntelOwl/tree/develop/integrations), unless there's very good reason not to.
82
80
* The dockerfile should be placed at `./integrations/<analyzer_name>/Dockerfile`.
83
-
* A docker-compose file should be placed under `./integrations` with the name `docker-compose.<analyzer_name>.yml`
84
-
* If your docker-image uses any environment variables, add them in the [`env_file_integrations_template`](https://github.com/intelowlproject/IntelOwl/blob/develop/env_file_integrations_template)
85
-
* Ultimately, append the name of your docker-compose file in the `COMPOSE_FILE` variables specified in [`.env`](https://github.com/intelowlproject/IntelOwl/blob/develop/.env). The reason for doing this is so that this service remains optional to the end-user.
81
+
* Two docker-compose files `compose.yml` for production and `compose-tests.yml` for testing should be placed under `./integrations/<analyzer_name>`.
82
+
* If your docker-image uses any environment variables, add them in the `docker/env_file_integrations_template`.
86
83
* Rest of the steps remain same as given under "How to add a new analyzer".
87
84
88
85
## Create a pull request
89
86
90
87
### Install testing requirements
91
-
1.Run `pip install -r test-requirements.txt` to install the requirements to validate your code.
88
+
Run `pip install -r test-requirements.txt` to install the requirements to validate your code.
92
89
93
90
#### Pass linting and tests
94
91
1. Run `psf/black` to lint the files automatically and then `flake8` to check:
@@ -140,7 +137,7 @@ Please create pull requests only for the branch **develop**. That code will be p
140
137
Also remember to pull the most recent changes available in the **develop** branch before submitting your PR. If your PR has merge conflicts caused by this behavior, it won't be accepted.
141
138
142
139
### Example: add an analyzer configuration for your own Yara signatures
Copy file name to clipboardExpand all lines: docs/source/Installation.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -157,7 +157,7 @@ There are 3 options to execute the web server:
157
157
158
158
Before using it, you should configure the configuration file `docker/traefik.override.yml` by changing the email address and the hostname where the application is served. For a detailed explanation follow the official documentation: [Traefix doc](https://docs.traefik.io/user-guides/docker-compose/acme-http/).
159
159
160
-
After the configuration is done, you can add the option `--traefik` while executing the `start.py`[script](https://intelowl.readthedocs.io/en/stable/Advanced-Usage.html#smart-start).
160
+
After the configuration is done, you can add the option `--traefik` while executing the [`start.py`](#run)
0 commit comments