Skip to content

Commit

Permalink
Merge branch 'develop' into fix-shadowserver-fixed-report-type
Browse files Browse the repository at this point in the history
  • Loading branch information
kamil-certat committed Feb 5, 2024
2 parents f679e90 + 51994d1 commit 2ec8227
Show file tree
Hide file tree
Showing 3 changed files with 18 additions and 18 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,9 @@
- Minor fixes/improvements and some refactoring (see also above: *Core*...).
- `intelmq.bots.collectors.shadowserver.collector_reports_api`:
- The 'json' option is no longer supported as the 'csv' option provides better performance (PR#2372 by elsif2).
- `intelmq.bots.collectors.alienvault_otx.collector` (PR#2449 by qux-bbb):
- Fix modified_pulses_only is always False.


#### Parsers
- `intelmq.bots.parsers.netlab_360.parser`: Removed as the feed is discontinued. (#2442 by Filip Pokorný)
Expand Down
29 changes: 15 additions & 14 deletions docs/user/bots.md
Original file line number Diff line number Diff line change
Expand Up @@ -2128,23 +2128,23 @@ correct mapping of the columns:

1. **Automatic report type detection**

Since IntelMQ version 2.1 the parser can detect the feed based on metadata provided by the collector.

When processing a report, this bot takes `extra.file_name` from the report and looks in `config.py` how the report
should be parsed. If this lookup is not possible, and the `feedname` is not given as parameter, the feed cannot be
parsed.

The field `extra.file_name` has the following structure: `%Y-%m-%d-${report_name}[-suffix].csv` where the optional
suffix can be something like `country-geo`. For example, some possible filenames
are `2019-01-01-scan_http-country-geo.csv` or `2019-01-01-scan_tftp.csv`. The important part is the `report_name`,
between the date and the suffix. Since version 2.1.2 the date in the filename is optional, so filenames
like `scan_tftp.csv` are also detected.
Since IntelMQ version 2.1 the parser can detect the feed based on metadata provided by the collector.
When processing a report, this bot takes `extra.file_name` from the report and looks in `config.py` how the report
should be parsed. If this lookup is not possible, and the `feedname` is not given as parameter, the feed cannot be
parsed.
The field `extra.file_name` has the following structure: `%Y-%m-%d-${report_name}[-suffix].csv` where the optional
suffix can be something like `country-geo`. For example, some possible filenames
are `2019-01-01-scan_http-country-geo.csv` or `2019-01-01-scan_tftp.csv`. The important part is the `report_name`,
between the date and the suffix. Since version 2.1.2 the date in the filename is optional, so filenames
like `scan_tftp.csv` are also detected.

2. **Fixed report type**

If the method above is not possible and for upgraded instances, the report type can be set with the `feedname`
parameter. Report type is derived from the subject of Shadowserver e-mails. A list of possible values of
the `feedname` parameter can be found in the table below in the column "Report Type".
If the method above is not possible and for upgraded instances, the report type can be set with the `feedname`
parameter. Report type is derived from the subject of Shadowserver e-mails. A list of possible values of
the `feedname` parameter can be found in the table below in the column "Report Type".

**Module:**

Expand Down Expand Up @@ -2200,6 +2200,7 @@ The schema revision history is maintained at https://github.com/The-Shadowserver
auto_update: true
run_mode: continuous
```

---

### Shodan <div id="intelmq.bots.parsers.shodan.parser" />
Expand Down
4 changes: 0 additions & 4 deletions intelmq/bots/collectors/alienvault_otx/collector.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,6 @@ def init(self):
if OTXv2 is None:
raise MissingDependencyError("OTXv2")

self.modified_pulses_only = False
if hasattr(self, 'modified_pulses_only'):
self.modified_pulses_only = self.modified_pulses_only

def process(self):
self.logger.info("Downloading report through API.")
otx = OTXv2(self.api_key, proxy=self.https_proxy)
Expand Down

0 comments on commit 2ec8227

Please sign in to comment.