Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update NEWS.md in preparation for 3.3.0 #2468

Merged
merged 1 commit into from
Feb 29, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 52 additions & 7 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,20 +14,65 @@ Please refer to the change log for a full list of changes.
---------------------------------

### Documentation
The documentation is now available at [docs.intelmq.org](https://docs.intelmq.org/). Documentation has been updated and restructured into User, Administrator and Developer Guide. It provides modern look with various quality of life improvements.
The documentation is now available at [docs.intelmq.org](https://docs.intelmq.org/). Documentation has been updated and restructured into User, Administrator and Developer Guide. It provides modern look with various quality of life improvements. Big thanks to to @gethvi.
We now have a slick, modern mkdocs based documentation. Please do check it out!

### Requirements

### Tools
### Bots
#### Shadowserver dynamic parser / collector

### Data Format
**Note well**: if you use shadowserver feeds, **please read this section carefully**.

### Configuration
Thanks to shadowserver (@elsif2), we have a new dynamic shadowserver reports API integration. What does it do?
It connects to the [Shadowserver API](https://www.shadowserver.org/what-we-do/network-reporting/api-documentation/),
requests a list of all the reports for a specific country and processes the ones that are new.

### Libraries
Motivation for this change:

### Postgres databases
Shadowserver adds new scans on a nearly weekly basis. IntelMQ's release cycle and the need for a stable release could not keep up with this high intensity of shadowserver parser changes.
We therefore (thanks to @eslif2) move the shadowserver reports collector and parser to a new, dynamic system. It can:

- fetch the shadowserver schema from shadowserver (https://interchange.shadowserver.org/intelmq/v1/schema)
- dynamically collect new reports (see also https://docs.intelmq.org/latest/user/bots/?h=shadow#shadowserver-reports-api)
- parse the new reports

**Note well**: if your IntelMQ system runs in an airgapped environment or if it may only reach out to specific IPs/sites, you should read the notes here:
https://docs.intelmq.org/latest/user/bots/#shadowserver.
You will need to download shadowserver-schema.json periodically yourself in this case.

**Note well:**: since dynamic changes are a bit tricky, we defined that there is a schema contract:

> Schema contract
>
> Once set in the schema, the classification.identifier, classification.taxonomy, and classification.type fields will remain static for a specific report.

This makes things deterministic again.

#### Alienvault OTX

Fix of a bug where a certain condition would have always evaluated to False. (PR#2449 by qux-bbb. Thanks)

#### AMQP
Quite a few changes (thanks to Kamil, @gethvi) on AMQP

#### Obsoleted bots

- `intelmq.bots.parsers.netlab_360.parser`: Removed as the feed is discontinued. (#2442 by Filip Pokorný)
- `intelmq.bots.parsers.webinspektor.parser`: Removed as the feed is discontinued. (#2442 by Filip Pokorný)
- `intelmq.bots.parsers.sucuri.parser`: Removed as the feed is discontinued. (#2442 by Filip Pokorný)

### General changes and bug fixes

Digital Trust Center fixed a bug where the config was loaded twice in intelmqctl which created quite some speedups. Thanks!
This speeds up IntelMQ API calls.

### Data Format

Shadowserver dynamic parser (see above).

### General remarks

The full list of changes can be seen in the CHANGELOG.md file.

3.2.1 Bugfix release (2023-08-23)
---------------------------------
Expand Down
Loading