Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
cca6498
update secrets baseline
AlyaGomaa Apr 16, 2026
482bc4a
add an auto update config param
AlyaGomaa Apr 16, 2026
b24031a
add an update.json file for deployed slips to check for updates
AlyaGomaa Apr 16, 2026
a0facae
call the auto update param auto_update_slips instead of auto_update
AlyaGomaa Apr 16, 2026
04ae7f0
add an UpdateManager class that handles checking for updates and doin…
AlyaGomaa Apr 16, 2026
9e1ff16
update manager: add a parser for update.json to check for backwards c…
AlyaGomaa Apr 16, 2026
a0d172a
add an undocumented flag (-u) that will only be used by and older sli…
AlyaGomaa Apr 16, 2026
289bdfa
add a function to check if a new slips version is available by compar…
AlyaGomaa Apr 16, 2026
2f9f47e
update_manager: only update slips if the auto_update param in the con…
AlyaGomaa Apr 16, 2026
4ed5e79
update_manager: check for available updates once a day
AlyaGomaa Apr 17, 2026
955e257
Add functions to keep track of the offset of the last line read in ea…
AlyaGomaa Apr 17, 2026
fe26c4f
rename the old Updatemanager to FeedsUpdateManager to avoid name matc…
AlyaGomaa Apr 17, 2026
1ec413c
process_manager: create a mp event and pass it to input and update ma…
AlyaGomaa Apr 17, 2026
b72d2b2
store the last read zeek logs offsets in the db when input detects th…
AlyaGomaa Apr 17, 2026
4f20328
send the current slips version with each pub/sub msg to avoid process…
AlyaGomaa Apr 17, 2026
0c9ee47
imodule: ensure that the pub/sub recvd msg doesn't belong to the upda…
AlyaGomaa Apr 17, 2026
52bf7fe
extract the txt msg only from the pubsub msgs and ignore metadata lik…
AlyaGomaa Apr 17, 2026
6356377
main: don't overwrite the output dir when slips is started with -u & …
AlyaGomaa Apr 20, 2026
aae1052
dont clear output log files on startup if slips is started with -u
AlyaGomaa Apr 20, 2026
ffe67f4
Add a centralized func (utils.initialize_logfile) for initializing lo…
AlyaGomaa Apr 20, 2026
1b76479
if slips is started with -u, read zeek log files from the offsets sto…
AlyaGomaa Apr 20, 2026
92656c5
dont flush the redis server as soon as slips connects to redis if sli…
AlyaGomaa Apr 20, 2026
a068bbb
delete unused timewindow updater thread
AlyaGomaa Apr 20, 2026
4c17fb7
fix the db unable to log when starting the cache redis db
AlyaGomaa Apr 20, 2026
6b29b7c
refactoring for clarity
AlyaGomaa Apr 20, 2026
d78e708
update unit tests
AlyaGomaa Apr 20, 2026
a5a1a49
removing the creation of the used zeek dir from main.py
AlyaGomaa Apr 21, 2026
aa821a4
add prints when slips is storing and restoring zeek offsets
AlyaGomaa Apr 21, 2026
752b048
move the creation of zeek dir to zeek_input_utils
AlyaGomaa Apr 21, 2026
2a300c8
each updated slips version now creates its own zeek dir inside output…
AlyaGomaa Apr 21, 2026
57d8615
remove the logic for creating and restoring offsets. each version of …
AlyaGomaa Apr 21, 2026
38ccd9a
Make sure any exports that happen at the end of the analysis don't ha…
AlyaGomaa Apr 22, 2026
7b6209a
Detect when slips is done updating, and start draining, and call shut…
AlyaGomaa Apr 22, 2026
5c4337d
git pull origin master before draining if slips detected that it shou…
AlyaGomaa Apr 22, 2026
c8d4170
start the new version of slips with -u before completely stopping the…
AlyaGomaa Apr 22, 2026
fdd3071
update unit tests
AlyaGomaa Apr 22, 2026
0e580dc
abort update if uncommitted changes were detected during a git pull
AlyaGomaa Apr 22, 2026
3c4048c
update_manager.py: fixthe func checking for slips new version
AlyaGomaa Apr 23, 2026
416f7d3
Make the new updated slips use the same CLI as the old slips. the old…
AlyaGomaa Apr 23, 2026
5532a78
if old slips is started by -m, make sure the new slips knows which po…
AlyaGomaa Apr 23, 2026
f6cd346
update unit tests
AlyaGomaa Apr 23, 2026
e623d50
update unit tests and docs
AlyaGomaa Apr 23, 2026
49f9445
add auto update docs
AlyaGomaa Apr 23, 2026
c0d4e19
update unit tests
AlyaGomaa Apr 23, 2026
0e3e89e
abort update on any git err
AlyaGomaa Apr 23, 2026
7eb7b76
update unit tests
AlyaGomaa Apr 23, 2026
9eca133
update unit tests
AlyaGomaa Apr 23, 2026
2ea4ff1
update iris main config for integration tests
AlyaGomaa Apr 24, 2026
d15c3c1
organize integration_tests dir to be able to group config files with …
AlyaGomaa Apr 25, 2026
d48e6e6
change the names of iris and slips config files used in test_iris()
AlyaGomaa Apr 25, 2026
75bacdc
test_iris(): split peer1 and peer2 handling into separate funcs
AlyaGomaa Apr 25, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .secrets.baseline
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@
"filename": "config/slips.yaml",
"hashed_secret": "4cac50cee3ad8e462728e711eac3e670753d5016",
"is_verified": false,
"line_number": 295
"line_number": 304
}
],
"dataset/test14-malicious-zeek-dir/http.log": [
Expand Down Expand Up @@ -7185,5 +7185,5 @@
}
]
},
"generated_at": "2026-04-08T14:13:03Z"
"generated_at": "2026-04-16T20:58:25Z"
}
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
* Add optional live Slips auto-update support controlled by `update.auto_update_slips`.


1.1.19 (Mar 31st, 2026)

* Add SSH bruteforce detection based on Zeek SSH, software, and notice logs.
Expand Down
4 changes: 2 additions & 2 deletions config/iris_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ Identity:
GenerateNewKey: true
Server:
Port: 9010
Host: 0.0.0.0
Host: null
DhtServerMode: 'true'
Redis:
Host: 127.0.0.1
Port: 6379
Port: 6644
Tl2NlChannel: iris_internal
PeerDiscovery:
DisableBootstrappingNodes: false
Expand Down
13 changes: 11 additions & 2 deletions config/slips.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
# This configuration file controls several aspects of the working of Slips.

---
update:
# Enable automatic live updates of the installed Slips version.
# This setting is separate from the feeds_update_manager module, which only
# updates TI feeds and related files during runtime.
# Automatic Slips updates may overwrite the default config files shipped
# with Slips. If you want to keep your local configuration changes intact, avoid editing the default config files.
# Instead, create separate config files with different names and use those.
auto_update_slips: false

output:
# Define the file names for the default output.
stdout: slips.log
Expand Down Expand Up @@ -206,7 +215,7 @@ modules:
# Add the names of other modules that you want to disable
# (use module snake_case names). Example,
# threat_intelligence, blocking, network_discovery, timeline, virustotal,
# rnn_cc_detection, flow_ml_detection, update_manager
# rnn_cc_detection, flow_ml_detection, feeds_update_manager
disable: [template]

# For each line in timeline file there is a timestamp.
Expand Down Expand Up @@ -302,7 +311,7 @@ virustotal:
#############################
threatintelligence:

# By default, slips starts without the TI files, and runs the update_manager
# By default, slips starts without the TI files, and runs the feeds_update_manager
# in the background. If this option is set to true, slips will not start
# analyzing the flows until the update manager finished and all TI files are
# loaded successfully.
Expand Down
147 changes: 147 additions & 0 deletions docs/immune/auto_update.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# Slips Auto Update

## Table of Contents

- [Overview](#overview)
- [How Auto-Update Works](#how-auto-update-works)
* [New version checks](#new-version-checks)
* [Updating Logic](#updating-logic)
+ [Redis handling](#redis-handling)
+ [Zeek log handling](#zeek-log-handling)
* [Draining and Shutdown of the old Slips](#draining-and-shutdown-of-the-old-slips)
- [How to use it](#how-to-use-it)
- [PR](#pr)

## Overview

Slips auto update functionality was designed to allow a running instance of
Slips to update itself with no downtime during the transition between versions.

Updates usually consist of:

A full application stop -> update -> restart sequence.
rather than simple restart.

That sequence would lead to downtime and temporarily missing of flows during the upgrade.

The implemented update mechanism was designed around "handover" instead of "restarts".
Where slips checks periodically for new compatible versions, pulls the update,
starts the new version, and orchestrates a controlled handover from the old
version to the new one.


## How Auto-Update Works

```text
Old Slips running
Check for compatible update
git pull origin master
Start new Slips with -u
New Slips restores state and starts processing
Old Slips drains
Old Slips graceful shutdown
New Slips continues normally
```


### New version checks

Slips checks for updates once per day.

This is handled by the UpdateManager, which:

- checks whether auto-update is enabled in the config file
- checks whether a new version exists
- checks compatibility before attempting update.

Compatibility is determined using an `update.json` file hosted with the
deployed version.

This file includes metadata about the new version such as:

- latest version,
- backwards compatibility,
- whether new dependencies are needed.


The compatibility parser was added so we avoid updating to incompatible new releases.


**The update is aborted if:**

- `auto_update_slips` is disabled in slips config.
- Slips is running on offline input instead of interface
- no newer version exists
- update is incompatible according to `update.json`
- local uncommitted changes are detected during `git pull`
- startup of the new version fails


### Updating Logic


When designing this, our main goal was zero downtime and zero missed flows during the update. This has the cost of maybe reading a very few duplicate flows, and this was done by
Starting the new Slips before stopping the old one.

How this is done is:
- The old version starts the new one with the undocumented `-u` flag.
- The `-u` flag tells the new Slips instance that: 1. this is not a fresh run and 2. Slips should continue existing analysis and handle database migrations.
- do not overwrite: output dir, log files, and previous analysis artifacts/metrics.


#### Redis handling

The new Slips does not flush Redis on startup.

Instead, it appends to the existing Redis state as if the old process never
stopped.

Without this, ongoing detections, states, and evidence state would be lost.


Now what happen when in the very few seconds during handover, a msg from the old slips' pub/sub is published, and the new updated slips receives it?

To avoid this we added Pub/Sub message versioning, now each pub/sub message includes
the Slips version and consumers ignore messages that belong to the updated version and only read msgs intended
for them.


#### Zeek log handling

The new Slips starts a new zeek process that uses new zeek log files in ```output/zeek_files/slips_vx.y.z```.
This ensures that the old zeek logs are not modified, re-read or overwritten during the update

This was a major simplification because sharing Zeek logs between versions
introduced complexity and race conditions.


### Draining and Shutdown of the old Slips

Once the updated Slips is confirmed to be running:

the old Slips begins draining.

Draining means:

- stop ingesting new flows.
- finish processing pending flows.


PS: the new updated slips version starts reading flows before the old one starts draining to ensure 0 downtime.

## How to use it

enable ```auto_update_slips``` in ```config/slips.yaml``` and run slips on your interface.

now whenever a new version of Slips is available, it will update itself and the new slips will use the same CLI as the old one.

## PR

https://github.com/stratosphereips/StratosphereLinuxIPS/pull/1915
2 changes: 1 addition & 1 deletion docs/iris_module.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ will be left upon the future developers.
* ```go test ./...```

### Integration Testing
Integration tests are located in ```tests/integration/test_iris.py```.
Integration tests are located in ```tests/integration/test_iris/test_iris.py```.

### Test Messaging
The scenario that was modeled in this test refers to a common use case.
Expand Down
13 changes: 13 additions & 0 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -529,6 +529,19 @@ Use ```permanent_dir``` to choose where Slips stores databases and runtime-gener

This includes persistent artifacts such as ```p2p_trust_runtime/``` and shared module databases like the Fides cache.

**Live Slips auto update**

Use ```update.auto_update_slips``` to enable or disable automatic live updates of the installed Slips version.

```yaml
update:
auto_update_slips: false
```

This setting is separate from the runtime ```feeds_update_manager``` module, which only updates TI feeds and related files.

Automatic Slips updates may overwrite the default config files shipped with Slips. If you want to keep local config changes safe, do not modify the default config files. Create and use your own config files with different names instead.



<div class="zoom">
Expand Down
8 changes: 4 additions & 4 deletions managers/metadata_manager.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# SPDX-FileCopyrightText: 2021 Sebastian Garcia <sebastian.garcia@agents.fel.cvut.cz>
import subprocess

# SPDX-License-Identifier: GPL-2.0-only
import subprocess
import psutil
import sys
import os
Expand Down Expand Up @@ -121,8 +120,9 @@ def set_input_metadata(self):
if self.main.args.interface:
info.update({"interface": self.main.args.interface})

if hasattr(self.main, "zeek_dir"):
info.update({"zeek_dir": self.main.zeek_dir})
zeek_dir = self.main.db.get_zeek_output_dir()
if isinstance(zeek_dir, str) and zeek_dir:
info.update({"zeek_dir": zeek_dir})

if hasattr(self.main, "zeek_bro") and self.main.zeek_bro:
info.update({"zeek_version": self.get_zeek_version()})
Expand Down
Loading
Loading