[feature] Implement Iperf3 check #385 #394

Aryamanz29 · 2022-06-06T09:25:46Z

Added initial code for Iperf3 check class.
Added tests for check class.
Added device connection logic.
Added charts & metric (TCP & UDP mode).

`Steps to test`

Make sure your client (openwrt-device) and server both have Iperf3, Openwrt Iperf3 package installed.
Do check the credential section of the device (It must be enabled and working with the right update strategy, i.e., ssh).

In tests/openwisp2/settings.py configure:

OPENWISP_MONITORING_IPERF3_SERVERS = {
    # Running on my local
    # Some Public Iperf3 Servers : https://iperf.fr/iperf-servers.php#public-servers
    # 'be63c4e5-a68a-4650-bfe8-733837edb8be': ['iperf.biznetnetworks.com'],
    'a9734710-db30-46b0-a2fc-01f01046fe4f': ['speedtest.uztelecom.uz'],
    # '<org-pk>': ['<ORG_IPERF3_SERVER>']
}

NOTE : The host can be specified by hostname, IPv4 literal, or IPv6 literal

# for ex
              iperf3 -c iperf.biznetnetworks.com

              iperf3 -c 192.168.5.109

              iperf3 -c 2001:db8::1

Run python3 manage.py run_checks (To run all checks manually).

`Demo`

iperf_demo.mp4

`TCP Charts`

`UDP Charts`

Closes #385

coveralls · 2022-06-06T09:43:09Z

Coverage remained the same at 98.709% when pulling 9213aaf on issue-385/iperf-check into e563520 on gsoc22-iperf.

nemesifier

I changed the target branch to gsoc22-iperf. Please let's use this from now on.

CC: @pandafy @okraits.

pandafy

Great work @Aryamanz29! 👏🏼 👏🏼

@nemesisdesign @okraits things that I have tested manually:

The iperf check is created automatically when a new device registers
The iperf check is not created if OPENWISP_MONITORING_AUTO_IPERF is set to False.
The iperf check execute for a device having an active SSH connection.
The iperf check does not execute for a device whose SSH connection is in not_working state
The charts are shown for the gathered data.

Edge cases to handle

The database reports that DeviceConnection is working, but the management tunnel goes down just before iperf check was executed. The code should be able to gracefully handle this and also notify users that SSH connection is not working. We already have a mechanism for sending notifications. @Aryamanz29 please check other checks for reference and ask us your doubts.

Things to discuss in next weekly meeting

What parameter should trigger alert for iperf check?
Which data should be shown to users with charts?

openwisp_monitoring/check/settings.py

openwisp_monitoring/check/classes/iperf.py

openwisp_monitoring/tests/test_selenium.py

tests/openwisp2/settings.py

Aryamanz29 · 2022-06-23T10:21:08Z

@pandafy In 8823581 I have fixed #399, Now you can see below that error is longer produced because earlier we're executing command without considering the device_connection working status just right after connect() method.

One more thing, do we really need to notify users for this event? I mean when I'm testing it locally, the "SSH not working" alert already popped out & it already fulfilled our purpose, what do you think? Although, I have added logger.warn() which is specific for the check.

pandafy · 2022-06-23T10:46:25Z

One more thing, do we really need to notify users for this event? I mean when I'm testing it locally, the "SSH not working" alert already popped out & it already fulfilled our purpose, what do you think? Although, I have added logger.warn() which is specific for the check.

As the user is already getting one notification for SSH not working, we don't need to create another notification. Logging a warning is sufficient.

I will do manual testing later today.

pandafy

@Aryamanz29 I did a manual test of the changes and I don't see an error for #399. I will do an in-depth code review tomorrow.

Good work @Aryamanz29! 👏🏼

pandafy

@nemesisdesign @okraits I need some clarity on how we are planning to store the data in the InfluxDB. The current implementation ignores the bitrate from the UDP test, which seems wrong to me.

Thinking out loud
A good argument could be "because UDP bitrate would be similar to TCP bitrate so we can ignore that". If so, then why bother doing a UDP test? Is it only for getting jitter and packet loss?

openwisp_monitoring/check/classes/iperf.py

openwisp_monitoring/db/backends/influxdb/queries.py

tests/openwisp2/settings.py

openwisp_monitoring/check/classes/iperf.py

- Added ping config checks in celery beat configuration. - If args is None executes all checks.

- Added username, password, rsa-public-key-path parameter to DEFAULT_IPERF_CHECK_CONFIG. Closes #414

- Changed iperf auth logic (RSA key in params instead of path). - Added IPERF_CHECK_RSA_KEY_DELETE (If true RSA will be delete from the device). - Added IPERF_CHECK_RSA_KEY_PATH (To set default iperf RSA public key). - Impoved existing tests as per review suggestion.

- Connected data points with new bar+lines chart type. - Added fill(0) to the chart query to fill blank data points(ie.N/A). Closes #417

- Combined all iperf settings into one setting. - Removed default from OPENWISP_MONITORING_IPERF_CHECK_CONFIG - The order of getting params for iperf check is: check_params(through admin panel)-> OPENWISP_MONITORING_IPERF_CHECK_CONFIG-> DEFAULT_IPERF_CHECK_CONFIG - Updated iperf tests. Closes #418

- Added bitrate param to tcp and udp mode. - Improved iperf tests. - Improved alert_on_related_field tests.

…ling

- Found that options.mode = 'lines+markers' for the stackedbar+lines charts which streches charts x-axis to some backward time we can use options.mode = 'lines' for iperf charts to avoid this. Fixes #422

- Combined transfer and bandwidth charts. - improved docs.

…_CONFIG

- Removed alert_on_related feature from this PR.

pandafy added this to In progress in [GSoC 22] Iperf3 check via automation Jun 6, 2022

nemesifier changed the base branch from master to gsoc22-iperf June 6, 2022 23:36

nemesifier reviewed Jun 6, 2022

View reviewed changes

Aryamanz29 self-assigned this Jun 10, 2022

Aryamanz29 added the enhancement New feature or request label Jun 10, 2022

Aryamanz29 force-pushed the issue-385/iperf-check branch 4 times, most recently from 335cf4f to bad881d Compare June 14, 2022 07:46

Aryamanz29 force-pushed the issue-385/iperf-check branch from f968f79 to 2b474ae Compare June 20, 2022 13:22

Aryamanz29 requested review from nemesifier, pandafy and okraits June 20, 2022 13:25

pandafy requested changes Jun 21, 2022

View reviewed changes

[GSoC 22] Iperf3 check automation moved this from In progress to Review in progress Jun 21, 2022

Aryamanz29 mentioned this pull request Jun 21, 2022

[bug] Management tunnel goes down just before iperf3 check was executed #399

Closed

Aryamanz29 force-pushed the issue-385/iperf-check branch 5 times, most recently from b535521 to 016ea9f Compare June 23, 2022 10:04

Aryamanz29 force-pushed the issue-385/iperf-check branch from 016ea9f to 8823581 Compare June 23, 2022 10:23

pandafy reviewed Jun 23, 2022

View reviewed changes

pandafy reviewed Jun 24, 2022

View reviewed changes

Aryamanz29 force-pushed the issue-385/iperf-check branch 3 times, most recently from 4bf1fd9 to 1d94482 Compare June 24, 2022 12:59

Aryamanz29 and others added 23 commits August 9, 2022 17:11

[requested-changes] Explicitly specify ping & config checks

02100af

- Added ping config checks in celery beat configuration. - If args is None executes all checks.

[docs] Moved images to docs/1.1

27732e6

[feature] Added support for iperf3 auth #414

8a9b826

- Added username, password, rsa-public-key-path parameter to DEFAULT_IPERF_CHECK_CONFIG. Closes #414

[tests] Added tests for iperf auth

fb1bc26

[docs] Added iperf auth

fa74e99

[change] Changed chart types for iperf check #417

1d71d7c

- Connected data points with new bar+lines chart type. - Added fill(0) to the chart query to fill blank data points(ie.N/A). Closes #417

[change] Changed chart look according to feedback

50256d2

[feature] Added --bitrate param to iperf check

c7b6cb1

- Added bitrate param to tcp and udp mode. - Improved iperf tests. - Improved alert_on_related_field tests.

[change] Added validation for settings config and non json error hand…

46d86f0

…ling

[change] Added connect_points to iperf charts

f9a4f98

[change] Docs & iperf charts improved

1e4196b

[feature] Added adaptive_prefix to monitoring charts

c4ce4a0

[requested-changes] Docs and minor changes

719932e

[fix] Fixed iperf bar charts x-axis incosistency #422

631d2bf

- Found that options.mode = 'lines+markers' for the stackedbar+lines charts which streches charts x-axis to some backward time we can use options.mode = 'lines' for iperf charts to avoid this. Fixes #422

[requested-changes] Fixed chart.js linting

3524a68

[requested-changes] Combine iperf charts & improved docs

abe40f8

- Combined transfer and bandwidth charts. - improved docs.

[requested-changes] Moved rsa_public_key_path to OPENWISP_IPERF_CHECK…

b12d9b5

…_CONFIG

[requested-changes] Removed alert_on_related field

e0e23d9

- Removed alert_on_related feature from this PR.

[requested-changes] Removed rsa_public_key_path from settings

13b6416

[docs] Improved README

1b1dd84

[requested-changes] Added iperf to available features

9213aaf

Aryamanz29 force-pushed the issue-385/iperf-check branch from 5f4c51f to 9213aaf Compare August 9, 2022 11:42

nemesifier approved these changes Aug 9, 2022

View reviewed changes

[GSoC 22] Iperf3 check automation moved this from Review in progress to Reviewer approved Aug 9, 2022

nemesifier merged commit dc0316a into gsoc22-iperf Aug 9, 2022

nemesifier deleted the issue-385/iperf-check branch August 9, 2022 12:39

[GSoC 22] Iperf3 check automation moved this from Reviewer approved to Done Aug 9, 2022

Aryamanz29 changed the title ~~[feature] Implement Iperf check #385~~ [feature] Implement Iperf3 check #385 Oct 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feature] Implement Iperf3 check #385 #394

[feature] Implement Iperf3 check #385 #394

Aryamanz29 commented Jun 6, 2022 •

edited

Loading

coveralls commented Jun 6, 2022 •

edited

Loading

nemesifier left a comment

pandafy left a comment

Aryamanz29 commented Jun 23, 2022 •

edited

Loading

pandafy commented Jun 23, 2022

pandafy left a comment

pandafy left a comment

[feature] Implement Iperf3 check #385 #394

[feature] Implement Iperf3 check #385 #394

Conversation

Aryamanz29 commented Jun 6, 2022 • edited Loading

Steps to test

Demo

TCP Charts

UDP Charts

coveralls commented Jun 6, 2022 • edited Loading

nemesifier left a comment

Choose a reason for hiding this comment

pandafy left a comment

Choose a reason for hiding this comment

Aryamanz29 commented Jun 23, 2022 • edited Loading

pandafy commented Jun 23, 2022

pandafy left a comment

Choose a reason for hiding this comment

pandafy left a comment

Choose a reason for hiding this comment

Aryamanz29 commented Jun 6, 2022 •

edited

Loading

`Steps to test`

`Demo`

`TCP Charts`

`UDP Charts`

coveralls commented Jun 6, 2022 •

edited

Loading

Aryamanz29 commented Jun 23, 2022 •

edited

Loading