Software: Sunray v2023.9.0-beta.0

[ethanjli]

v2023.9.0 will be our first software update in over a year and a half, and we’re excited to make it easier for you to use your PlanktoScope in this update and in future updates! The v2023.9.0 software release focuses on improving the software infrastructure of the PlanktoScope software distro in preparation for future software improvements and maintenance, and making various usability improvements.

This release’s name is “Sunray” (named after the Sunray Venus clam). We will keep this name for future versions until the next release which introduces big changes for users.

Planning, management, and maintenance of the v2023.9.0 software release is led by Ethan Li (@ethanjli), with approval and oversight from Thibaut Pollina (@tpollina). If this software release causes new problems for how you are using PlanktoScope, Ethan will take responsibility for addressing any new issues caused by this release - just tag them in a new Github Discussions post or in the #6-dev-software channel on the PlanktoScope Slack workspace!

About this pre-release

v2023.9.0-beta.0 is a pre-release testing version for you to test out; it probably has software bugs and other problems, so you should not use it for collecting scientific data - expect bugs and software breakage! We would greatly appreciate it if you could flash the v2023.9.0-beta.0 image to an SD card and let us know about any bugs or problems you encounter while trying to use your PlanktoScope. This will help us to deliver a higher-quality software release with our v2023.9.0 release.

If you decide to try out v2023.9.0-beta.0, please file bug reports on GitHub Issues and/or let us know about any bugs you discover in the #6-dev-software channel on the PlanktoScope Slack workspace!

Because this pre-release includes many changes, the changelogs below only list items we think users are likely to notice or to need to know about; for a full list of changes, please refer to software/CHANGELOG.md.

Changelog since v2.3

Highlights:

• To use the PlanktoScope from your web browser, now you should just open http://home.planktoscope/ or http://planktoscope.local/ or http://192.168.4.1/.
• You no longer need to use FileZilla to download and delete the datasets on your PlanktoScope - now you can do it from your web browser.
• The PlanktoScope now acts like a router, so if it has internet access then it will share that internet access with your computer/phone.
• PlanktoScopes will no longer stay connected to any Wi-Fi network which doesn’t provide internet access.
• PlanktoScopes now use a new naming scheme for machine names.
• We will eventually transition the Node-RED dashboard for the v2.1 hardware (Adafruit HAT) into maintenance mode.
• The Node-RED dashboard will no longer try to use incorrect URLs to show the camera preview stream.
• Entering invalid white balance values in the Node-RED dashboard should no longer break the PlanktoScope software upon restart.

Added:

• The PlanktoScope can now be accessed from your web browser at http://home.planktoscope/ (assuming your web browser isn’t configured to use a custom DNS server such as 1.1.1.1 or 8.8.8.8), in addition to the other URLs which previously worked (http://planktoscope.local/, http://192.168.4.1/, and http://192.168.5.1/).
  • Each PlanktoScope can now also be accessed at machine-specific URLs of format http://<machine-name>.planktoscope and http://planktoscope-<machine-name>.local, where <machine-name> is replaced with your PlanktoScope’s machine name (see the note in the “Changed” section below about machine names).
  • If your web browser tries to do a Google search for the URL instead of actually opening it as a webpage, make sure you type http:// at the start or / at the end (e.g. home.planktoscope/ or http://home.planktoscope/ instead of home.planktoscope)
  • If you connect directly from your computer/phone to the PlanktoScope and then use your web browser to open http://home.planktoscope/, http://<machine-name>.planktoscope, http://planktoscope-<machine-name>.local, http://planktoscope.local/, http://192.168.4.1/, or http://192.168.5.1/, you will now see a landing page with a list of useful links to software running on the PlanktoScope.
• The PlanktoScope now shares any internet access it has from Wi-Fi with all devices connected to its Ethernet port; and it also shares any internet access it has from Ethernet with all devices connected to its Wi-Fi hotspot, if it’s operating in wireless AP mode.
• An offline version of the PlanktoScope documentation is now available. It's accessible by a link from the PlanktoScope's landing page. The online version of the documentation is now at https://planktoscope.github.io/PlanktoScope/
  • The PlanktoScope documentation now includes the reference protocol for quantitative imaging with PlanktoScope, which is normally available at https://www.protocols.io/view/planktoscope-protocol-for-plankton-imaging-bp2l6bq3zgqe/v2 . You can access the protocol from the bottom of the “Usage” → “Getting Started” page of the PlanktoScope documentation.
  • The PlanktoScope documentation pages have been redone in a new system, and improved in various ways. The hardware documentation now describes the v2.5 PlanktoScope hardware. Thanks to Sebastian Wendel (@sourceindex) for starting this work, and for contributing lots of documentation!

Changed:

• The Node-RED dashboard (previously available at http://planktoscope.local:1880/ui) has been moved to a different URL, accessible by a link on the new landing page (see the note in the “Added” section about new URLs and the landing page).
• The file gallery (previously available at http://planktoscope.local/) has now been renamed to the dataset file manager and moved to a different URL, accessible by a link on the new landing page. It now allows you to delete folders, preview images, edit text files, download folders as ZIP files, rename folders, and perform other common tasks.
  • This way, you no longer have to install FileZilla in order to copy or delete any of your datasets!
• Previously, PlanktoScope machine names were generated as gibberish words like "Babaxio-Detuiau", and the machine names were used as the names of the Wi-Fi hotspot networks made by the PlanktoScope. However, the machine names created by this naming scheme were often difficult to pronounce, remember, and type for people in various languages, and the naming scheme sometimes generated names which sounded like curses or insults in some languages. Now, PlanktoScope machine names are generated as a combination of two words and a number up to five digits long; words are selected from pre-built lists in a language which can be chosen based on localization settings. Currently, word lists are only provided in US English, resulting in names like "metal-slope-23501", "conscious-pocket-1684", and "plant-range-10581"; however, word lists can be added for other languages in the future, and a user interface will eventually be provided for changing localization settings.
  • This change makes it easier for you to tell someone else which PlanktoScope to connect to if multiple PlanktoScope Wi-Fi hotspot networks are available nearby.
  • If you know a language besides US English and you’d like to help us add support for generating machine names in that language, please start a discussion on Github or in the #6-dev-software channel on the PlanktoScope Slack workspace!
• Previously, the PlanktoScope would stay connected to a wifi network even if it didn’t have internet access on that network. Now, if it connects to a Wi-Fi network but can’t get internet access on that network (specifically, if it’s unable to find google.com), then it will revert to creating its own Wi-Fi hotspot for your computer or phone to connect to.

Deprecated:

• In a future release (timeline not yet decided), the version of the Node-RED dashboard for the Adafruit HAT (hardware v2.1) will stop receiving new features even as the version of the Node-RED dashboard for the custom PlanktoScope HAT (hardware v2.3+) continues receiving new features. However, we will continue to fix bugs in the Node-RED dashboard for the Adafruit HAT, and we will continue to build SD card images for the Adafruit HAT which will also include new features in other software components.

Fixed:

• When an invalid value is entered for the red or blue white balance gain on the Node-RED dashboard's "Optic Configuration" page, that value is now ignored, a notification is displayed about the invalid value, and the white balance gain is reset to the last valid value (loaded from the hardware.json configuration file). Previously, invalid white balance values were written to the hardware.json configuration file, which would cause the PlanktoScope software to crash after every subsequent restart until the hardware.json file was manually removed or fixed.
  • Thanks to Gilles Orazi (@glx314) for filing an issue about this problem!
• In various use cases (e.g. when accessing the PlanktoScope over a mesh VPN, or in some bug reports by users accessing the PlanktoScope over Ethernet), the Node-RED dashboard would be unable to show the MJPEG streams of the camera preview and the object segmentation preview, resulting in a potentially misleading error message being shown about the Python scripts. Now, the Node-RED dashboard uses URLs for the MJPEG streams which we expect to be accessible wherever the Node-RED dashboard itself is accessible.
  • Thanks to Rodrigo Gonçalves on the PlanktoScope Slack for reporting a bug related to this issue and helping us to troubleshoot it!
• The adafruit-blinka and adafruit-platformdetect software dependencies are now updated to their latest version. As a result, the PlanktoScope software should not crash just because it is being used with recent versions of the Adafruit Stepper HAT.
  • This fix removes the need for you to give your PlanktoScope internet access and then connect to the PlanktoScope over SSH to manually run the pip3 install --upgrade adafruit-blinka adafruit-platformdetect command before your PlanktoScope can work properly.

Changelog since v2023.9.0-alpha.0

Highlights:

• You can now access offline copies of the quantitative imaging protocols for the PlanktoScope.
• We will eventually transition the Node-RED dashboard for the v2.1 hardware (Adafruit HAT) into maintenance mode.
• The Node-RED dashboard will no longer try to use incorrect URLs to show the camera preview stream.
• Entering invalid white balance values in the Node-RED dashboard should no longer break the PlanktoScope software upon restart.

Added:

• The PlanktoScope documentation now includes the reference protocol for quantitative imaging with PlanktoScope, which is normally available at https://www.protocols.io/view/planktoscope-protocol-for-plankton-imaging-bp2l6bq3zgqe/v2. You can access the protocol from the bottom of the “Usage” → “Getting Started” page of the PlanktoScope documentation.

Deprecated:

• In a future release (timeline not yet decided), the version of the Node-RED dashboard for the Adafruit HAT (hardware v2.1) will stop receiving new features even as the version of the Node-RED dashboard for the custom PlanktoScope HAT (hardware v2.3+) continues receiving new features. However, we will continue to fix bugs in the Node-RED dashboard for the Adafruit HAT, and we will continue to build SD card images for the Adafruit HAT which will also include new features in other software components.

Fixed:

• When an invalid value is entered for the red or blue white balance gain on the Node-RED dashboard's "Optic Configuration" page, that value is now ignored, a notification is displayed about the invalid value, and the white balance gain is reset to the last valid value (loaded from the hardware.json configuration file).
  • Thanks to Gilles Orazi (@glx314) for filing an issue about this problem!
• In various use cases (e.g. when accessing the PlanktoScope over a mesh VPN, or in some bug reports by users accessing the PlanktoScope over Ethernet), the Node-RED dashboard would be unable to the MJPEG streams of the camera preview and the object segmentation preview, resulting in a potentially misleading error message being shown about the Python scripts. Now, the Node-RED dashboard uses URLs for the MJPEG streams which we expect to be accessible wherever the Node-RED dashboard itself is accessible.
  • Thanks to Rodrigo Gonçalves on the PlanktoScope Slack for reporting a bug related to this issue and helping us to troubleshoot it!

Known Issues:

• The Node-RED dashboard’s “System Monitoring” page has an “Information” panel whose “Instrument Type” and “Camera Name” fields are often empty.

Pull Requests

• Properly manage versions of the backend's direct & indirect dependencies by @ethanjli in Properly manage versions of the backend's direct & indirect dependencies #175
• Fix docs links, improve software setup & usage docs by @ethanjli in Fix docs links, improve software setup & usage docs #176
• build(deps-dev): bump poethepoet from 0.21.1 to 0.22.0 in /documentation by @dependabot in build(deps-dev): bump poethepoet from 0.21.1 to 0.22.0 in /documentation #178
• build(deps-dev): bump mkdocs-material from 9.2.0 to 9.2.1 in /documentation by @dependabot in build(deps-dev): bump mkdocs-material from 9.2.0 to 9.2.1 in /documentation #179
• Simplify the Docker image tag used for pushed Git tags by @ethanjli in Simplify the Docker image tag used for pushed Git tags #180
• Upgrade forklift to v0.3.0 (replacing Docker Swarm Mode with Docker Compose) by @ethanjli in Upgrade forklift to v0.3.0 (replacing Docker Swarm Mode with Docker Compose) #181
• Reduce the size of the build artifacts for the documentation site by @ethanjli in Reduce the size of the build artifacts for the documentation site #184
• Set favicon of docs to the PlanktoScope logo by @ethanjli in Set favicon of docs to the PlanktoScope logo #185
• Lock all dependencies of the Node-RED dashboard by @ethanjli in Lock all dependencies of the Node-RED dashboard #186
• Unify support for different HAT types into the same branch of the repo by @ethanjli in Unify support for different HAT types into the same branch of the repo #187
• Update the "Hardware Settings" and "System Monitoring" dashboard pages by @ethanjli in Update the "Hardware Settings" and "System Monitoring" dashboard pages #191
• Update setup scripts and bump version numbers for v2023.9.0-beta.0 by @ethanjli in Update setup scripts and bump version numbers for v2023.9.0-beta.0 #196

Full Changelog: software/v2023.9.0-alpha.0...software/v2023.9.0-beta.0

---

This discussion was created from the release Software: Sunray v2023.9.0-beta.0.