Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement PowerMon #4136

Closed
geeksville opened this issue Jun 18, 2024 · 0 comments · Fixed by #4155
Closed

Implement PowerMon #4136

geeksville opened this issue Jun 18, 2024 · 0 comments · Fixed by #4155
Assignees

Comments

@geeksville
Copy link
Member

geeksville commented Jun 18, 2024

PowerMon

(eventually I'll move this someplace better - but placing it here for feedback from other meshtastic devs)

  • different PM states can either be on or off.
  • changes to PM states are logged for eventual CVS annotation (with measured power consumption)
  • multiple PM on/off states can exist at a time.
  • the log messages will be short and complete (see PowerMon.Event in the protobufs for details). something like "PwrMon,C,0x00001234,REASON" where the hex number is the bitmask of all current states. (We use a bitmask for states so that if a log message gets lost it won't be fatal)
  • a PwrMon message will be emitted for each state change (so an xor of old mask and new mask should indicate which changed)
  • If a time is known it will be included in the log message header, so there is no need to include time in PowerMon.Event
  • We serialize the messages as simple text to avoid any dependence on the meshtastic messaging layer (which we might be measuring with this)
  • PowerMon support can be included in the build via the -DUSE_POWERMON definition. If it is not included the class will still exist but the methods will be NOPs. So interested devs can easily add PowerMon tags as needed (with no runtime cost for users)
  • If -DUSE_POWERSTRESS is defined the board will automatically walk through a series of power changing actions in hopes of generating a standard/repeatable power report for that particular board type. Eventually: instead have the python code call send a message to the PowerStress class (so we don't need to build particular 'powerstress' builds). Similar to the GPIO by message access we already have (possibly with identical python glue).

On device API

PowerMon class

  • PowerMon.setState(PowerState.DeepSleep, "user pressed button")
  • PowerMon.clearState(PowerState.DeepSleep, "user released button)

Users should try to call setState/clearState just before/after and significant power consuming device changes. It is assumed that the python code will slightly delay after receiving the log message (100ms?) before capturing the power meter measurement.

PowerStress class

  • Uses timers to walk the node through a number of different states (using the existing state machine and message sending APIs).
  • Intended to generate a 'repeatable' powermon report that can be used eventually to prevent regressions in our power management.
  • While testing if we want to test RX performance without needing a faraday cage, add support for changing the lora framing byte to something non-standard. omg - I'm tired of random SF nodes talking to my test units ;-).

PowerMon states

Use protobuf enums to define these states, so that python and device code can share definitions. Any eng can add new states by just adding a new enum value.

  • PM0 CPU-DeepSleep
  • PM1 CPU-Sleep
  • PM2 CPU-Awake
  • PM3 Lora-RXOn
  • PM4 Lora-TXOn
  • PM5 Lora-RXActive
  • PM6 BT-On
  • PM7 LED-On
  • PM8 Screen-On
  • PM9 Screen-Drawing

etc...

Runtime report collection

  • The meshtastic python tool ('meshtastic --powermon') will collect all PM log messages and at the end of the run emit report csv files to powermon//foo.csv. The idea here is that you can just always leave this option set and the data files will go to 'the right place'.
  • If a power supply is connected to python it will set the power supply to the correct voltage for tests/usage.
  • If --powerstress is specified it will direct the unit under test (via meshtastic API messages) to enter different test configs for measurement. It will dump report files to powermon//stress/foo.csv. These files will be highly standardized so we can eventually detect regressions.
  • export the raw data as a csv intended for use in https://dash.plotly.com/minimal-app or excel or similiar
  • If a programmable bench supply is connected (initially only Riden supplies are supported), measured power will be included on all rows
  • If a programmable bench supply is connected we Will export a new record every 100ms even if there are no state changes (to capture measured power levels)

CSV report format

A row heading will be emitted based on the protobuf defs (so spreadsheet will have nice names)

Data rows

  • time (as decimal seconds, i.e. 4.44 or Unix epoch seconds)
  • measured power or emptystring
  • reason or emptystring
  • pm0 state
  • pm1 state
  • etc...

CSV summary report

Summary report (generated afterwards from that csv)

  • % time on for each of the different states
  • Somehow try to decouple average power consumption due to each PwrMon state. Perhaps:
    Have the python code look 200msish after a transition and do something like powerDueToState = currentPower - preStatePower
  • Somehow include deviation for each of these power per state estimates, so we can know if we need to capture longer runs.

Graph results

Possibly use excel, but probably plottly-express (i.e. python) to make a nice graph showing stacked bars for states and the total height of the bars be based on the measured power. Each state segment in the bar will be sized based on average power consumption attributed to that state alone.

Structured log messages

We have defined a mini standard for log messages that are intended for machine parsing. We do this in the hopes that these messages will remain mostly stable for use over a long time (i.e. old logs can be compared to new for regression testing).

The standard designed to be short and easily machine parsable. The format for these messages must be:

S:SUBSYS:arg1,arg2,arg3,...

Where "S:" is a string constant prefix always used on such messages. Current defined subsystems:

  • B: Board info boot message (contains hw model code and sw version)
  • PM: PowerMon

Current notes-to-self TODO:

  • DONE use config flag to control powermon
  • DONE refactor the python power meter classes
  • DONE add powerstress framework for setting configs, cycling power
  • add default slog destination based on board type. auto start slogging when we see the board announcement
@geeksville geeksville self-assigned this Jun 18, 2024
geeksville added a commit to geeksville/Meshtastic-protobufs that referenced this issue Jun 18, 2024
geeksville added a commit to geeksville/Meshtastic-esp32 that referenced this issue Jun 18, 2024
geeksville added a commit to geeksville/Meshtastic-esp32 that referenced this issue Jun 18, 2024
geeksville added a commit to geeksville/Meshtastic-esp32 that referenced this issue Jun 19, 2024
geeksville added a commit to geeksville/Meshtastic-protobufs that referenced this issue Jun 21, 2024
geeksville added a commit to geeksville/Meshtastic-esp32 that referenced this issue Jun 21, 2024
geeksville added a commit to geeksville/Meshtastic-esp32 that referenced this issue Jun 21, 2024
geeksville added a commit to geeksville/Meshtastic-protobufs that referenced this issue Jun 21, 2024
geeksville added a commit to geeksville/Meshtastic-esp32 that referenced this issue Jun 21, 2024
geeksville added a commit to geeksville/Meshtastic-protobufs that referenced this issue Jun 21, 2024
geeksville added a commit to geeksville/Meshtastic-protobufs that referenced this issue Jun 21, 2024
geeksville added a commit to geeksville/Meshtastic-esp32 that referenced this issue Jun 30, 2024
…s...

Which is currently only tested with the LED but eventually
will be used for shared GPIO/screen power rail enable
and LED forcing (which is a sanity check in the power stress
testing)
geeksville added a commit to geeksville/Meshtastic-esp32 that referenced this issue Jun 30, 2024
…s...

Which is currently only tested with the LED but eventually
will be used for shared GPIO/screen power rail enable
and LED forcing (which is a sanity check in the power stress
testing)
thebentern pushed a commit that referenced this issue Jul 3, 2024
* Turn off vscode cmake prompt - we don't use cmake on meshtastic

* Add rak4631_dap variant for debugging with NanoDAP debug probe device.

* The rak device can also run freertos (which is underneath nrf52 arduino)

* Add semihosting support for nrf52840 devices
Initial platformio.ini file only supports rak4630
Default to non TCP for the semihosting log output for now...
Fixes #4135

* powermon WIP (for #4136 )

* oops - mean't to mark the _dbg variant as an 'extra' board.

* powermon wip

* Make serial port on wio-sdk-wm1110 board work
By disabling the (inaccessible) adafruit USB

* Instrument (radiolib only for now) lora for powermon
per #4136

* powermon gps support
#4136

* Add CPU deep and light sleep powermon states
#4136

* Change the board/swversion bootstring so it is a new "structured" log msg.

* powermon wip

* add example script for getting esp S3 debugging working
Not yet used but I didn't want these nasty tricks to get lost yet.

* Add PowerMon reporting for screen and bluetooth pwr.

* make power.powermon_enables config setting work.

* update to latest protobufs

* fix bogus shellcheck warning

* make powermon optional (but default enabled because tiny and no runtime impact)

* tell vscode, if formatting, use whatever our trunk formatter wants
without this flag if the user has set some other formatter (clang)
in their user level settings, it will be looking in the wrong directory
for the clang options (we want the options in .trunk/clang)

Note: formatOnSave is true in master, which means a bunch of our older
files are non compliant and if you edit them it will generate lots of
formatting related diffs.  I guess I'll start letting that happen with
my future commits ;-).

* add PowerStress module

* nrf52 arduino is built upon freertos, so let platformio debug it

* don't accidentally try to Segger ICE if we are using another ICE

* clean up RedirectablePrint::log so it doesn't have three very different implementations inline.

* remove NoopPrint - it is no longer needed

* when talking to API clients via serial, don't turn off log msgs instead encapsuate them

* fix the build - would loop forever if there were no files to send

* don't use Segger code if not talking to a Segger debugger

* when encapsulating logs, make sure the strings always has nul terminators

* nrf52 soft device will watchdog if you use ICE while BT on...
so have debugger disable bluetooth.

* Important to not print debug messages while writing to the toPhone scratch buffer

* don't include newlines if encapsulating log records as protobufs

* update to latest protobufs (needed for powermon goo)

* PowerStress WIP

* fix linter warning
@geeksville geeksville reopened this Jul 4, 2024
thebentern added a commit that referenced this issue Jul 8, 2024
* Fix protobuf structs handling (#4140)

* Fix protobuf structs handling

* Log instead of assert

---------

Co-authored-by: Ben Meadors <[email protected]>

* BLE based logging (#4146)

* WIP log characteristic

* Bluetooth logging plumbing

* Characteristic

* Callback

* Check for nullptr

* Esp32 bluetooth impl

* Formatting

* Add thread name and log level

* Add settings guard

* Remove comments

* Field name

* Fixes esp32

* Open it up

* Whoops

* Move va_end past our logic

* Use `upload_protocol = esptool` as with the other heltec devices instead of `esp-builtin` (#4151)

* Standardize lat/lon position logs (#4156)

* Standardize lat/lon position logs

* Missed sone and condensed logs

* [create-pull-request] automated change (#4157)

Co-authored-by: thebentern <[email protected]>

* Pause BLE logging during want_config flow (#4162)

* Update NimBLE to 1.4.2 (#4163)

* Implement replies for all telemetry types based on variant tag (#4164)

* Implement replies for all telemetry types based on variant tag

* Remove check for `ignoreRequest`: modules can set this, don't need to check

---------

Co-authored-by: Ben Meadors <[email protected]>

* Esptool is better

* Explicitly set characteristic

* fix INA3221 sensor (#4168)

- pass wire to begin()
- remove redundant setAddr() (already set in header)

* Show compass on waypoint frame; clear when waypoint deleted (#4116)

* Clear expired or deleted waypoint frame

* Return 0 to CallbackObserver

* Add a missing comment

* Draw compass for waypoint frame

* Display our own waypoints

* [create-pull-request] automated change (#4171)

Co-authored-by: thebentern <[email protected]>

* Add semihosting support for nrf52 devices (#4137)

* Turn off vscode cmake prompt - we don't use cmake on meshtastic

* Add rak4631_dap variant for debugging with NanoDAP debug probe device.

* The rak device can also run freertos (which is underneath nrf52 arduino)

* Add semihosting support for nrf52840 devices
Initial platformio.ini file only supports rak4630
Default to non TCP for the semihosting log output for now...
Fixes #4135

* fix my botched merge - keep board_level = extra flag for rak3631_dbg

---------

Co-authored-by: Ben Meadors <[email protected]>

* [create-pull-request] automated change (#4174)

Co-authored-by: thebentern <[email protected]>

* Display alerts (#4170)

* Move static functions into Screen.h, show compass during calibration

* Move to _fontHeight macro to avoid collision

* Move some alert functions to new alert handler

* Catch missed reboot code

* ESP32 fixes

* Bump esp8266-oled-ssd1306

* Fixes for when a device has no screen

* Use new startAlert(char*) helper class

* Add EINK bits back to alert handling

* Add noop class for no-display devices

---------

Co-authored-by: Ben Meadors <[email protected]>

* Send file system manifest up on want_config (#4176)

* Send file system manifest up on want_config

* Platform specific methods

* Helps to actually make the change

* Clear

* tell vscode, if formatting, use whatever our trunk formatter wants (#4186)

without this flag if the user has set some other formatter (clang)
in their user level settings, it will be looking in the wrong directory
for the clang options (we want the options in .trunk/clang)

Note: formatOnSave is true in master, which means a bunch of our older
files are non compliant and if you edit them it will generate lots of
formatting related diffs.  I guess I'll start letting that happen with
my future commits ;-).

* fix the build - would loop forever if there were no files to send (#4188)

* Show owner.short_name on boot (and E-Ink sleep screen) (#4134)

* Show owner.short_name on boot and sleep screen (on e-ink)

* Update Screen.cpp - new line for short_name

Boot screen short_name now below the region setting.
Looks better on small screens.

* Draw short_name on right

---------

Co-authored-by: Thomas Göttgens <[email protected]>
Co-authored-by: todd-herbert <[email protected]>
Co-authored-by: Ben Meadors <[email protected]>

* nrf52 soft device will watchdog if you use ICE while BT on... (#4189)

so have debugger disable bluetooth.

* correct xiao_ble build preventing sx1262 init (#4191)

* Force a compile time failur if FromRadio or ToRadio get larger than (#4190)

a BLE packet size. We are actually very close to this threshold so
important to make sure we don't accidentally pass it.

* Clear vector after complete config state (#4194)

* Clear after complete config

* Don't collect . entries

* Log file name and size

* [create-pull-request] automated change (#4200)

Co-authored-by: thebentern <[email protected]>

* Make the logs Colorful! (#4199)

* Squash needlessly static functions (#4183)

* Trim extra vprintf and filter for unprintable characters

* Deprecate Router Client role (and make it Client) (#4201)

* [create-pull-request] automated change (#4205)

Co-authored-by: thebentern <[email protected]>

* Move waypoint (#4202)

* Move waypoint screen draw into the waypoint module

* Get the observer set up for the waypoint screen draw

* Static squashing: screen dimensions
Macros moved back to Screen.cpp, as a band-aid until we eventually move all those static functions into the Screen class.

* Move getCompassDiam into Screen class
(supress compiler warnings)
At this stage, the method is still static, because it's used by drawNodeInfo, which has no tidy reference to our screen instance.
This is probably just another band-aid until these static functions all move.

* Use new getCompassDiam function in AccelerometerThread

* Properly gate display code in WaypointModule

---------

Co-authored-by: Todd Herbert <[email protected]>

* Fix flakey phone api transition from file manifest to complete (#4209)

* Try fix flakey phone api transition from file manifest to complete

* Skip

* enable colors in platformio serial monitor (#4217)

* When talking via serial, encapsulate log messages in protobufs if necessary (#4187)

* clean up RedirectablePrint::log so it doesn't have three very different implementations inline.

* remove NoopPrint - it is no longer needed

* when talking to API clients via serial, don't turn off log msgs instead encapsuate them

* fix the build - would loop forever if there were no files to send

* don't use Segger code if not talking to a Segger debugger

* when encapsulating logs, make sure the strings always has nul terminators

* nrf52 soft device will watchdog if you use ICE while BT on...
so have debugger disable bluetooth.

* Important to not print debug messages while writing to the toPhone scratch buffer

* don't include newlines if encapsulating log records as protobufs

---------

Co-authored-by: Ben Meadors <[email protected]>

* [create-pull-request] automated change (#4218)

Co-authored-by: thebentern <[email protected]>

* Fix SHT41 support (#4222)

* Add SHT41 Serial to I2c Detection Code

On the Seeed Wio-WM1110 Dev Kit board, the SHT41 chip was being
incorrectly detected as SHT31.

This patch adds the necessary serial number for the SHT41 chip to
be correctly detected.

fixes #4221

* Add missing sensor read for SHT41

* Typo fix in logs - mhz - MHz (#4225)

As reported by karamo, a few different places in our logs had
incorrect capitalization of MHz.

fixes #4126

* New new BLE logging characteristic with LogRecord protos  (#4220)

* New UUID

* New log radio characteristic with LogRecord protobuf

* LogRecord

* Merge derp

* How did you get there

* Trunk

* Fix length

* Remove assert

* minor cleanup proposal (#4169)

* MESHTASTIC_EXCLUDE_WIFI and HAS_WIFI cleanup...
Our code was checking HAS_WIFI and the new MESHTASTIC_EXCLUDE_WIFI
flags in various places (even though EXCLUDE_WIFI forces HAS_WIFI
to 0).  Instead just check HAS_WIFI, only use EXCLUDE_WIFI inside
configuration.h

* cleanup: use HAS_NETWORKING instead of HAS_WIFI || HAS_ETHERNET
We already had HAS_NETWORKING as flag in MQTT to mean 'we have
tcpip'.  Generallize that and move it into configuration.h so that
we can use it elsewhere.

* Use #pragma once, because supported by gcc and all modern compilers
instead of #ifdef DOTHFILE_H etc...

---------

Co-authored-by: Jonathan Bennett <[email protected]>

* Add PowerMon support (#4155)

* Turn off vscode cmake prompt - we don't use cmake on meshtastic

* Add rak4631_dap variant for debugging with NanoDAP debug probe device.

* The rak device can also run freertos (which is underneath nrf52 arduino)

* Add semihosting support for nrf52840 devices
Initial platformio.ini file only supports rak4630
Default to non TCP for the semihosting log output for now...
Fixes #4135

* powermon WIP (for #4136 )

* oops - mean't to mark the _dbg variant as an 'extra' board.

* powermon wip

* Make serial port on wio-sdk-wm1110 board work
By disabling the (inaccessible) adafruit USB

* Instrument (radiolib only for now) lora for powermon
per #4136

* powermon gps support
#4136

* Add CPU deep and light sleep powermon states
#4136

* Change the board/swversion bootstring so it is a new "structured" log msg.

* powermon wip

* add example script for getting esp S3 debugging working
Not yet used but I didn't want these nasty tricks to get lost yet.

* Add PowerMon reporting for screen and bluetooth pwr.

* make power.powermon_enables config setting work.

* update to latest protobufs

* fix bogus shellcheck warning

* make powermon optional (but default enabled because tiny and no runtime impact)

* tell vscode, if formatting, use whatever our trunk formatter wants
without this flag if the user has set some other formatter (clang)
in their user level settings, it will be looking in the wrong directory
for the clang options (we want the options in .trunk/clang)

Note: formatOnSave is true in master, which means a bunch of our older
files are non compliant and if you edit them it will generate lots of
formatting related diffs.  I guess I'll start letting that happen with
my future commits ;-).

* add PowerStress module

* nrf52 arduino is built upon freertos, so let platformio debug it

* don't accidentally try to Segger ICE if we are using another ICE

* clean up RedirectablePrint::log so it doesn't have three very different implementations inline.

* remove NoopPrint - it is no longer needed

* when talking to API clients via serial, don't turn off log msgs instead encapsuate them

* fix the build - would loop forever if there were no files to send

* don't use Segger code if not talking to a Segger debugger

* when encapsulating logs, make sure the strings always has nul terminators

* nrf52 soft device will watchdog if you use ICE while BT on...
so have debugger disable bluetooth.

* Important to not print debug messages while writing to the toPhone scratch buffer

* don't include newlines if encapsulating log records as protobufs

* update to latest protobufs (needed for powermon goo)

* PowerStress WIP

* fix linter warning

* Cleanup buffer

* Merge hex for wm1110 target(s)

* Only sdk

* Sudo

* Fix exclude macros (#4233)

* fix MESHTASTIC_EXCLUDE_BLUETOOTH

* fix HAS_SCREEN=0

* fix MESHTASTIC_EXCLUDE_GPS

* fix typo in build-nrf52.sh (#4231)

chmod is the command, '+x' is the argument.

* Tidy Wireless Paper variant files (#4238)

* Quick tidy of pins_arduino.h
Matches requests made at #4226 (comment))

* Tidy variant.h

* Change deprecated ADC attenuation parameter
From 11dB to 12dB. Resolves compiler warning. Allegly, no impact on function: `This is deprecated, it behaves the same as `ADC_ATTEN_DB_12`

* Updated raspbian CI to update apt repository ahead of libbluetooth. (#4243)

* Fix BLE logging on nrf52 (#4244)

* allow ble logrecords to be fetched either by NOTIFY or INDICATE ble types

This allows 'lossless' log reading.  If client has requested INDICATE
(rather than NOTIFY) each log record emitted via log() will have to fetched
by the client device before the meshtastic node can continue.

* Fix serious problem with nrf52 BLE logging.
When doing notifies of LogRecords it is important to use the
binary write routines - writing using the 'string' write won't work.
Because protobufs can contain \0 nuls inside of them which if being
parsed as a string will cause only a portion of the protobuf to be sent.
I noticed this because some log messages were not getting through.

---------

Co-authored-by: Ben Meadors <[email protected]>

* Fix build when HAS_NETWORKING is false on nrf52 (#4237)

(tested on a rak4631 by setting HAS_ETHERNET false when shrinking
image)

* If `toPhoneQueue` is full, still increment `fromNum` to avoid client never getting packets (#4246)

* Update to SoftDevice 7.3.0 for wio-sdk-wm1110 and wio-tracker-wm1110 (#4248)

* Update variant.h

* Update wio-tracker-wm1110.json

* Update wio-sdk-wm1110.json

* Update platformio.ini

* Update platformio.ini

* Add files via upload

* Add files via upload

* Update variant.h

---------

Co-authored-by: Mike <[email protected]>
Co-authored-by: Ben Meadors <[email protected]>
Co-authored-by: Mike G <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: thebentern <[email protected]>
Co-authored-by: GUVWAF <[email protected]>
Co-authored-by: Warren Guy <[email protected]>
Co-authored-by: todd-herbert <[email protected]>
Co-authored-by: geeksville <[email protected]>
Co-authored-by: Jonathan Bennett <[email protected]>
Co-authored-by: Alexander <[email protected]>
Co-authored-by: Thomas Göttgens <[email protected]>
Co-authored-by: quimnut <[email protected]>
Co-authored-by: Manuel <[email protected]>
Co-authored-by: Agent Blu, 006 <[email protected]>
Co-authored-by: Mark Trevor Birss <[email protected]>
thebentern added a commit that referenced this issue Jul 14, 2024
* Fix protobuf structs handling (#4140)

* Fix protobuf structs handling

* Log instead of assert

---------

Co-authored-by: Ben Meadors <[email protected]>

* BLE based logging (#4146)

* WIP log characteristic

* Bluetooth logging plumbing

* Characteristic

* Callback

* Check for nullptr

* Esp32 bluetooth impl

* Formatting

* Add thread name and log level

* Add settings guard

* Remove comments

* Field name

* Fixes esp32

* Open it up

* Whoops

* Move va_end past our logic

* Use `upload_protocol = esptool` as with the other heltec devices instead of `esp-builtin` (#4151)

* Standardize lat/lon position logs (#4156)

* Standardize lat/lon position logs

* Missed sone and condensed logs

* [create-pull-request] automated change (#4157)

Co-authored-by: thebentern <[email protected]>

* Pause BLE logging during want_config flow (#4162)

* Update NimBLE to 1.4.2 (#4163)

* Implement replies for all telemetry types based on variant tag (#4164)

* Implement replies for all telemetry types based on variant tag

* Remove check for `ignoreRequest`: modules can set this, don't need to check

---------

Co-authored-by: Ben Meadors <[email protected]>

* Esptool is better

* Explicitly set characteristic

* fix INA3221 sensor (#4168)

- pass wire to begin()
- remove redundant setAddr() (already set in header)

* Show compass on waypoint frame; clear when waypoint deleted (#4116)

* Clear expired or deleted waypoint frame

* Return 0 to CallbackObserver

* Add a missing comment

* Draw compass for waypoint frame

* Display our own waypoints

* [create-pull-request] automated change (#4171)

Co-authored-by: thebentern <[email protected]>

* Add semihosting support for nrf52 devices (#4137)

* Turn off vscode cmake prompt - we don't use cmake on meshtastic

* Add rak4631_dap variant for debugging with NanoDAP debug probe device.

* The rak device can also run freertos (which is underneath nrf52 arduino)

* Add semihosting support for nrf52840 devices
Initial platformio.ini file only supports rak4630
Default to non TCP for the semihosting log output for now...
Fixes #4135

* fix my botched merge - keep board_level = extra flag for rak3631_dbg

---------

Co-authored-by: Ben Meadors <[email protected]>

* [create-pull-request] automated change (#4174)

Co-authored-by: thebentern <[email protected]>

* Display alerts (#4170)

* Move static functions into Screen.h, show compass during calibration

* Move to _fontHeight macro to avoid collision

* Move some alert functions to new alert handler

* Catch missed reboot code

* ESP32 fixes

* Bump esp8266-oled-ssd1306

* Fixes for when a device has no screen

* Use new startAlert(char*) helper class

* Add EINK bits back to alert handling

* Add noop class for no-display devices

---------

Co-authored-by: Ben Meadors <[email protected]>

* Send file system manifest up on want_config (#4176)

* Send file system manifest up on want_config

* Platform specific methods

* Helps to actually make the change

* Clear

* tell vscode, if formatting, use whatever our trunk formatter wants (#4186)

without this flag if the user has set some other formatter (clang)
in their user level settings, it will be looking in the wrong directory
for the clang options (we want the options in .trunk/clang)

Note: formatOnSave is true in master, which means a bunch of our older
files are non compliant and if you edit them it will generate lots of
formatting related diffs.  I guess I'll start letting that happen with
my future commits ;-).

* fix the build - would loop forever if there were no files to send (#4188)

* Show owner.short_name on boot (and E-Ink sleep screen) (#4134)

* Show owner.short_name on boot and sleep screen (on e-ink)

* Update Screen.cpp - new line for short_name

Boot screen short_name now below the region setting.
Looks better on small screens.

* Draw short_name on right

---------

Co-authored-by: Thomas Göttgens <[email protected]>
Co-authored-by: todd-herbert <[email protected]>
Co-authored-by: Ben Meadors <[email protected]>

* nrf52 soft device will watchdog if you use ICE while BT on... (#4189)

so have debugger disable bluetooth.

* correct xiao_ble build preventing sx1262 init (#4191)

* Force a compile time failur if FromRadio or ToRadio get larger than (#4190)

a BLE packet size. We are actually very close to this threshold so
important to make sure we don't accidentally pass it.

* Clear vector after complete config state (#4194)

* Clear after complete config

* Don't collect . entries

* Log file name and size

* [create-pull-request] automated change (#4200)

Co-authored-by: thebentern <[email protected]>

* Make the logs Colorful! (#4199)

* Squash needlessly static functions (#4183)

* Trim extra vprintf and filter for unprintable characters

* Deprecate Router Client role (and make it Client) (#4201)

* [create-pull-request] automated change (#4205)

Co-authored-by: thebentern <[email protected]>

* Move waypoint (#4202)

* Move waypoint screen draw into the waypoint module

* Get the observer set up for the waypoint screen draw

* Static squashing: screen dimensions
Macros moved back to Screen.cpp, as a band-aid until we eventually move all those static functions into the Screen class.

* Move getCompassDiam into Screen class
(supress compiler warnings)
At this stage, the method is still static, because it's used by drawNodeInfo, which has no tidy reference to our screen instance.
This is probably just another band-aid until these static functions all move.

* Use new getCompassDiam function in AccelerometerThread

* Properly gate display code in WaypointModule

---------

Co-authored-by: Todd Herbert <[email protected]>

* Fix flakey phone api transition from file manifest to complete (#4209)

* Try fix flakey phone api transition from file manifest to complete

* Skip

* enable colors in platformio serial monitor (#4217)

* When talking via serial, encapsulate log messages in protobufs if necessary (#4187)

* clean up RedirectablePrint::log so it doesn't have three very different implementations inline.

* remove NoopPrint - it is no longer needed

* when talking to API clients via serial, don't turn off log msgs instead encapsuate them

* fix the build - would loop forever if there were no files to send

* don't use Segger code if not talking to a Segger debugger

* when encapsulating logs, make sure the strings always has nul terminators

* nrf52 soft device will watchdog if you use ICE while BT on...
so have debugger disable bluetooth.

* Important to not print debug messages while writing to the toPhone scratch buffer

* don't include newlines if encapsulating log records as protobufs

---------

Co-authored-by: Ben Meadors <[email protected]>

* [create-pull-request] automated change (#4218)

Co-authored-by: thebentern <[email protected]>

* Fix SHT41 support (#4222)

* Add SHT41 Serial to I2c Detection Code

On the Seeed Wio-WM1110 Dev Kit board, the SHT41 chip was being
incorrectly detected as SHT31.

This patch adds the necessary serial number for the SHT41 chip to
be correctly detected.

fixes #4221

* Add missing sensor read for SHT41

* Typo fix in logs - mhz - MHz (#4225)

As reported by karamo, a few different places in our logs had
incorrect capitalization of MHz.

fixes #4126

* New new BLE logging characteristic with LogRecord protos  (#4220)

* New UUID

* New log radio characteristic with LogRecord protobuf

* LogRecord

* Merge derp

* How did you get there

* Trunk

* Fix length

* Remove assert

* minor cleanup proposal (#4169)

* MESHTASTIC_EXCLUDE_WIFI and HAS_WIFI cleanup...
Our code was checking HAS_WIFI and the new MESHTASTIC_EXCLUDE_WIFI
flags in various places (even though EXCLUDE_WIFI forces HAS_WIFI
to 0).  Instead just check HAS_WIFI, only use EXCLUDE_WIFI inside
configuration.h

* cleanup: use HAS_NETWORKING instead of HAS_WIFI || HAS_ETHERNET
We already had HAS_NETWORKING as flag in MQTT to mean 'we have
tcpip'.  Generallize that and move it into configuration.h so that
we can use it elsewhere.

* Use #pragma once, because supported by gcc and all modern compilers
instead of #ifdef DOTHFILE_H etc...

---------

Co-authored-by: Jonathan Bennett <[email protected]>

* Add PowerMon support (#4155)

* Turn off vscode cmake prompt - we don't use cmake on meshtastic

* Add rak4631_dap variant for debugging with NanoDAP debug probe device.

* The rak device can also run freertos (which is underneath nrf52 arduino)

* Add semihosting support for nrf52840 devices
Initial platformio.ini file only supports rak4630
Default to non TCP for the semihosting log output for now...
Fixes #4135

* powermon WIP (for #4136 )

* oops - mean't to mark the _dbg variant as an 'extra' board.

* powermon wip

* Make serial port on wio-sdk-wm1110 board work
By disabling the (inaccessible) adafruit USB

* Instrument (radiolib only for now) lora for powermon
per #4136

* powermon gps support
#4136

* Add CPU deep and light sleep powermon states
#4136

* Change the board/swversion bootstring so it is a new "structured" log msg.

* powermon wip

* add example script for getting esp S3 debugging working
Not yet used but I didn't want these nasty tricks to get lost yet.

* Add PowerMon reporting for screen and bluetooth pwr.

* make power.powermon_enables config setting work.

* update to latest protobufs

* fix bogus shellcheck warning

* make powermon optional (but default enabled because tiny and no runtime impact)

* tell vscode, if formatting, use whatever our trunk formatter wants
without this flag if the user has set some other formatter (clang)
in their user level settings, it will be looking in the wrong directory
for the clang options (we want the options in .trunk/clang)

Note: formatOnSave is true in master, which means a bunch of our older
files are non compliant and if you edit them it will generate lots of
formatting related diffs.  I guess I'll start letting that happen with
my future commits ;-).

* add PowerStress module

* nrf52 arduino is built upon freertos, so let platformio debug it

* don't accidentally try to Segger ICE if we are using another ICE

* clean up RedirectablePrint::log so it doesn't have three very different implementations inline.

* remove NoopPrint - it is no longer needed

* when talking to API clients via serial, don't turn off log msgs instead encapsuate them

* fix the build - would loop forever if there were no files to send

* don't use Segger code if not talking to a Segger debugger

* when encapsulating logs, make sure the strings always has nul terminators

* nrf52 soft device will watchdog if you use ICE while BT on...
so have debugger disable bluetooth.

* Important to not print debug messages while writing to the toPhone scratch buffer

* don't include newlines if encapsulating log records as protobufs

* update to latest protobufs (needed for powermon goo)

* PowerStress WIP

* fix linter warning

* Cleanup buffer

* Merge hex for wm1110 target(s)

* Only sdk

* Sudo

* Fix exclude macros (#4233)

* fix MESHTASTIC_EXCLUDE_BLUETOOTH

* fix HAS_SCREEN=0

* fix MESHTASTIC_EXCLUDE_GPS

* fix typo in build-nrf52.sh (#4231)

chmod is the command, '+x' is the argument.

* Tidy Wireless Paper variant files (#4238)

* Quick tidy of pins_arduino.h
Matches requests made at #4226 (comment))

* Tidy variant.h

* Change deprecated ADC attenuation parameter
From 11dB to 12dB. Resolves compiler warning. Allegly, no impact on function: `This is deprecated, it behaves the same as `ADC_ATTEN_DB_12`

* Updated raspbian CI to update apt repository ahead of libbluetooth. (#4243)

* Fix BLE logging on nrf52 (#4244)

* allow ble logrecords to be fetched either by NOTIFY or INDICATE ble types

This allows 'lossless' log reading.  If client has requested INDICATE
(rather than NOTIFY) each log record emitted via log() will have to fetched
by the client device before the meshtastic node can continue.

* Fix serious problem with nrf52 BLE logging.
When doing notifies of LogRecords it is important to use the
binary write routines - writing using the 'string' write won't work.
Because protobufs can contain \0 nuls inside of them which if being
parsed as a string will cause only a portion of the protobuf to be sent.
I noticed this because some log messages were not getting through.

---------

Co-authored-by: Ben Meadors <[email protected]>

* Fix build when HAS_NETWORKING is false on nrf52 (#4237)

(tested on a rak4631 by setting HAS_ETHERNET false when shrinking
image)

* If `toPhoneQueue` is full, still increment `fromNum` to avoid client never getting packets (#4246)

* Update to SoftDevice 7.3.0 for wio-sdk-wm1110 and wio-tracker-wm1110 (#4248)

* Update variant.h

* Update wio-tracker-wm1110.json

* Update wio-sdk-wm1110.json

* Update platformio.ini

* Update platformio.ini

* Add files via upload

* Add files via upload

* Update variant.h

* Cleanup NRF s140 Softdevice variants (#4252)

Note: This idea is originally from @caveman99 and should be
credited as such. Submitting as a separate PR so the work in
#4148 can be a bit cleaner and Seeed boards
can build while that work is ongoing.

The nrf52 boards that depend on the v7 softdevice all use the same
code and linker files. Rather than duplicate the code, keep it
all together with the platform.

* Remove tracker variant specific soft device headers (#4255)

* [create-pull-request] automated change (#4247)

Co-authored-by: thebentern <[email protected]>

* Add wio-sdk-wm1110 to build. (#4258)

The wio-sdk-wm1110 is distinct from the wio-tracker-wm1110, with
different platformio build options and pin config.

This change adds the wio-sdk-wm1110 to the CI matrix so firmware
is built as part of release.

* fix python warning in uf2conf (#4235)

the old regex worked but was technically incorrect.  fixes:
Generating NRF52 uf2 file
/home/kevinh/development/meshtastic/firmware/bin/uf2conv.py:195: SyntaxWarning: invalid escape sequence '\s'
  words = re.split('\s+', line)
Converting to uf2, output size: 1458688, start address: 0x26000

* Collect hex files and specifically wm1110 sdk

* Skip dfu file for sdk (for now)

* Helps if you remove the original clause

* Add Heltec new boards. (#4226)

* Add Heltec new boards

* Update variant.h

disable RTC by default

* Add Heltec New boards

* Add Heltec new boards

* Update Heltec Mesh Node definition.

* Update Heltec Vision Mater E290

* [create-pull-request] automated change (#4259)

Co-authored-by: thebentern <[email protected]>

* Trunk fmt

* Fix macros

* Move e290 to board level extra while CI is broken

* Tell trunk to ignore bin folder

* Fix missing

* Update trunk.yaml, fix whitespace

* Update trunk.yaml

* Update build_raspbian_armv7l.yml --fix-missing

* [create-pull-request] automated change (#4263)

Co-authored-by: thebentern <[email protected]>

* GPS Power State tidy-up  (#4161)

* Refactor GPSPowerState enum
Identifies a case where the GPS hardware is awake, but an update is not yet desired

* Change terminology

* Clear old lock-time prediction on triple press

* Use exponential smoothing to predict lock time

* Rename averageLockTime to predictedLockTime

* Attempt: Send PMREQ with duration 0 on MCU deep-sleep

* Attempt 2: Send PMREQ with duration 0 on MCU deep-sleep

* Revert "Attempt 2: Send PMREQ with duration 0 on MCU deep-sleep"

This reverts commit 8b697cd.

* Revert "Attempt: Send PMREQ with duration 0 on MCU deep-sleep"

This reverts commit 9d29ec7.

* Remove unused notifyGPSSleep Observable
Handled with notifyDeepSleep, and enable() / disable()

* WIP: simplify GPS power management
An initial attempt only.

* Honor #3e9e0fd

* No-op when moving between GPS_IDLE and GPS_ACTIVE

* Ensure U-blox GPS is awake to receive indefinite sleep command

* Longer pause when waking U-blox to send sleep command

* Actually implement soft and hard sleep..

* Dynamically estimate the threshold for GPS_HARDSLEEP

* Fallback to GPS_HARDSLEEP, if GPS_SOFTSLEEP unsupported

* Move "excessive search time" behavior to scheduler class

* Minor logging adjustments

* Promote log to warning

* Gratuitous buffer clearing on boot

* Fix inverted standby pin logic
Specifically the standby pin for L76B, L76K and clones
Discovered during T-Echo testing: totally broken function, probe method failing.

* Remove redundant pin init
Now handled by setPowerState

* Replace max() with if statements
Avoid those platform specific implementations..

* Trunk formatting
New round of settings.json changes keep catching me out, have to remember to re-enable my "clang-format" for windows workaround.

* Remove some asserts from setPowerState
Original aim was to prevent sending a 0 second PMREQ to U-blox hardware as part of a timed sleep (GPS_HARDSLEEP, GPS_SOFTSLEEP). I'm not sure this is super important, and it feels tidier to just allow the 0 second sleeptime here, rather than fudge the sleeptime further up.

* Fix an error determining whether GPS_SOFTSLEEP is supported

* Clarify a log entry

* Set PIN_STANDBY for MCU deep-sleep
Required to reach TTGO's advertised 0.25mA sleep current for T-Echo. Without this change: ~6mA.

* Optimize the shutdown current of RAK10701 to around 25uA (#4260)

Co-authored-by: Jonathan Bennett <[email protected]>

* INA3221 sensor: use for bus voltage & environment metrics (#4215)

* use INA3221 for bus voltage; fixes for telemetry variants

- add to sensors available for environment telemetry
  (to report voltage/current)
- add vars to define channels to use for battery voltage
  (for getBusVoltage) and environment metrics (default
  to CH1 for both)
- write to the correct fields on the measurement struct
  depending on the measurement variant, and DRY up the
  sensor measurement collection code a bit
- this might be suitable for a common implementation for
  the INA* sensors in a future PR...

* formatting

* derp

---------

Co-authored-by: Ben Meadors <[email protected]>

* WM1110 SDK kit enter serial DFU and add deployment packages (#4266)

* Switch default upload protocol to nrfutil so that pio generates zip deploy packages

* Enter serial DFU on SDK board

* Remove guard for DFU zip from SDK build

* NRF_USE_SERIAL_DFU macro instead

* Show specific frame when updating screen (#4264)

* Updated setFrames in Screen.cpp

Added code to attempt to revert back to the same frame that user was on prior to setFrame reload.

* Space added Screen.cpp

* Update Screen.cpp

Make screen to revert to Frame 0 if the originally displayed frame is no longer there.

* Update Screen.cpp

Inserted boolean holdPosition into setFrames to indicate the requirement to stay on the same frame ( if =true) or else it will switch to new frame .
Only Screen::handleStatusUpdate calls with setFrame(true). ( For Node Updates)
All other types of updates call as before setFrame(), so it will change focus as needed.

* Hold position, even if number of frames increases

* Hold position, if handling an outgoing text message

* Update Screen.cpp

* Reverted chnages related to devicestate.has_rx_text_message

* Reset to master

* CannedMessages only handles routing packets when waiting for ACK
Previously, this was calling Screen::setFrames at unexpected times

* Gather position info about screen frames while regenerating

* Make admin module observable
Notify only when relevant. Currently: only to handle remove_nodenum.

* Optionally specify which frame to focus when setFrames runs

* UIFrameEvent uses enum instead of multiple booleans

* Allow modules to request their own frame to be focussed
This is done internally by calling MeshModule::requestFocus()
Easier this way, insteady of passing the info in the UIFrameEvent:
* Modules don't always know whether they should be focussed until after the UIFrameEvent has been raised, in dramFrame
* Don't have to pass reference to module instance as parameter though several methods

* E-Ink screensaver uses FOCUS_PRESERVE
Previously, it had its own basic implementation of this.

* Spelling: regional variant

* trunk

* Fix HAS_SCREEN guarding

* More HAS_SCREEN guarding

---------

Co-authored-by: BIST <[email protected]>
Co-authored-by: Ben Meadors <[email protected]>
Co-authored-by: slash-bit <[email protected]>

* Move up telemetry defaults to every 30 minutes (#4274)

* Don't send node info interrogation when ch. util is >25% (#4273)

* Moar LR1110 Targets

* update SD_FLASH_SIZE to 0x27000 (#4232)

The 7.3.0 softdevice needs the extra 1000 :)

* Fix spacing.

---------

Co-authored-by: Mike <[email protected]>
Co-authored-by: Ben Meadors <[email protected]>
Co-authored-by: Mike G <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: thebentern <[email protected]>
Co-authored-by: GUVWAF <[email protected]>
Co-authored-by: Warren Guy <[email protected]>
Co-authored-by: todd-herbert <[email protected]>
Co-authored-by: geeksville <[email protected]>
Co-authored-by: Jonathan Bennett <[email protected]>
Co-authored-by: Alexander <[email protected]>
Co-authored-by: Thomas Göttgens <[email protected]>
Co-authored-by: quimnut <[email protected]>
Co-authored-by: Manuel <[email protected]>
Co-authored-by: Agent Blu, 006 <[email protected]>
Co-authored-by: Mark Trevor Birss <[email protected]>
Co-authored-by: Aaron.Lee <[email protected]>
Co-authored-by: Daniel.Cao <[email protected]>
Co-authored-by: BIST <[email protected]>
Co-authored-by: slash-bit <[email protected]>
thebentern added a commit that referenced this issue Aug 6, 2024
* Turn off vscode cmake prompt - we don't use cmake on meshtastic

* Add rak4631_dap variant for debugging with NanoDAP debug probe device.

* The rak device can also run freertos (which is underneath nrf52 arduino)

* Add semihosting support for nrf52840 devices
Initial platformio.ini file only supports rak4630
Default to non TCP for the semihosting log output for now...
Fixes #4135

* powermon WIP (for #4136 )

* oops - mean't to mark the _dbg variant as an 'extra' board.

* powermon wip

* Make serial port on wio-sdk-wm1110 board work
By disabling the (inaccessible) adafruit USB

* Instrument (radiolib only for now) lora for powermon
per #4136

* powermon gps support
#4136

* Add CPU deep and light sleep powermon states
#4136

* Change the board/swversion bootstring so it is a new "structured" log msg.

* powermon wip

* add example script for getting esp S3 debugging working
Not yet used but I didn't want these nasty tricks to get lost yet.

* Add PowerMon reporting for screen and bluetooth pwr.

* make power.powermon_enables config setting work.

* update to latest protobufs

* fix bogus shellcheck warning

* make powermon optional (but default enabled because tiny and no runtime impact)

* tell vscode, if formatting, use whatever our trunk formatter wants
without this flag if the user has set some other formatter (clang)
in their user level settings, it will be looking in the wrong directory
for the clang options (we want the options in .trunk/clang)

Note: formatOnSave is true in master, which means a bunch of our older
files are non compliant and if you edit them it will generate lots of
formatting related diffs.  I guess I'll start letting that happen with
my future commits ;-).

* add PowerStress module

* nrf52 arduino is built upon freertos, so let platformio debug it

* don't accidentally try to Segger ICE if we are using another ICE

* clean up RedirectablePrint::log so it doesn't have three very different implementations inline.

* remove NoopPrint - it is no longer needed

* when talking to API clients via serial, don't turn off log msgs instead encapsuate them

* fix the build - would loop forever if there were no files to send

* don't use Segger code if not talking to a Segger debugger

* when encapsulating logs, make sure the strings always has nul terminators

* nrf52 soft device will watchdog if you use ICE while BT on...
so have debugger disable bluetooth.

* Important to not print debug messages while writing to the toPhone scratch buffer

* don't include newlines if encapsulating log records as protobufs

* update to latest protobufs (needed for powermon goo)

* PowerStress WIP

* for #4154 and #4136 add concept of dependent gpios...
Which is currently only tested with the LED but eventually
will be used for shared GPIO/screen power rail enable
and LED forcing (which is a sanity check in the power stress
testing)

* fix linter warning

* Transformer is a better name for the LED input > operation > output classes

* PMW led changes to work on esp32-s3

* power stress improvements

* allow ble logrecords to be fetched either by NOTIFY or INDICATE ble types

This allows 'lossless' log reading.  If client has requested INDICATE
(rather than NOTIFY) each log record emitted via log() will have to fetched
by the client device before the meshtastic node can continue.

* Fix serious problem with nrf52 BLE logging.
When doing notifies of LogRecords it is important to use the
binary write routines - writing using the 'string' write won't work.
Because protobufs can contain \0 nuls inside of them which if being
parsed as a string will cause only a portion of the protobuf to be sent.
I noticed this because some log messages were not getting through.

* fix gpio transformer stuff to work correctly with LED_INVERTED

Thanks @todd-herbert for noticing this and the great stack trace.
The root cause was that I had accidentially shadowed outPin in a subclass
with an unneeded override.  It would break on any board that had inverted
LED power.

fixes
#4230 (review)

* Support driving multiple output gpios from one input.

While investigating #4230 (review)
I noticed in variant.h that there are now apparently newer TBEAMs than mine
that have _both_ a GPIO based power LED and the PMU based LED.  Add a splitter
so that we can drive two output GPIOs from one logical signal.

---------

Co-authored-by: Ben Meadors <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants