Invalid FQBN assembled when discovery specifies custom board options

[per1234]

Describe the problem

Arduino boards platforms may provide additional configurability of a given board definition via "custom board options".

These options are specified as part of the FQBN, via the following format:

<vendor ID>:<architecture>:<board ID>[:<menu ID>=<option ID>[,<menu ID>=<option ID>]...]

The default option is used for custom board options when not specified via the FQBN, so those components are optional even for boards that have custom options.

Previously, due to limitations in the pluggable discovery infrastructure, it was not possible for custom board options settings to be identified along with the base board identification. That limitation has now been overcome (arduino/arduino-cli#1674). Board options can now be set according to the identification properties provided by a discovery tool:

https://arduino.github.io/arduino-cli/dev/pluggable-discovery-specification/#identification-of-board-options

This means the fqbn field of the cc.arduino.cli.commands.v1.BoardListItem message may now contain FQBNs that contain custom board options.

🐛 Board options from the Tools menu selections are appended to the FQBN from the cc.arduino.cli.commands.v1.BoardList response under the assumption the FQBN will have the basic <vendor ID>:<architecture>:<board ID> form. This produces an invalid FQBN.

To reproduce

Equipment

• A board that is configured for identification of custom board options via discovery properties.

Currently, the only boards platform taking advantage of this capability is the "Teensy" platform. Any Teensy board can be used. If this is an obstacle for those working on this issue, I can provide instructions for setting up custom board option identification for any board.

Steps

1. Connect the board to your computer.
2. Select the board from the "Board Selector" menu on the Arduino IDE toolbar.
   ❗ The board must be selected from this specific menu. The issue does not occur when selecting the board from the Tools > Board menu or "Select Other Board and Port" dialog.
3. Select Sketch > Verify/Compile from the Arduino IDE menus.

🐛 Compilation fails:

Invalid FQBN: not an FQBN: teensy:avr:teensy41:usb=rawhid:usb=rawhid,speed=600,opt=o2std,keys=en-us

Compilation error: Invalid FQBN: not an FQBN: teensy:avr:teensy41:usb=rawhid:usb=rawhid,speed=600,opt=o2std,keys=en-us

Compare the generated FQBN:

teensy:avr:teensy41:usb=rawhid:usb=rawhid,speed=600,opt=o2std,keys=en-us

To the correct FQBN:

teensy:avr:teensy41:usb=rawhid,speed=600,opt=o2std,keys=en-us

Note that the full set of custom board options usb=rawhid,speed=600,opt=o2std,keys=en-us was appended to the FQBN provided by BoardList teensy:avr:teensy41:usb=rawhid (the usb option value is identified via the discovery properties).

Expected behavior

Arduino IDE always generates valid FQBN.

Arduino IDE version

2.0.1

Operating system

Windows

Operating system version

10

Additional context

The workaround for this bug is to always select the board from the Tools > Board menu in the Arduino IDE.

---

Originally reported by @KurtE at https://forum.arduino.cc/t/invalid-fqbn-not-an-fqbn/1046630

---

Additional reports:

• https://forum.pjrc.com/threads/71370-Teensy4-1-reading-from-Serial-Monitor-of-Arduino-IDE?p=314905&viewfull=1#post314905
• https://forum.pjrc.com/threads/71557-Arduino-2-0-3-FQBN-Error

Issue checklist

• I searched for previous reports in the issue tracker
• I verified the problem still occurs when using the latest nightly build
• My report contains all necessary details

[PaulStoffregen]

I've added version teensy-discovery version 1.57.1 to package_teensy_index.json. It removes the extra identification properties which allow Arduino CLI to report the menu info in FQBN.

For the sake of ability to reproduce this issue, I've made a copy of the package index at the time this issue was opened.

https://www.pjrc.com/teensy/package_issue1588_index.json

[kittaakos]

As a side effect of the flawed FQBN handling, the language server fails to start:

Failed to start language server. Original FQBN: teensy:avr:teensy41:usb=serial Error: Expected a sanitized FQBN with three segments in the following format: 'VENDOR:ARCHITECTURE:BOARD_ID'. Got teensy:avr:teensy41:usb=serial instead.
    at assertSanitizedFqbn (file:///Users/a.kitta/dev/git/arduino-ide/electron-app/lib/node_modules_the…esm_vs_base_common_sync_recursive-node_modules_moment_l-bf9e6e.js:29224:15)
    at InoLanguage.startLanguageServer (file:///Users/a.kitta/dev/git/arduino-ide/electron-app/lib/node_modules_the…_esm_vs_base_common_sync_recursive-node_modules_moment_l-bf9e6e.js:9347:48)

[PaulStoffregen]

Was hoping to test the artifact build on #2113 but trying to download it says "The logs for this run have expired and are no longer available." Is there a way to ping the CI system to rebuild?

[per1234]

Is there a way to ping the CI system to rebuild?

I triggered a build for that PR branch but it failed due to some bit rot that has occurred in the build infrastructure since the time that work was done.

The problem is already fixed in the main branch via 7e8f723, but that fix would need to be pulled into the PR branch before a tester build could be generated via the CI system.

[PaulStoffregen]

Any chance to create a new branch from latest main and then apply this PR to it?

Or some other "easy" way?

[kittaakos]

I will take care of it. The work on the PR needs to be completed, though. I am busy with the release today, but I will rebase the PR this week and let you know when it's in a state that can be verified. Thanks for your patience

[PaulStoffregen]

Thanks. No need to hurry on this.

[PaulStoffregen]

Congratulations on the 2.3.0 release!

Downloaded it just now. I'm very happy to say an issue updating the boards drop-down list, which I've been struggling to make a reproducible test case, is completely fixed in 2.3.0. :-)

[kittaakos]

Any chance to create a new branch from latest main and then apply this PR to it?

I reimplemented the feature with the latest (0.36.0 RC) CLI. Please try out the IDE2 tester builder from #2334. Here is the documentation on how to use the testers builds.

To achieve the most in the next few weeks, I decided to close #2113 and dump all CLI features + fixes into one PR. Sorry for the inconvenience.

[PaulStoffregen]

Running the artifact built Linux. Seems to be working quite well. :-)

I installed a copy of teensy-discovery which reports the USB Type info. I have 2 boards on my desk right now, a Teensy 4.0 programmed as Audio and a Teensy 3.6 programmed as MIDI.

Here's the starting condition, Tools > Board is "Teensy 4.0" and Tools > USB Type is "Audio".

When I unplug the Teensy 4.0 and plug in the Teensy 3.6, indeed the Board Select drop-down menu updates and the popup tip shows the partial-FQBN has "usb=midi".

After selecting this, indeed the Tools menu is properly updated. Board is now Teensy 3.6 and USB Type is now MIDI.

Very nice!

[PaulStoffregen]

Today my discovery tool can only discern 1 of the menu options, because the other info just isn't available. Over the next few days I'll try extending my USB descriptors and update the discovery tool to detect more.

[PaulStoffregen]

Testing with 2 menu options also looks good.

Here's a test where the partial FQBN from discovery gives 2 menu options, viewable in the popup tip.

Happy to report both did get applied. USB Type got set to "Audio" and CPU Speed got set to "528 MHz".

Looks like it's working quite well.

[PaulStoffregen]

#2334 looks really good to me!

[PaulStoffregen]

Anything else I can do to help?

I see #2334 already has merge conflicts. Would be really unfortunate if it also suffers bit rot and later needs to be redone again.

[kittaakos]

Anything else I can do to help?

Yes. I merged the changes. Please start using the next nightly build (from 21.02) and report any strange behavior to us. Please name me if you find any issues. We have to fix all of them this week. Thanks for the help!

Update
I corrected the date