Smoothen root key provisioning and device claiming

[johanstokking]

Summary

Smoothen the provisioning of root keys when creating new devices, and claiming end devices. This should be combined in one flow that covers 99% of the use cases.

Replaces #2777
Replaces https://github.com/TheThingsIndustries/lorawan-stack/issues/2250
References https://github.com/TheThingsIndustries/lorawan-stack/issues/2345

Why do we need this?

For users, creating and claiming end devices can be confusing. The goal is often the same: onboarding new end devices.

I'd like to cover the following cases:

Where are the keys	What to enter	What mechanism to use	Where does it go	JoinEUI
[1] Exposed to the user	Root keys	Creation	Cluster-local Join Server	Cluster-local JoinEUI
[2] Exposed to the user	Root keys	Creation *	The Things Join Server	70B3D57ED0000000/36
[3] The Things Join Server	Claim authentication code	Claiming ** / ***	N/A	70B3D57ED0000000/36
[4] Cluster-local Join Server	Claim authentication code	Claiming ***	N/A	Any
[5] Semtech Join Server	PIN	Claiming ***	N/A	58A0CB0000000000/24

* this should be configured via config, and should be based on a JoinEUI
** which claiming mechanism should be used is configurable but that is out of scope of this issue. References https://github.com/TheThingsIndustries/lorawan-stack/issues/2347
*** scanning a QR code can also work

What is already there? What do you see now?

Currently, there's a checkbox "External Join Server". Checking this assumes that the device is already claimed on the external JS, but it's not clear that this step is necessary. With that disabled, only use case 1 is supported.

What is missing? What do you want to see?

Support all the provisioning and claim flows above.

How do you propose to implement this?

I think we need a combination of:

1. Front-end configuration of whether it is possible to create a device on The Things Join Server (use case 2). Generally speaking, we probably want a dropdown list of JoinEUI ranges that map to a Join Server address that is in the same realm as the cluster. That is, as The Things Join Server is part of The Things Stack Cloud, authentication from The Things Network does not work, but from The Things Stack Cloud does work
2. Configuration in the implementation of the End Device Claiming Server to either claim end devices directly, or only configure the NetID and NSID. The latter would need to go via different interfaces: The Things Stack Join Server needs to expose a "claim API". The Semtech Join Server already exposes such an API: https://www.loracloud.com/documentation/join_service?url=appo_api.html#claim-devices
3. We should make this really easy for the user. When we have the JoinEUI (by selecting from a dropdown, entering a custom one or scanning a QR code), we can already narrow the options. If the JoinEUI indicates a Join Server (use case 2, 3, 4 and 5), the next question is what the user has: the root key (use case 2) or a claim authentication code (potentially in the QR code, use cases 3, 4 and 5). In the latter case, the use case is defined by the JoinEUI.
4. Authentication mechanism between components. We probably want to figure out a way to get rid of TLS mutual authentication for The Things Join Server. Can we use API keys?
5. Documentation to use The Things Join Server using LoRaWAN Backend Interfaces

Thoughts?

How do you propose to test this?

Lots of testing

Can you do this yourself and submit a Pull Request?

Can test

[johanstokking]

@htdvisser @kschiffer @bafonins please provide thoughts so we can finalize the implementation plan and land it in 3.12.

[htdvisser]

This looks like quite a big and complicated feature, and I doubt that we'll be able to finish the UX design and implementation for v3.12.0.

Perhaps you (@johanstokking) should first work with @kschiffer on the UX design, so that it's more clear what exactly we're building. After that I think we should tackle the different provisioning flows one-by-one.

Other than that, I don't have much to say about this.

[elsalahy]

It would be great if we can prioritize this issue so that we can claim the 500 batch of generic nodes on the community cluster.
We have a production deadline on the 15th of August.
We need to test this before placing the QR codes on the back of the devices.
cc: @azerimaker @johanstokking @NicolasMrad

[johanstokking]

Replaced by #4847