Features/beacon network

[gsfk]

First pass at code for networking Bento beacons together.

• On startup, contacts all beacons in a list of urls passed in from a config file.
• routes requests/responses for any beacon in the network.

Aggregation is done in the client rather than here. This allows for real-time updates in the frontend, rather than waiting for the slowest beacon to respond. However, aggregation is easy and should come in a later version, since it lets us:

• treat the network like a single beacon
• add the network to an existing network
• search the network from a non-Bento client that doesn't know how to aggregate the data

Some features are still experimental:

• uses bento_public search config for filtering terms discovery (eg sex=MALE or BMI<18) since that's what our beacons currently use. Once changes are rolled out to all beacons, we can use beacon spec filtering terms instead.
• The network itself is hosted on a particular beacon instance, which will be expected (but not required) to be part of the network, so there is some extra handling to avoid non-terminating circular http requests. Whether we are better off hosting the network separately is left as an exercise for the reviewer.

Related PRS:

• bento #259
• bento_public #165

[davidlougheed]

first pass

[davidlougheed]

type hint seems wrong here if request can be false-y - should be Request | None?

[gsfk]

this one is fiddly.. r is not actually a request, it's a werkzeug LocalProxy that acts like a request when a request happens to present. When no request is present it's just LocalProxy rather than none.

[davidlougheed]

hmm...

[gsfk]

I'm considering just removing this and handling the no-request edge case elsewhere. Seems inelegant to ask for headers when there's no request.

[gsfk]

removed

[davidlougheed]

it'd be better to accept the katsu URL from a configuration variable and then build from there (so just katsu URL + /api/public_search_fields)

[gsfk]

ah, hmm. These are calls to other katsus in the network:

1. PUBLIC_SEARCH_FIELDS_PATH probably belongs in config rather than hidden in this file. I probably left it here because this entire file will be gone in the next version.
2. beacon network is configured by passing in beacon base urls. We find the katsu search fields by figuring out the base bento url (ie remove /api/beacon) and adding on PUBLIC_SEARCH_FIELDS_PATH. So, you have your choice of which is least inelegant:
     - url surgery as above
     - configure network by using bento base urls instead of beacon urls
     - configure network by passing both beacon and katsu urls

In the next iteration this will all be replaced by calls to network beacons only, so this sort of url modification will no longer be needed.

[davidlougheed]

if it's temporary it's fine to leave in, but in general I think not relying on Bento's particular URL setup at time of code-writing (so configuring Katsu URL separately) is the best long-term approach if any contact is needed.

[davidlougheed]

this shouldn't rely on katsu being under the portal. subdomain forever in bento. instead, take the Katsu URL as an env var (similar to aggregation or WES) and build it from Katsu + whatever subpath is required.

[gsfk]

see above