Make the default list of catalogs configurable

[mishaschwartz]

Motivation

Allows setting a custom list of catalogs to be displayed to the user when catalogUrl is not set.

Currently:

If catalogUrl is empty or set to null STAC Browser switches to a mode where it defaults to a screen where you can either insert a catalog URL or select a catalog from stacindex.org.

This PR allows the user to choose a URL other than https://stacindex.org and https://stacindex.org/api/catalogs in order to display a different list of catalogs if catalogUrl is not set.

Changes

Adds three new config variables:

• indexURL: The URL of a webpage that displays information about the STAC catalogs
• indexCatalogsURL: A URL that returns a JSON array of STAC catalog objects
• indexName: The name of the page referred to by indexURL

The defaults for all these variables are the same as the current hard-coded values. This means that all changes are fully backwards compatible and users should see no change to their stac-browser instances if these variables are not changed in their configuration.

Related Issues

#328

[huard]

Hi @m-mohr,

I'm hoping to see this PR moving. Is there anything that I could do ?

Many thanks,

David

[m-mohr]

@huard STAC Browser is currently unfunded (see also https://github.com/radiantearth/stac-browser?tab=readme-ov-file#stac-browser ). Unless someone provides funding for STAC Browser, all work is voluntary and a free-time activity of mine. This includes e.g. the recent 3.2.0 release. I can only do so much and in summer I enjoy being outside. ;-)

PRs take considerable amount of time to review and in this specific case I'm not sure I like the idea of operating STAC Browser without a pre-defined catalogUrl, i.e. all catalogs are external. This was only ever meant to be a demo mode and not meant to be supported officially. Although this works, the technical solution is not ideal. @mishaschwartz

The best way to help is provide some base funding for STAC Browser maintenance. This would help to fix bugs, communicate with users, potentially discuss better solutions for PRs, and to review PRs.

[mishaschwartz]

@m-mohr Thank you for the response and I appreciate the time you are able to dedicate to managing this repo in your spare time.

When you have time/funding to have a look at this I'd be happy to explain our use case and why I think this change would improve STAC browser.

All the best,

Misha

[m-mohr]

I think it would be good to explain your use case and why you think this change would improve STAC browser in any case. Thanks. @mishaschwartz

[mishaschwartz]

Sure thing...

We have data hosted on multiple servers across multiple institutions. Each institution manages their data and the corresponding STAC catalog independently. However, we would like users to be able to view the catalogs of all institutions from a single location so that they don't have to navigate to multiple sites (STAC-browser instances).

This was only ever meant to be a demo mode and not meant to be supported officially

That makes sense. I think that this demo mode feature could also be used to show the user multiple related STAC instances in cases like ours where we have data set up in a more federated architecture.

I'm not sure I like the idea of operating STAC Browser without a pre-defined catalogUrl

That's fair and I'd be interested in why that is the case.

Another possibility is that we could have a catalogUrl and a indexCatalogsURL defined which would show the regular browser by default and then show the additional catalogs (as defined by the indexCatalogsURL) on a path other than /.

Our original plan was to build something custom but since STAC-browser supports displaying multiple catalogs already I thought it would be better to contribute to the community instead of hoarding features in a separate codebase.

[m-mohr]

Appreciated, thanks.

The way it is now, all catalogs are external catalogs. (see /external/... in the URL of STAC Browser).

This is not ideal:

1. Relies on the full URL being the URL
2. You have to load data in sequence rather in parallel during page (re)load etc
3. You can't disallow external URLs, because everything is external
4. Authentication can't be used properly
5. ...

Example:

• Catalog URL: https://abc.example/stac/catalog.json
• Example Item: https://abc.example/stac/items/item1.json

With the proposal in this PR, you could get the URL https://browser.example/external/abc.example/stac/items/item1.json for the STAC Item. Without reading the (potentially missing) root link from the Item, you can't properly render a page in STAC Browser. You need to first load item1.json, then follow the root link. It's sequential, not parallel and as such slower than needed.

A deployment for a single catalog would currently configure something like this:
catalogUrl = "https://abc.example/stac/catalog.json"
and you'd get the following URL in STAC Browser:
https://browser.example/items/item1.json

So for multiple catalogs I'd prefer is something like this in config:

catalogUrl = {
  "abc": "https://abc.example/stac/catalog.json",
  "def": "https://def.example/stac/catalog.json"
}

The URLs would then get a prefix, e.g.
https://browser.example/abc/items/item1.json

Due to the abc part in the STAC Browser URI, you avoid the issues above. You have an identifier for config (e.g. Auth) and can in parallel load root and item for example.

Hope this is not too confusing and makes sense to you?

Unfortunately this task is a little bigger than this PR and potentially needs a couple days of work. That's why I'm worried to merge this, because it encourages users to configure a way that is not really tested, optimized etc. And once we can implement my proposal, we'd have two approaches for the same functionality and removing it would be breaking for some.

[mishaschwartz]

Ok great! Yes that all makes sense and thank you for highlighting the sequential loading issues.

I'll have to look a bit deeper into the code to see how all that would interact but at I high level I think I understand what you're saying.

I see now that this will take a bit more work to accomplish. I'm going to chat with my team and we can figure out if we have some time to dedicate to adding this feature. If we come back with a PR some time soon will you have the capacity to review it or should we wait until you have more time/resources to dedicate to STAC-browser?

[m-mohr]

I think you could start with the work if you want. We may need a couple of iterations though, this also needs very good testing as it is a very fundamental change to the architecture. I'm not even sure myself how deep we need to dig. I try to help as much as I can as I'd appreciate the contribution, but it could certainly take some time...