Align articles/events filters on API faceting principles

[agnesgaroux]

NEW TICKET DESCRIPTION

After discussing this further, we decided the former ticket description wasn't exactly what was needed, but keeping it for legacy conversations.

See comments in ticket for more information as to what was decided upon, starting here.

FORMER TICKET DESCRIPTION

https://github.com/wellcomecollection/docs/tree/main/rfcs/037-api-faceting-principles
See Principle 1.

Filters are named by the JSON paths of the identified object that they filter or, if applied to an attribute other than the identifier, the path of that attribute

This is not something that has been abided by in content-api:
NOTE: this example is articles but same thing with events
Article doc XkvmsREAACUAUWwN has a filter object like

        "filter": {
            "contributorIds": [ // should be "contributors.contributor"
              "XjQP8BQAACQANdYu",
              "XjFs8xQAACMAKkcQ"
            ],
            "formatId": "W7TfJRAAAJ1D0eLK",  // should be "format"
            "publicationDate": "2020-02-27T11:31:56.000Z"

and display

 "display": {
      "type": "Article",
      "id": "XkvmsREAACUAUWwN",
      "title": "The ‘undesirable epileptic’",
      "caption": "Abused in her marriage for being 'a sick woman', Aparna Nair looked to history to make sense of the response to her epilepsy. 
She discovered how centuries of fear and discrimination were often endorsed by science and legislation.",
      "format": {
        "type": "ArticleFormat",
        "id": "W7TfJRAAAJ1D0eLK",
        "label": "Article"
      },
      "publicationDate": "2020-02-27T11:31:56+0000",
      "contributors": [
        {
          "type": "Contributor",
          "contributor": {
            "type": "Person",
            "id": "XjQP8BQAACQANdYu",
            "label": "Aparna Nair"
          },
          "role": {
            "type": "EditorialContributorRole",
            "id": "WcUWeCgAAFws-nGh",
            "label": "Author"
          }
        },
        {
          "type": "Contributor",
          "contributor": {
            "type": "Person",
            "id": "XjFs8xQAACMAKkcQ",
            "label": "Tracy Satchwill"
          },
          "role": {
            "type": "EditorialContributorRole",
            "id": "XVQa_xMAACMAqbwo",
            "label": "Illustrator"
          }
        }
      ],
    }

This entails

• changing the transformers
• reindexing the data
• change the value of filters/aggs in the API
• wc.org to use new values for query params

[rcantin-w]

Bringing in as in progress; we had a catch-up earlier with Agnès where we determined a re-write of the ticket might be necessary.

I'll aim to grab some of Paul's time to run through what the changes should look like when he's back ✨

[rcantin-w]

Booked a chat w Paul on Monday; moving to blocked in meantime.

[rcantin-w]

Discussing this with Paul;

• The faceting principle date from before the display and filter objects were split, so there's possibly a piece of revision to get done there. Not in scope for this ticket, but indicates that we might not want to take them as gospel.
• A change that would be ideal; not to reference Ids in our filter property name. e.g. contributorIds could/should be contributor. IDs are just what we happen to use to identify them.
• We could do this as part of the work that will get done to add new filters as it will require a new pipeline/reindex anyway. The front-end will need to support both names for a while, and then just go with the new name.

[paul-butcher]

The faceting principles are still appropriate, but could probably do with an addendum following the filter/display split.

When we wrote those principles, the data being interrogated/filtered and the data being returned was one and the same, so there was a concrete relationship between the filter names used in the API and the name/path to the field being filtered on in the output.

In order to simplify the relationship between pipelines and APIs, we introduced the filter/display split, whereby the data returned is an opaque block as far as the API is concerned. This allows us to make changes to the shape of the data to serve the requirements of the front end or an API client without having to change the way a query is formatted.

Because of this display/filter split, the relationship between the data seen by an API client and the querystring argument used to filter it is not so strict.

Ideally, we would continue to make sure that the query string parameter is sensible and predictable according to the faceting principles, but I also think they should remain consistent in the face of data changes. (e.g. if we move a field deeper into the tree structure of the output data, we should not change the name of the filter).

I think that unless there is a good reason, filters should probably be as flat as possible, so prefer contributor to contributors.contributor.