Investigate new AVAnnotate manifests

[elynema]

Description

Investigate AVAnnotate manifests (listed on #430) to review any changes with updated AVAnnotate tool and compare with list of features that we have defined as needing to be supported (see #430). May also need to determine if these manifests are valid.

Done Looks Like

• Identify any questions we have about how the AVAnnotate manifests work to pass back, including any manifests that aren't valid or seem to have problems
• Identify types of parsing changes that will be needed to support these manifests
• Create any new issues needed to support this functionality beyond those already linked to AVAnnotate Grant Work #430

[Dananji]

Validity of each new manifest (validated against Presentation 3.0 API in IIIF Presentation Validator:

AVAnnotate

• Audio: Zora Neale Hurston field recordings at the Library of Congress : valid ✅
• Video: The Lonely Villa (1909) & Suspense (1913): invalid ❌ , manifest link: https://avannotate.github.io/SuspenseTheLonelyVilla/manifests/manifest.json
• Comparison: https://tanyaclement.github.io/shoes/: invalid ❌ , manifest link: https://tanyaclement.github.io/shoes/manifests/manifest.json, cause: PointSelector has a t value, which is a string. This probably should be fixed as this would be something Ramp needs to parse for the annotations work.
• External linked AnnotationPage manifest: https://tanyaclement.github.io/znh_jacksonville_1939/s1576-t86-244/manifest.json: valid ✅

Rest of the items have the same Manifest link, https://avannotate.github.io/dev-demo/manifests/manifest.json: invalid ❌, cause: format: 'unknown' for YouTube videos listed in the Manifest. This is not a blocker for the annotations work.

• Clips: Demo
• Imported Manifest with Captions: UAlberta Writer in Residence Ifeoma Chinwuba and Edmonton Poet Laureate Titilope Sonuga
• Multi-AV Video Event
• Multi-AV Audio Event

Aviary

• item with multiple files (need to support navigating across multiple canvases in Ramp): https://weareavp.aviaryplatform.com/iiif/hm52f7jz70/manifest: invalid ❌
• transcripts as textual annotations: https://yalemssa.aviaryplatform.com/iiif/028pc2tq0m/manifest: invalid ❌
  For both manifests the validator gives the error: Additional properties are not allowed ('logo' was unexpected) This is not a blocker for the annotations work.

[Dananji]

Questions:

1. annotations list in an item contains a AnnotationPage with multiple annotations. These annotations have different sets of motivations;

• a supplementing motivation for the Annotation while motivation is set to tagging/commenting or both in the body of the annotation. The target for these annotations has a media fragment (similar to supplementing annotations in Avalon manifests for transcripts). E.g. https://tanyaclement.github.io/znh-1939/manifests/manifest.json. In Aviary, these annotations are presented with 2 linked timestamps (start and end of media fragment) alongside the text. In AVAnnotate viewer they are displayed similar to transcript lines in Ramp's Transcript component.
• a motivation: [ 'commenting', 'tagging'] for the Annotation while motivation is set totagging/commenting in the body of the annotation. Target for these annotations has a point selector with a single time point (similar to highlighting annotations in Avalon playlist manifests for markers). E.g. https://tanyaclement.github.io/shoes/manifests/manifest.json.

In both AVAnnotate and Aviary both these representations are displayed under 'Annotations'.
Question: why do these 2 have different sets of motivations at the Annotation level?
Although it doesn't seem like this makes a difference, seeing both Aviary and AVAnnotate display them as annotations. I wanted to ask the question; are these expected to be painted on the page the same way?

2. Annotation body items with the purpose: 'tagging' has differently formatted values:

• In the item named Comparison, the values have the format value:"Editing:Superimposition", which is then used to create a legend for filters in the UI.
• In Zora Neale Hurston field recordings at the Library of Congress, the values have a single text value without a colon. E.g. value: "Herbert Halpert".
  Both these pages have an index, which has categorization of the tags present in the annotations. But only the first Manifest contains these category information.
  Note: These tags are not present in the Aviary interface for annotations.
  Question: which representation has the correct format for the value property of the tagging annotation?

3. Does AnnotationPage label has any meaning when it comes to displaying annotations on the page? Both Zora Neale Hurston field recordings at the Library of Congress and https://tanyaclement.github.io/shoes/ manifests have labels for the AnnotationPage that is under annotations list for each item. While AVAnnotate doesn't display this label, Aviary uses this value as a tag displayed next to the value of the annotation.

4. When there are multiple sets of annotations for a single Canvas, does Ramp display one set of annotations (like AVAnnotate) or multiple sets (like Aviary) at a given time? See more on the issue comment.

[Dananji]

Implementation notes:

1. Creates a body of an Annotation as a list of items, similar to Aviary. This is not invalid, but something that Ramp needs to accommodate in the parsing step. Related work: Fix annotation parsing for Aviary annotations #740. This needs to be re-checked against AVAnnotate manifests.

2. A single Annotation body can have multiple tags with purpose: 'tagging'. This should be accommodated in data structure built in the parser for annotations work.

3. Navigating to the next/previous canvases using the player controls doesn't work as expected for the Multi-AV Video Event example. Gives a blank player with the following message in the Web console:

   MEDIA_ERR_SRC_NOT_SUPPORTED: The media resource indicated by the src attribute was not suitable.
   

   Steps to re-produce:

   1. Load the Manifest into ramp, observe the first canvas loads without any issues.
   2. Click the next button, and it works without any issues.
   3. Click the next button again (now we are on the 3rd Canvas in the list, which is a YouTube video), and it gives an empty player without any message.

   Another example of a multi-video Manifest from AVAnnotate: https://avannotate.github.io/annotating-adler/manifests/manifest.json (from the A/V working group interoperability spreadsheet), and the previous/next navigation using player controls works fine for this one.
   The above could be related to the fact that particular media file with the error is a YouTube video?

4. Annotations within the AnnotationPage are not in chronological order. These need to be ordered.

[Dananji]

No further investigation is necessary at this point.