Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indexing MEI files for folios that don't exist in CantusDB is hacky #891

Open
dchiller opened this issue Aug 6, 2024 · 0 comments
Open

Comments

@dchiller
Copy link
Collaborator

dchiller commented Aug 6, 2024

When chants end on a folio where no other chants begin, the folio does not exist in CantusDB but does have an MEI file. Folio A14r in Salzinnes is a good example of this:

image

There is only one line of music on this folio -- the ending of a chant that begins on the previous folio. Thus, we have no chant in CantusDB that has folio "A14r", but we do have a MEI file for folio "A14r" from the OMR process.

#892 introduced a way to handle these cases:

  1. A user indexed the mei for a manuscript using the index_manuscript_mei command. When this command encounters MEI files for folios that don't exist, it does a preliminary sanity check (it ensures that the folio name "looks" like a folio name in that it has some numbers and maybe a trailing letter so that it wasn't just an accidentally copied file or some such) and then creates a folio in the CU database with that name. The command also alerts the user that it has done this.
  2. The user must then manually map these newly created folios to image uri's in the IIIF manifest either through the django admin panel or the map folios process. The alerts from the command will be helpful in this regard.
  3. The user must purge the index of omr_ngrams for that manuscript but running python manage.py index_manuscript_mei [manuscript_id] --flush-index.
  4. The user must reindex the omr_ngrams for that manuscript. If the above has been completed, no warnings will appear.

This issue exists to simplify this process after changes to CU's indexing and data import process are made to more closely mirror CU's index with the contents of CantusDB

dchiller added a commit to dchiller/cantus that referenced this issue Aug 6, 2024
… mei

When chants end on a folio where no other chants begin, the folio
does not exist in CantusDB but does have an MEI file. For example, see
folio A14r in Salzinnes. Here, we modify the index_manuscript_mei
command to create a folio in such cases (we check that it's a "real"
folio by making sure the mei file follows a naming convention and then
create the folio if it doesn't exist). The user is alerted to this, and
they must manually add the image_uri to the folio (either through the admin
panel or the map folios process) and then reindex the mei. This is detailed
in issue DDMAL#891. This is convoluted, but given that we're going to change the
structure of the CU database soon so that it is more closely coupled with
CantusDB, we'll figure out a more permanent solution then -- solving DDMAL#891).
dchiller added a commit that referenced this issue Aug 15, 2024
… mei

When chants end on a folio where no other chants begin, the folio
does not exist in CantusDB but does have an MEI file. For example, see
folio A14r in Salzinnes. Here, we modify the index_manuscript_mei
command to create a folio in such cases (we check that it's a "real"
folio by making sure the mei file follows a naming convention and then
create the folio if it doesn't exist). The user is alerted to this, and
they must manually add the image_uri to the folio (either through the admin
panel or the map folios process) and then reindex the mei. This is detailed
in issue #891. This is convoluted, but given that we're going to change the
structure of the CU database soon so that it is more closely coupled with
CantusDB, we'll figure out a more permanent solution then -- solving #891).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant