How to handle modes and tonalities

[jacobdgm]

Looking at how to reconcile objects from CantusDB and SIMSSADB, @malajvan, @annamorphism and I noted that CantusDB has mode values that look like 2 and 4T etc., while SIMSSADB doesn't have a mode/tonailty property, but it does have "Final Pitch Class", with values like 7.0.

I imagine we want people to be able to search for "pieces in G" and to find results from SIMSSADB with Final Pitch Class of 7.0 and results from CantusDB with mode 7 or 8. Will these be converted/standardized by the contributors (i.e. the database managers who are providing the .csv files / database dumps), will we be converting/standardizing these fields, or will we not be standardizing things at all except during searching?

For the purposes of doing a test run with some chants from CantusDB, in addition to providing a mode column, I'll also provide a mode_name column where 1 is translated into dorian, 2 to hypodorian, and so on. While reconciling things, @malajvan and others can either take this new column into account or ignore it, depending on the answer to the question in the second paragraph of this comment.

[annamorphism]

The most relevant WikiData property for this seems to be "tonality", P826, which would presumably be the eventual title of the column.
Values this can have are often keys ("A flat major") but we also have some mode names:
Dorian (Q960729)
Mixolydian Q321814
Lydian Q686115
Hypodorian Q842766

as well as
minor key, Q113031008 (strangely, not "major")
and
"E" Q1622022 (a single use, for a bell).

We could add to these categories ("hypophrygian" exists, but it isn't labeled as a tonality) and then assign all of Cantus's 1 and 1T values to Q960729, 2 and 2T to Q842766, etc. (Musicological pedantry: mode 5, in practice, is almost always closer to major mode than "true" lydian, so translating the one to the other is sort of fudging the original information).
Simssa does not have a "mode" property as far as I can tell, though we could assign some more or less randomly for the purpose of exercise.

We will also, separately, have to create some sort of property for "final pitch" to encompass what Cantus calls "Finalis" and what Simssa calls "final pitch class". (there's something called "primary note", P7600, which is intended for ragas; modes don't seem to be defined this way on wikidata.) We would want both, in order to distinguish 4 from 4T and G-dorian from D-dorian.
Here what Simssa calls 7.0 and what Cantus calls G both become Q1440231 in some sort of reconciling step.

Then for Jacob's imagined query "Things in G" we presumably want to find all of those pieces with Q1440231 (which are likely to be in major/mixolydian/hypomixolydian or dorian/hypodorian/minor; not so likely to be phrygian or lydian). This would include all of Cantus's 7s and 8s (though not any 7Ts or 8Ts), and possibly some assorted 1T/2T/5T/6Ts (though no untransposed 1/2/3/4/5/6).

It bugs me somewhat that the reconciliation step carries with it the possibility of ending up with, say, all the "Regina caeli" pieces (a classic mode 5 melody) as "F Lydian" if they come from Cantus, while identical melodies in other databases will show up as "F major". Is there a way to avoid this?

[fujinaga]

This is a great example of what LinkedMusic project should be dealing with. We should be able to accommodate diverse users, e.g., for people who just want all the "pieces in G": major/minor, hypo-/mixolydian, etc. or people who want strictly "F Lydian" chant melodies. So, yes, we will find a way to achieve this.

[malajvan]

@annamorphism for the point about creating a property for "Final Pitch Class" (and other features that are too domain-specific to be put on Wikidata), I was thinking maybe we shouldn't create a new predicate in Wikidata but link it to another, more music-focused source. I'll make another discussion post about this.

[ahankinson]

or will we not be standardizing things at all except during searching?

In general that's the idea of a data lake... that you dump all of your data in one place in its native form, and then extract what you need from that data when you need to use it. This way you don't get caught up in trying to standardize everything before you can use it. Instead, you get something working with one set of data, and then work on improving it incrementally across all your data.

It also means your data is more re-usable, and less likely to be lossy. Conversion always results in some loss of precision. "Tonality" and "Modality", for example, are similar but not the same, so converting X into Y, or Y into X, means fudging the semantics a bit. If you start with the data in its native form, then that fudging happens at data use time, not at data storage time, which gives you more flexibility to design systems that can go either way without pre-determining a "standard" and then trying to fit square pegs into round holes just to get your data into the system.

I would recommend reading about the difference between "data lakes" and "data warehouses".

https://www.oreilly.com/library/view/the-enterprise-big/9781491931547/ch01.html
https://www.talend.com/resources/data-lake-vs-data-warehouse/#:~:text=Data%20lakes%20primarily%20store%20raw,to%20be%20text%20and%20numbers.

[annamorphism]

Related to the topic of modes, tonalities, etc. I noticed today that somebody proposed a "mode" property on Wikidata a few years ago but it was rejected as "too small" https://www.wikidata.org/wiki/Wikidata:Property_proposal/IFLA_mode_of_musical_work_ID.
I think @dchiller discussed the difficulty of adding properties to Wikidata, which this illustrates...
@kyrieb-ekat did you have some other findings about modes on Wikidata?

[dchiller]

If I'm interpreting the discussion correctly, it's actually a recommendation to create something slightly different that what we would use.

They wanted to create a mapping between the mode objects in Wikidata to this external source, not say "Musical piece A has mode 1."

I think we would have to answer a similar question if we proposed a "is in mode" property: are there objects on Wikidata that it would be relevant for? If we aren't adding pieces to wikidata, I wonder if there would be that many things on it that we would save have mode.

Right now, it looks like there are just a few musical works that "have tonality" of a certain mode (in particular, someone seems to have catalogued a few items of the genevan psalter this way). Here's "has tonality dorian" for example: