Does OpenRefine has to be used for MusicBrainz Reconciling?

[Yueqiao12Zhang]

For every database for single entity subjects, there is only one Wikidata link referencing the entity subject. For other objects in the same database, they do not have a Wikidata link, but they have a MusicBrainz ID. If we correctly process that MusicBrainz ID, we can refer to another database for that specific object. In that database, there will be a Wikidata link referencing that object.
In this case, we might not need to go to OpenRefine if we merge all the databases into a single CSV / TTL file. If we import this merged file into Virtuoso, every entity that has a Wikidata link will be involved inside a triple of its Wikidata page.

For example:

Label database:
id,label-code,sort-name,type-id,type,relations_wiki,name,country,annotation,disambiguation,area_iso-3166-2-codes,area_name,area_id,area_iso-3166-1-codes,area,genres_id,genres_name
https://musicbrainz.org/label/6c6c3bbb-76aa-4260-b842-f04936573c71,,Abstract Logix,,,https://www.wikidata.org/wiki/Q96371418,Abstract Logix,,,,US-NC,North Carolina,https://musicbrainz.org/area/d4ab49e7-1d25-45e2-8659-b147e0ea3684,,,,
...

This does not have a Wikidata link for the area, but we can track the area_id to another database.

Area database:
id,sort-name,relations_wiki,type,annotation,name,disambiguation,type-id,genres_name,genres_id
https://musicbrainz.org/area/b0f5611f-a6ad-470f-9da9-21ff76e5686d,Kashima,https://www.wikidata.org/wiki/Q684494,City,,Kashima,,6fd8f29a-3d0a-32fc-980d-ea697b69da78,,
...

This has the Wikidata link for the area.
If we merge these two databases together, then we do not have to reconcile the 'area' column in the label. If we merge all the databases together, then all the necessary data should be reconciled.

[Yueqiao12Zhang]

@ahankinson Is this why you think OpenRefine is not needed?

[Yueqiao12Zhang]

Also, if we have

id,sort-name,relations_wiki,type,annotation,name,disambiguation,type-id,genres_name,genres_id
https://musicbrainz.org/area/b0f5611f-a6ad-470f-9da9-21ff76e5686d,Kashima,https://www.wikidata.org/wiki/Q684494,City,,Kashima,,6fd8f29a-3d0a-32fc-980d-ea697b69da78,,

Is the 'type' attribute reconciled in MusicBrainz? In the web page, the 'type' and the 'City' is not a link.

[dchiller]

What does it mean? Aren't you saying that since most of the data is reconciled, we do not have to re-reconcile them again? If I do not do string concatenation, how do I get the MusicBrainz web link?

I think what @ahankinson is saying is that you don't need OpenRefine to do string concatenation. So if that's all we're really using OpenRefine for in the case of MusicBrainz, then we don't need to use OpenRefine.

Since in the Label database, we do not have a Wikidata link for Area, but we do have a Wikidata link for Area in the Area database, if we put these two databases in a same CSV, then we do not have to reconcile the Area column. If we do not put these two databases in a same CSV, then we would have to reconcile the Area column.

Correct...so we need to follow those references to get the Wikidata ID rather than reconciling them in OpenRefine. I would suggest we don't convert to csv in this case (my understanding was that the only purpose of the csv was to use it in OpenRefine, so if we don't use OpenRefine, we don't need the csv).

[dchiller]

Is the 'type' attribute reconciled in MusicBrainz? In the web page, the 'type' and the 'City' is not a link.

(a) Did you look?
(b) Do we need "City" to be reconciled?

[Yueqiao12Zhang]

Yes, it's full of cities, countries, etc.
If they do not need to be reconciled, then maybe we don't need OpenRefine at all.
To import to Virtuoso, it should be a CSV or an RDF graph, I think since we already have a CSV to RDF script, then going into CSV might be better?
JunJun is working on the R2RML script of Virtuoso that can properly import CSV into Virtuoso. If that works fine, then we can skip the RDF part.

[ahankinson]

So, if you do this:

$ curl -vvvv -L -H "Accept: application/ld+json" "https://musicbrainz.org/area/b0f5611f-a6ad-470f-9da9-21ff76e5686d"

The server will respond with some JSON-LD:

{"name":"Kashima",
  "sameAs":["http://sws.geonames.org/1859936/","http://sws.geonames.org/1859941/","http://www.wikidata.org/entity/Q684494","https://en.wikipedia.org/wiki/Kashima,_Saga"],
  "containedIn":
  {"@type":"AdministrativeArea",
    "@id":"http://musicbrainz.org/area/6ef6b0e0-7aca-4700-8d01-c14c2f36dbcc",
    "containedIn":
    {"name":"Japan",
      "@type":"Country",
      "@id":"http://musicbrainz.org/area/2db42837-c832-3c27-b4a3-08198f75693c"},
    "name":"Saga"},
  "@type":"City",
  "@id":"http://musicbrainz.org/area/b0f5611f-a6ad-470f-9da9-21ff76e5686d",
  "@context":"https://schema.org/docs/jsonldcontext.json"}

If you then take this to the JSON-LD playground and paste it in, you'll see that it gets expanded to RDF.

The point of the exercise is simply to point out that they have reconciled the types -- they're simple schema.org types. Given the @context, it will expand the @type: City to the following:

<http://musicbrainz.org/area/b0f5611f-a6ad-470f-9da9-21ff76e5686d> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://schema.org/City>

BTW, if you aren't already aware of it, LinkedBrainz was a thing:

https://wiki.musicbrainz.org/LinkedBrainz

It's not active now, but I think its legacy is this JSON-LD output.

[Yueqiao12Zhang]

How does MusicBrainz reconcile their entities? Is it inspected by human?

[ahankinson]

See my comment here: #52 (reply in thread)