authorisation to republish metadata from external sources

[pvgenuchten]

This question came up when harvesting from the impact4soil platform

The Orcasa team discussed with some partners if they can harvest and republish their content within Impact4soil. Orcasa suggests Soilwise to connect to those partners again to understand if they are interested to share their content also with soilwise/euso. Or to harvest the content directly from the remote partner.

Some other aspects relevant to this topic:

• Does google ask to all websites, if they can index their content? They turn it around, they say: if you don't want to be indexed, provide a robots.txt file to indicate which aspects (not) to be indexed. Impact4soil currently has no robots.txt file. Also consider that SWR currently does not read and acts on robots.txt yet. On top of that, soilwise might need to verify the robots.txt of the external repository to understand its crawling policy?
• As a platform it would be good to advertise what license applies to metadata, usually an open one (cc-by) is endorsed
• the 'soilwise bot' advertises itself in the client header of each request, so administrators will notice that a system is harvested by Soilwise, and can even filter traffic on that header.
• Soilwise should never use a password to harvest private content and then publish it publicly.
• Indexation may be a less intrusive word to use, then harvesting.