Useful sources

[jure]

I'll use this issue to compile a list of useful sources for getting citations of code.

These sources must have full-text search, as this significantly increases the sensitivity of the search.

Having an API is optional, scraping can also be useful.

[sckott]

Springer

Has some APIs http://dev.springer.com/, but in my experience have been a mess, and they don't respond to bug reports, etc.

[jure]

HighWire

Corpus: 4 million full-text articles (https://en.wikipedia.org/wiki/HighWire_Press)

Example of a search: http://highwire.stanford.edu/cgi/searchresults?RESULTFORMAT=1&hits=150&sortspec=match&sendit=Update+display&tyear=2014&resourcetype=1&tmonth=Apr&fulltext=vegan%3A+community+ecology+package&andorexacttitleabs=and&src=hw&fmonth=Jan&y=8&fyear=1753&x=23&fdatedef=1+January+1753&andorexacttitle=and&searchsubmit=redo&andorexactfulltext=phrase&tdatedef=5+Apr+2014

[jure]

Oxford Journals

Corpus: ?

Example of search: http://services.oxfordjournals.org/cgi/searchresults?pubdate_year=&volume=&firstpage=&DOI=&title=&andorexacttitle=and&titleabstract=&andorexacttitleabs=and&fulltext=%22vegan%3A+community+ecology+package%22&andorexactfulltext=phrase&author1=&author2=&fmonth=Nov&fyear=1849&tmonth=May&tyear=2014&fdatedef=3+November+1849&tdatedef=21+May+2014&flag=&RESULTFORMAT=1&hits=150&sortspec=relevance&x=43&y=5

[jure]

Elsevier

Corpus: 7 million articles

Example of search: http://api.elsevier.com/content/search/index:SCIDIR?query="vegan: community ecology package"&APIkey=

API key needed, also notorious for its licensing of this data, so perhaps we should stay clear. It's a very good resource in terms of scope though!

[jure]

Microsoft Academic Search

Corpus: 10 million articles?

Example of search: http://academic.research.microsoft.com/Search?query=r-project.org%2fweb

I don't think MAS has 10 million full text indexed articles. Has an API.

[sckott]

@jure I heard Microsoft was deprecating that API, but perhaps I'm wrong?

[sckott]

@jure I think @hpiwowar has experiencing working with the Elsevier API?

[jure]

You could be right, @sckott, looks like the last update to MAS in general was in January 2013. That's aeons ago.

It would be awesome if @hpiwowar pitched in, although with Elsevier it's not really using the API that's the question, it's their strict licensing that is bothersome. But then again, licensing is always an issue.

[sckott]

@jure curious if in this

These sources must have full-text search, as this significantly increases the sensitivity of the search.

you really need full text. Couldn't you just have the list of references from the paper? Wonder if there are some APIs that provide that in their metadata?

[jure]

From what I've seen, APIs that have reference lists also have full-text. I agree that if we can get reference lists for a paper, that would be a fantastic start and full-text is not needed. But again, usually these two are coupled.

[pietsch]

BASE

Just use the search API of BASE (Bielefeld Academic Search Engine): http://www.base-search.net/about/en/about_develop.php?menu=2
BASE indexes >60 million publications, many of them open access.