This is a fork of the ESTC repository, focusing on development of features to be demonstrated in the DR2 conference in Turin in January 16-18th 2017. Development will cease after the conference, and relevant parts will be merged back into the original repository.
This is an algorithmic toolkit for R, designed for transparent quantitative analysis of the British Library English Short Title Catalogue (ESTC) data collection. The package is under active, open development; the tools, analysis, and documentation are preliminary and constantly updated. Your contributions, bug reports and feedback are welcome (but please, don't ask us if we know who is Livy if he is temporarily on discarded author list. Serious data science takes time)!
An overview of knowledge production between 1477-1800 based on the ESTC metadata on almost half million documents is provided in the following automatically generated files. This is work in progress. The analyses may contain errors but we provide the complete source code and results already at this preliminary stage to improve the transparency of our work.
- Overall summary of the ESTC catalogue
- Document sizes across time
- Edinburgh summaries
- North America summaries
- Princeton summaries
- Scotland summaries
- Page count summaries
- Various summary tables of the data.
The steps to reproduce these summaries from the raw data are fully described at the tutorial page. This includes several steps from raw data extraction to harmonizing the textual annotation fields, preprocessing the information, and carrying out statistical analysis and visualization. Whereas this package focuses on the ESTC data, it utilizes additional tools from the more generic bibliographica and many other R packages listed in the DESCRIPTION file. The ESTC raw data is confidential and available only on a separate agreement, so we can only publish statistical summaries and our own analysis source code at this site. The process is fully automated, and can be easily repeated with different subsets of the data.
We have frozen the analysis for already published material:
Authors: Leo Lahti, Ville Vaara, Mikko Tolonen. Part of rOpenGov.
You are welcome to:
- submit suggestions and bug reports
- send a pull request (we will acknowledge contributions)
- join IRC at !ropengov@Freenode
- contact or follow us
