Text and Data Mining Tools Overview

Please see below for the University of Toronto's major Text and Data Mining (TDM) platforms and collections. For help regarding the platforms or collections below, please contact Digital Scholarship Services. Exception: please contact the Map & Data Library regarding LDC questions.

APIs

Application Programming Interfaces, or APIs, are a common way to access large amounts of data.

Collaborative Archive & Data Research Environment (CADRE)

CADRE is a cloud-based text and data mining service for large datasets. Over 220 million scientific publications and 1.7 billion citations can be queried and analyzed. No programming experience is required, but programming tools are available for more advanced analyses and extractions. The University of Toronto Libraries has purchased a membership to the CADRE platform.

Constellate

Constellate is a browser-based tool for creating datasets from collections, such as JSTOR, and then teaches and facilitates text analysis on those datasets. It has a number of tutorials, including well-documented Jupyter notebooks.

Gale Digital Scholar Lab (DSL)

The Gale Digital Scholar Lab is a platform that allows users to discover and create collections of digitized texts from the Gale Historical Collections, run a variety of statistical analyses on them, and visualize the resulting data.

HathiTrust Research Center (HTRC)

The University of Toronto is a member of the HathiTrust Research Center, which allows researchers to run text analysis scripts on the HathiTrust corpus.

Linguistic Data Consortium (LDC)

The University of Toronto is a subscriber to the Linguistic Data Consortium which licenses language corpora and other language resources.

ProQuest TDM Studio

TDM Studio is a web platform for running text analyses on the ProQuest datasets, including ProQuest Dissertations & Theses and the New York Times. TDM Studio has two components: 

  • Visualizations, which allow for working with 10 000 results and are entirely in a point-and-click interface with data visualization
  • The Workbench, for researchers and their teams coding with R or Python in Jupyter notebooks

If you wish to have access to both components, you must request them separately.

Web of Science (WoS)

The Web of Science (WoS) Raw Data Product includes metadata from over 12,500 journals from around the world in over 250 Science, Social Science and Humanities disciplines. Conference proceedings and book data are also included. Data are available from 1900 and currently include over 63 million article records and 1 billion cited references (as of 2018).