Bernhard Rieder

Research software

Over the years I have written quite a number of research tools, mainly for the data-driven analysis of social media platforms. My main goal is to gain a deeper understanding of the logics embedded in these platforms and their APIs, and writing research tools is an excellent way to pursue this kind of exploration. Nothing beats first-hand experience.

Development and maintenance of some of these tools are funded by the Dutch Platform Digitale Infrastructuur Social Science and Humanities as part of the CAT4SMR project.

Support. Most tools have basic descriptions or FAQ sections, and there are instruction videos on my YouTube channel. Questions, feature requests, and high-quality bug reports are much appreciated, but only if filed via GitHub. Please do not use email or social media. If you have never written a bug report, please read this piece, before submitting.

Current tools

YouTube Data Tools

Modules for extracting data from YouTube via the YouTube API v3: channel and video lists, video comments, channel networks, and co-commenting networks. Developed and supported as part of the CAT4SMR project.

Launch toolSource codeIntro video

Tutorials

Archive

Older tools and scripts. They are no longer maintained, and several stopped working when platforms closed their APIs. The source code stays available for reference.

Data extraction

  • DMI-TCAT no longer works The Digital Methods Initiative Twitter Capture and Analysis Toolset – developed with Erik Borra and Emile den Tex – retrieves and collects tweets and provides modules to analyze tweet collections. Requires server installation. Source
  • Netvizz no longer works Extracted data from Facebook groups and pages for research purposes from 2010 onward. Facebook removed page data access on 4 September 2019 and Netvizz is no longer functional. Video
  • TumblrTool Gets posts tagged with a specific term and creates tabular statistics and co-tag networks. LaunchSource
  • Visual Tagnet Explorer no longer works Got media from Instagram tagged with a term or posted around a location and created statistics and co-tag networks. No longer works since Instagram changed its API rules. SourceVideo

Analysis

  • RankFlow A visualization tool for analyzing changes in ordered lists (e.g. rankings) over time. LaunchSource
  • LineMiner Analyzes textual data stored in timestamped lines of text, with fast text search and some statistical and visual text analysis. Work in progress. Requires server installation. Source
  • SimilarityNet Calculates cosine similarity between lists of quantified variables (feature vectors) and outputs a similarity network. LaunchSource
  • Tag Cloud HTML Generator Turns tags and values in Wordle format into an HTML tag cloud or tag list. Launch
  • Textanalysis A small text analysis tool for emoji statistics and bigram/collocation extraction. Launch

Scripts

  • MemeSpector A PHP script for Google's Vision API: takes a CSV/TSV with image URLs, sends the images to the API and writes the detected annotations back into the file. Source
  • PiperScraper PHP scripts to scrape and analyze Pipermail list archives. Source
  • Reddit Tools PHP command line scripts that grab data from Reddit and turn it into CSV files. Source
  • Textprocessing Basic Python scripts that connect common data formats with more complex forms of text processing. Source
  • YouTube Transcript Scraper Uses browser automation to click through the YouTube interface and download transcripts. A basic example for starting with Selenium. Source

Fun stuff

  • Spotify Artist Network no longer works Created networks of related artists, based on data from Spotify.