Stanford ~ Programming in Journalism

Sundry notes and code bits for Stanford's Programming in Journalism class (COMM 177P/277P).

Important links

• Syllabus
• Assignments and Grading
• Bookshelf - recommended books, tutorials, cheatsheets, etc.
• Code solutions - A private repo containing solutions to class exercises.
• Datakit - overview and details on install and usage
• Getting Help - Resources and strategies for finding help.
• Glossary - technical terms used in class
• Python - overview, tutorials, etc.
• Technical setup and FAQ - recommended and required software (all free)
• Workflow advice - working on the command line, etc.

Class notes

Week 1

Day 1 - Course Intro

• Course overview
• Discuss the history of code and data analysis in journalism
• Software install party (w00t!!)

Day 2 - The Owl, Problem Solving, and the Unix Workbench

• The Owl, Problem Solving, and the Unix Workbench
• Bash intro
• Bash drill
• Software installs, part deux (config and Python install)
• Assignment 0 - Unix practice and failed banks script

Week 2

Day 3 - MLK Day

No class.

Day 4 - Python Intro

• Bash questions?
• Python overview and coding contexts
• Python Syntax Crash Course
• Assignment 1 - Python reading/practice and code challenge

Week 3

Day 5 - Python Intro Part Deux

• Assignment 1 questions?
• Dictionaries crash course
• Reading and writing text files
• Assignment 2 - Python lists/dicts and DataKit overview

Day 6 - Automating Workflows and DataKit

• Automating workflows
• DataKit overview
• Complete Tech Setup configuration and install DataKit
• Assignment 3 - Practical Python skills and Failed Banks in Python

Week 4

Day 7 - APIs and the News

• APIs and the News overview and presentation
• Working with APIs - As of Feb. 1, 2021, certain unauthenticated OpenCorporates API endpoints were disabled due to excessive bot usage, so not all portions of the tutorial are functional.
• Quakebot exercise - hands-on practice with a JSON feed

Day 8 - Quakebot and Python Functions

• Quakebot exercise - hands-on practice with JSON feed
• The Art of Writing Functions
• Assignment 4 - Study Python functions, sorting and web basics
• Assignment 5 - Senate compromisers Python script

Week 5

Day 9 - Web scraping for the news

• Web scraping for the news - a high-level overview of web scraping in a news context.
• Web scraping 101 - how to dissect a website, various obstacles to scraping and strategies for overcoming them
• Web scraping exercises - some websites to challenge your newfound scraping skills
• Web scraping resources - tutorials, etc.
• Assignment 6 - Web Scraping Ramp-up

Day 10 - API Services and Data Pipelines with Modules

• API Services
• Refinitiv Entity Extraction
• Data Pipelines with Modules
• Project: FDA Recall Entities - Scrape FDA recalls and extract entities using the OpenCalais/Refinitiv API

Week 6

Day 11 - Prez Day

No class.

Day 12 - Code review and workshop

• Review and discuss solutions to recent assignments
• Strategies for gaining visibility into code while you work
• Assignment 7 - Data Journalism intro readings

Week 7

Day 13 - Data Journalism Overview and Final Project Preview

• Data Journalism - A Very Brief Tour
• Data Journalism in the wild - Dilcia Mercedes and Justin Mayo
• Preview of Final Project - SF Data Analysis and begin working on story ideas and data sleuthing/vetting.
• Assignment 8 - Prepare and present final project story ideas

Day 14 - The DJ Process and Intro to Data Analysis

• Whirlwind Tour of the Data Journalism Process
• Data Analysis with Jupyter and pandas
• Assignment 9 - First Python Notebook

Week 8

Day 15 - Final Project Story Idea Presentations

• Present story ideas for the final project as part of Assignment 8.

Day 16 - Final Project Workshop

• Overview of requirements for final project
• One-on-one technical consults