Statistical Computing and Introduction to Data Science

GR5206 - Fall 2023

Learning outcomes

• Understand basic programming
  • Manipulate data with different structures
  • Control flow
  • Functions
• Explore data via visualization
• Study statistical concepts via simulations
• Automate tasks with programming
• Understand basic optimization

Prerequisites

• An introductory statistics class
  • Basic probability distributions (e.g. Gaussian, binomial distributions and their likelihoods)
  • Basic hypothesis testing (e.g. t-test)
  • Summary statistics
  • Histograms, boxplots, etc
• Multivariate calculus
  • Derivatives and functions
• Matrix operations and inverses of matrices
• You should be at least co-enrolled in a modeling class like regression

Textbooks and references

• Google!
• Python concept notes
• Python Data Science Handbook
• Basics only - Programming with Python by Software Carpentry
• LearningPython.org
• Data engineering references (not covered in this class):
  • Designing Data-Intensive Applications by Martin Kleppmann (available in NYPL)
  • System Design Interview - An Insider’s Guide by Alex Xu

Timeline

I reserve the right to change the ordering and the content for the course throughout the semester.

Date	Topic	Reference	Due
2023-09-08	Introduction + python as a calculator	- Python concepts 1, 2, 3, 4 - Software Carpentry - Python Fundamentals + Analyzing Patient Data - Python Data Science Handbook Chapter 2: Understanding Data Types in Python to The Basics of Numpy Arrays
2023-09-15	Numpy, objects, and subsetting AB Testing	Python concepts 5, 6, 8, 10, 11	Set up your jupyter notebook environment with the command line
2023-09-22	For-loop, if/else, working with files, AB testing assignment	Python concepts 7, 9, 21, 23	HW1 Due
2023-09-29	Pandas, summaries, and visualization Exploratory data analysis	Python concepts 12, 14, 15, 16
2023-10-06	Nested data and data wrangling Basic data engineering	Python concept 24 and all previous chapters	HW2 Due
2023-10-13	Regular expression and interacting with APIs	Python concepts 13, 17
2023-10-20	Designing data pipelines	All previous Python concepts	HW3 Due
2023-10-27	Midterm
2023-11-03	Data use cases, relational data, and SQL Data quality concepts and data engineering	Python concepts 20
2023-11-10	Modeling data “Data science methods”	Python concepts 18
2023-11-11	Optimization Objective functions	Python concepts 19	HW4 Due
2023-11-24	Thanksgiving NO CLASS
2023-12-01	Bootstrap, permutation, and other simulations Model evaluation
2023-12-08	Validation and what we don’t know		HW5 Due on 5/2
TBD	Final

Logistics

Class time: F 10:10am - 12:40pm, Location: 301 Uris

Teaching Team

See Ed for offiec hours

Grading

If your final grade is in [93-97), you will earn at least an A, [90-93) will earn at least an A-, [87-90) will earn at least a B+, etc. A grading curves may occur depending on the class performance but I will not curve downwards. I will not give out A+ for this class.

- Homeworks (25%)

• Late homeworks will receive 0 credit
• No make-up homeworks will be granted even if you registered late to the class
• If you want to learn how to use Google Colab, follow these instructions
• Please read these important things related to submitting homeworks on Ed

- Exams (70%)

• Midterms
• Final

- Participation (5%)

• In class participation
• Online question posting (non-private) and answers are all ways to achieve this
• I will reach out after the midterm if you are at risk of missing some points here.

Exam accomodations

In order to receive disability-related academic accommodations for this course, students must first be registered with their school Disability Services (DS) office. Detailed information is available online for both the Columbia and Barnard registration processes.

Refer to the appropriate website for information regarding deadlines, disability documentation requirements, and drop-in hours(Columbia)/intake session (Barnard).

For this course, students are not required to have testing forms or accommodation letters signed by faculty. However, students must do the following:

· The Instructor section of the form has already been completed and does not need to be signed by the professor.

· The student must complete the Student section of the form and submit the form to Disability Services.

· Master forms are available in the Disability Services office or online: https://health.columbia.edu/services/testing-accommodations

Expectations

• Take chances!
  • Break the code in lecture
• Give feedback in office hours or e-mail, don’t waste your time if you think a topic is not helpful
• Participate and ask questions, this is not easy!
  • In class: forecast what should be done, compare with what is happening, then summarize the difference.
  • Online: describe what you observe, describe what you expect, communicate clearly.
  • To each other: summarize the conversation to ensure you’re listening and think constructively before criticizing.
• Academic honesty: https://www.cs.columbia.edu/education/honesty/

Acknowledgement

A lot of these materials are based off the materials from Prof Thibault Vatter and Prof Gabriel Young.