UN2103 - Applied Linear Regression Model

This class introduces linear regression as a foundational tool for prediction and inference with an emphasis on simulations and challenges faced in application.

Expectations

- Learning outcomes

• Transition from learning via imitation to working backwards from the question
• Understand when to use linear regression for data mining, prediction, or inference
• Be able to articulate counter examples for linear regression to “fail”
• Be able to fit, evaluate, and improve models
• Learn to simulate and confirm basic mathematical results

- Your Job

• Come to class, bring your laptop, take chances!
  • Run through the code and derivations in each lecture
  • Take summarized notes that augment the lectures
• Give feedback in office hours or email, I don’t want to waste your time
• Participate and ask questions, this is not easy!
  • In class: forecast what should be done, compare with what is happening, then summarize the difference.
  • Online: describe what you observe, describe what you expect, communicate clearly.
  • To each other: summarize the conversation to ensure you’re listening and think constructively before criticizing.
• Academic honesty: https://www.cs.columbia.edu/education/honesty/

How to get help

• Office hours:
  • Mon/Wed 10-Noon with Wayne Tai Lee (wtl2109) at 324 Uris Hall (except 11/1)
  • Th 10-noon with Irene Chang (bnc2119) at 201 Uris Hall
• Statistics daily help room

Timeline

I reserve the right to change the ordering and the content for the course throughout the semester.

Date	Topic	Reference	Due
2023-09-06	Introduction, expectations, and transitions oh my!
2023-09-11	Intro Stat Review Slides	Intro Stat Review Notes
2023-09-13	Quick R crash course (if you’ve taken 2102 you can skip)	How to simulate the LLN	Homework 0
2023-09-18	Reviewing Hypothesis Testing with R Simulations and code template	How to simulate the LLN
2023-09-20	Deriving simple linear regression and code template	Text 2.1.1
2023-09-25	Linking regression coefficients to the data generation process and code template	Text 2.7.1 + 2.7.2	Homework 1
2023-09-27	Linking math to simulations and code template	Text 2.7.2 + 2.7.3	Read Global Evidence on Economic Preferences until Chapter IV.A
2023-10-02	Diagnostics of SLR - inference; if time allows then Properties of the regression coefficients	Text 2.7.2 + 2.7.3 + 3.1
2023-10-04	code template for diagnostics; Diagnostics of SLR - prediction;		Homework 2; Read Statistical Models and Shoe Leather
2023-10-09	Review	Text 2.3
2023-10-11	Midterm 1
2023-10-16	Inferring the true line and Predicting new data points; code template	Text 2.4
2023-10-18	Bootstrapping and code template	Stanford Notes
2023-10-23	Interactions, polynomials, and categorical variables for X: - part1 on categorical X - part2 on polynomials - part3 on interactions; code_template		Homework 3
2023-10-25	Issues with multiple variables and lecture - can bad features hurt and lecture - collinearity; code template	Text 2.6 + Text 5.2
2023-10-30	Simultaneous inference on coefficients; code template
2023-11-01	Cross Validation and lecture & Logistic regression with vimeo link; code template	Text 4.1	Homework 4
2023-11-06	US ELECTIONS - NO CLASS
2023-11-08	Guest Speaker: John Andrew Chwe from Psychology & Review		Trustworthiness of crowds is gleaned in half a second;
2023-11-13	DAGs: Changing significance when adding/deleting features with vimeo link; simulation code
2023-11-15	Review		Homework 5
2023-11-20	Midterm 2
2023-11-22	THANKSGIVING HOLIDAY - NO CLASS
2023-11-27	Midterm retro & WLS with code template	Special case of chapter 9
2023-11-29	Mixed models, code template	Chapter 10
2023-12-04	Paper discussion		Read Why do people stay poor?; Homework 6
2023-12-06	Controlling for variables code template
2023-12-11	What we know and don’t know		Homework 7 (due 12/13)
2023-12-15	Final project or exam

Logistics

Lectures: MW 2:40pm - 3:55pm, Location: 517 Hamilton Hall

Grading

If your final grade is in [93-100], you will earn at least an A, [90-93) will earn at least an A-, [87-90) will earn at least a B+, etc. A grading curves may occur depending on the class performance but will not curve downwards. I may not give out A+

- Homeworks (15%)

• Late homeworks will receive 0 credit
• Homework solutions will exist in R
• Your lowest homework grade will be dropped
• No make-up homeworks will be granted even if you registered late to the class
• Please export all homeworks in PDF files following these instructions
• If you are not comfortable with LaTex, you can “write-in” the math afterwards.

  - Exams (80%)

• Midterms (15% for midterm 1 and 30% for Midterm 2)
• Final Exam or Project (35%)

  - Participation (5%)

• This will be based on in-class online activities
• You will receive the full 5% if you obtain 75% of this.

Prerequisites

• An introductory statistics class
  • Basic probability distributions (e.g. Gaussian, binomial distributions and their likelihoods)
  • Basic hypothesis testing (e.g. t-test)
  • Properties of summary statistics
• Co-requisite: some familiarity with computing or UN2102 Applied Statistical Computing

Textbooks / Supplies

• I’m hoping to write up some notes here
• Free text: A modern approach to regression with R by Simon J. Sheather, available via CLIO
• For mathematically curious students: Statistical models: theory and practice

Acknowledgement

A lot of these materials are based off the materials from Prof Ronald Neath and Prof Gabriel Young.