GU4205/5205 - Linear Regression Model

This class is designed for advanced undergraduates or master students who will need a solid mathematical understanding in regression to help their future learnings for more advanced models.

Expectations

- Learning outcomes

• Understand when linear regression is used for a descriptive, predictive, or prescriptive purpose
• Understand how the linear algebra behind linear regression can help us diagnose the behavior in regression
• Be able to run, diagnose, and improve your linear regression models using common methodologies
• Be able to construct counter examples for linear regression to fail
• Be able to simulate and confirm the mathematical derivations

- Your Job

• Come to class, bring your laptop, take chances!
  • Run through the code and derivations in lecture
  • Take notes that augment the lectures
• Give feedback in office hours or e-mail, I don’t want to waste your time.
• Participate and ask questions, this is not easy!
  • In class: forecast what should be done, compare with what is happening, then summarize the difference.
  • Online: describe what you observe, describe what you expect, communicate clearly.
  • To each other: summarize the conversation to ensure you’re listening and think constructively before criticizing.
• Academic honesty: https://www.cs.columbia.edu/education/honesty/

People

Instructor: Wayne Tai Lee: wtl2109

Teaching Assistant(s): TBD

Timeline

I reserve the right to change the ordering and the content for the course throughout the semester.

Date	Topic	Reference	Due
2019-09-04	Introduction, expectations, and transitions oh my!	Textbook Chapter 1.1 + 1.2 + 1.5 + 1.6.1
2019-09-09	Programming for simulations	Any R Tutorial Videos on For-Loop	Reading if you don’t know R: R Tutorial Videos
2019-09-11	From optimizing objective functions to modeling data generation	Textbook Chapter 1.3 + 1.6	Homework 1: Pre-requisites
2019-09-16	Purpose 1 - inference on the model	Textbook Chapter 2.3
2019-09-18	Purpose 2 - predictions	Textbook Chapter 2.4-2.6	Homework 2: Simulations
2019-09-23	Same estimation but different uncertainties	All previous references
2019-09-25	Diagnostics of SLR	Textbook Chapter 3.1-3.3
2019-09-30	Two extremes: likelihoods and various bootstrap methods	Textbook Chapter 1.8	Homework 3: Inference and Prediction
2019-10-02	Midterm review
2019-10-07	Midterm		Homework 4: Diagnostics
2019-10-09	Categorical variables and transformations	Textbook Chapter 14.1-14.3
2019-10-14	Linear algebra review and more	Textbook Chapter 5.1-5.8
2019-10-16	Multivariate linear regression	Textbook Chapter 6.1-6.7 (not 6.5)
2019-10-21	Simultaneous inference and Impact of adding a variable and interactions	Textbook Chapter 7.1, 8.2, 8.4
2019-10-23	Ecological correlation and WLS	Textbook Chapter 11.1	Homework 5: categorical data and bootstrap
2019-10-28	BLUE and James-Stein	Textbook Theorem 1.11
2019-10-30	F-test, ANOVA, and regression	Textbook Chapter 6.5	Homework 6: multivariate regression
2019-11-04	No Class - Election Day
2019-11-06	Significance changing when adding/deleting features
2019-11-11	Practice		Homework 7: WLS and Biased Estimators
2019-11-13	Midterm 2
2019-11-18	PCA regression
2019-11-20	Regularization
2019-11-25	Wrong models in linear regression + instrumental variables		Homework 8: simulating counter examples
2019-11-27	No Class
2019-12-02	Variable Selection & Delta Method
2019-12-04	Linear model workflow + Review
2019-12-09	Review and wrap-up!		Homework 9: Something easy
Final Schedule	Final Exam	You!

Logistics

Lectures: MW 2:40pm - 3:55pm, Location: 301 Pupin Laboratories Office Hours: Tu 2:00pm - 4:30pm, Location 610 Watson Hall (612 W 115th St 6F), led by Wayne Th 2:00pm - 4:30pm, Location 610 Watson Hall (612 W 115th St 6F), led by Wayne F 9:00am-12:00pm, Location 10th floor School of Social Works lounge area, led by Yian Huang Tu 12:30-3:20pm (sharp, not delayed), Location 10th floor School of Social Works lounge area, led by Navid Ardeshir

Grading

If your final grade is in [93-100], you will earn at least an A, [90-93) will earn at least an A-, [87-90) will earn at least a B+, etc. A grading curves may occur depending on the class performance but will not curve downwards. I may not give out A+

- Homeworks (15%)

• Late homeworks will receive 0 credit
• Homework solutions will exist in R
• Your lowest homework grade will be dropped
• No make-up homeworks will be granted even if you registered late to the class

  - Exams (80%)

• Midterms (25% each)
• Final (30%)

  - Participation (5%)

• This will be based on in-class online activities

  - Possible recovery

  Your final percentage grade will be the maximum of the following 4 values, the idea is that your final can cover part of your midterm grade:

• Homework * 0.15 + Midterm1 * 0.25 + Midterm2 * 0.25 + Final * 0.3 + Surveys * 0.05
• Homework * 0.15 + Midterm1 * 0.15 + Midterm2 * 0.15 + Final * 0.5 + Surveys * 0.05
• Homework * 0.15 + Midterm1 * 0.15 + Midterm2 * 0.25 + Final * 0.4 + Surveys * 0.05
• Homework * 0.15 + Midterm1 * 0.25 + Midterm2 * 0.15 + Final * 0.4 + Surveys * 0.05

Prerequisites

• Some familiarity with R or Python
  • Need to know how to write a loop and visualize data
• An introductory statistics class
  • Basic probability distributions (e.g. Gaussian, binomial distributions and their likelihoods)
  • Basic hypothesis testing (e.g. t-test)
  • Summary statistics
• Basic linear algebra

Textbooks / Supplies

• Applied Linear Regression Models 4th Edition by Kutner, Nachtscheim, and Neter
• Statistical Models Theory and Practice by David A. Freedman

Acknowledgement

A lot of these materials are based off the materials from Prof Ronald Neath and Prof Gabriel Young.