cs7616 - pattern recognition - introductionhic/cs7616/pdf/lecture1.pdf · cs7616 - pattern...

35
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia Institute of Technology, Atlanta, GA 30332-0280 [email protected] Henrik I Christensen (RIM@GT) CS7616 - PR 1 / 35

Upload: others

Post on 26-May-2020

31 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

CS7616 - Pattern Recognition - Introduction

Henrik I Christensen

Robotics & Intelligent Machines @ GTGeorgia Institute of Technology,

Atlanta, GA [email protected]

Henrik I Christensen (RIM@GT) CS7616 - PR 1 / 35

Page 2: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Outline

1 Introduction

2 Objective / Motivation

3 Schedule / Structure

4 Homework / Exercises

5 Material ...

6 Background examples

7 Questions

Henrik I Christensen (RIM@GT) CS7616 - PR 2 / 35

Page 3: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Introduction

Welcome to CS7616

Pattern Recognition

Today:

Outline of the course - Objective / MotivationSchedule of lecturesStyle of the courseExercises / ProjectsMaterial to be used in the course

Henrik I Christensen (RIM@GT) CS7616 - PR 3 / 35

Page 4: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Information

Class website:http://www.cc.gatech.edu/∼hic/CS7616Schedule, Material, Slide copies, General Information

T-Square - Usual stuff, announcements, ...

Slides - PDF copy will be posted after class with summary

Piazza - You will receive an invitation for the class forumUse it for general questions / discussions

Henrik I Christensen (RIM@GT) CS7616 - PR 4 / 35

Page 5: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Staffing

Henrik I ChristensenLecturer

Sidd Choudhary, TA Steven Hickson, TA Ruffin White, TA

Henrik I Christensen (RIM@GT) CS7616 - PR 5 / 35

Page 6: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Outline

1 Introduction

2 Objective / Motivation

3 Schedule / Structure

4 Homework / Exercises

5 Material ...

6 Background examples

7 Questions

Henrik I Christensen (RIM@GT) CS7616 - PR 6 / 35

Page 7: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Objective

Get a solid knowledge of key methods in pattern recognition

Discuss state of the art methods / techniques in pattern recognition

Explore a few representative data sets that illustrate use of patternrecognition

Explore increasingly complex methods over the semester

This is not a general machine learning course

Henrik I Christensen (RIM@GT) CS7616 - PR 7 / 35

Page 8: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Motivation

PR is used everywhere in daily lives

Henrik I Christensen (RIM@GT) CS7616 - PR 8 / 35

Page 9: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Speech Recognition

Henrik I Christensen (RIM@GT) CS7616 - PR 9 / 35

Page 10: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Financial trading

Henrik I Christensen (RIM@GT) CS7616 - PR 10 / 35

Page 11: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Medical Diagnostics

Henrik I Christensen (RIM@GT) CS7616 - PR 11 / 35

Page 12: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Face Recognition

Henrik I Christensen (RIM@GT) CS7616 - PR 12 / 35

Page 13: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Scene Labeling

Henrik I Christensen (RIM@GT) CS7616 - PR 13 / 35

Page 14: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Netflix Movie Recommendation

Henrik I Christensen (RIM@GT) CS7616 - PR 14 / 35

Page 15: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Generic Problem Structure

Feature Extraction Classification Decision

Henrik I Christensen (RIM@GT) CS7616 - PR 15 / 35

Page 16: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Outline

1 Introduction

2 Objective / Motivation

3 Schedule / Structure

4 Homework / Exercises

5 Material ...

6 Background examples

7 Questions

Henrik I Christensen (RIM@GT) CS7616 - PR 16 / 35

Page 17: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Structure

A mixture of foundational lectures and

Group discussions of influential/current papers

We will divide class into 4 groups for smaller discussionsEvery student is expected to present 1 paper during term as part of thegroup discussions

Large group lectures are a challenge for in-depth discussions

Henrik I Christensen (RIM@GT) CS7616 - PR 17 / 35

Page 18: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Lecture topics

1 Bayes Decision Theory

2 Linear Methods for Classification

3 Sub-space Methods

4 Ensemble Methods

5 Hidden Markov Models

6 Prototype/memory based methods

7 Kernels and other tricks

8 Tree based techniques

9 Large Margin Classifiers

10 Deep Learning

Henrik I Christensen (RIM@GT) CS7616 - PR 18 / 35

Page 19: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Discussion Sessions

Discuss two papers per class:

Each paper:

Student presentation of paper ≈ 15 minutes introGroup: What are the main lessons/key insight from the paperGroup: How could it be improved / what would you do differently?TA/Lecturer: guide discussion / presentation

Henrik I Christensen (RIM@GT) CS7616 - PR 19 / 35

Page 20: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Outline

1 Introduction

2 Objective / Motivation

3 Schedule / Structure

4 Homework / Exercises

5 Material ...

6 Background examples

7 Questions

Henrik I Christensen (RIM@GT) CS7616 - PR 20 / 35

Page 21: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Homework

Leverage of a set of datasets - varying in complexity, ...

A homework assignment roughly every month

4 assignments in total

First three will use the common datasets (Gaussian / Ensemble /Temporal)

Final homework - option to use your own dataset - large margin /deep learning

Henrik I Christensen (RIM@GT) CS7616 - PR 21 / 35

Page 22: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Credit / Grading

45% Homeworks 1-3

25% Homework 4

25% Class Presentation / Discussions

5% Class participation

There will no final exam!

Henrik I Christensen (RIM@GT) CS7616 - PR 22 / 35

Page 23: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Submission of material

Please submit home work on time.

Late submissions will be 75% for 1 day late, 50% for 2 days late andthen 25% after that

You can ask for permission with a good motivation, but have to do itwell ahead of time (not an hour before!)

Do not expect that we are online the last hour before a deadline.Unfair to the TAs and others.

Henrik I Christensen (RIM@GT) CS7616 - PR 23 / 35

Page 24: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Outline

1 Introduction

2 Objective / Motivation

3 Schedule / Structure

4 Homework / Exercises

5 Material ...

6 Background examples

7 Questions

Henrik I Christensen (RIM@GT) CS7616 - PR 24 / 35

Page 25: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Book 1: Elements of Statistical Learning

Main textbookElements of Statistical LearningT. Hastie, R. Tibshirani & J.FreiedmanSpringer Verlag, 2nd Edition, 2009http://www-stat.stanford.edu/

~tibs/ElemStatLearn

Henrik I Christensen (RIM@GT) CS7616 - PR 25 / 35

Page 26: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Book 2: Machine Learning - Kevin Murphy

Machine LearningK. MurphyMIT-Press, 2013

Henrik I Christensen (RIM@GT) CS7616 - PR 26 / 35

Page 27: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Book 3: Duda, Hart and Stork

Pattern ClassificationR. O. Duda, P. E. Hart and D. G.StorkWiley Interscience, 2nd, 2001, ISBN0-471-05669-3

Henrik I Christensen (RIM@GT) CS7616 - PR 27 / 35

Page 28: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Software

You can use Matlab or Python - we will try to support both

Some demonstrations using Matlab / K. Murphy Toolkit

https://github.com/probml/pmtk3

Some examples using Scikit-Learn Toolkit

http://scikit-learn.org

Still try to finalize 2-3 datasets for homework

Henrik I Christensen (RIM@GT) CS7616 - PR 28 / 35

Page 29: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Outline

1 Introduction

2 Objective / Motivation

3 Schedule / Structure

4 Homework / Exercises

5 Material ...

6 Background examples

7 Questions

Henrik I Christensen (RIM@GT) CS7616 - PR 29 / 35

Page 30: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Model based recognition

Google has 2.5 million objects inthe 3D object warehouse

Can we use these for recognitionof objects?

Can we provide context forobject recognition?

Henrik I Christensen (RIM@GT) CS7616 - PR 30 / 35

Page 31: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Gesture based recognition

Tracking of hands for person forrobot interaction

Color classification of hands andhead of user

Tracking of objects usingKalman filter

HMM based recognition ofgestures

Henrik I Christensen (RIM@GT) CS7616 - PR 31 / 35

Page 32: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Recognition of daily activities

Images of standard objects torecognize daily activities

Example application forassistance to people withmemory challenges

Using Deep Learning forRecognition of situations

Henrik I Christensen (RIM@GT) CS7616 - PR 32 / 35

Page 33: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Outline

1 Introduction

2 Objective / Motivation

3 Schedule / Structure

4 Homework / Exercises

5 Material ...

6 Background examples

7 Questions

Henrik I Christensen (RIM@GT) CS7616 - PR 33 / 35

Page 34: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Next Lecture

Thursday - Bayes Decision Theory

DHS: Chapter 2 (2.1-2.6)

We will provide the initial list of papers for class discussions

Discuss the datasets for home work

Henrik I Christensen (RIM@GT) CS7616 - PR 34 / 35

Page 35: CS7616 - Pattern Recognition - Introductionhic/CS7616/pdf/lecture1.pdf · CS7616 - Pattern Recognition - Introduction Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia

Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions

Questions?

Questions?

Henrik I Christensen (RIM@GT) CS7616 - PR 35 / 35