back to Computer Vision (CSE 40535/60535)

Materials available here were prepared for students of the University of Notre Dame attending the course in Fall 2017. If you find these notes helpful in your work, please provide the following reference:

"Adam Czajka, Computer Vision (CSE 40535/60535), Lecture Notes, Fall 2017, available online http://zbum.ia.pw.edu.pl/EN/node/77"

Note: only basic materials that are publicly available are presented here. Students can find complete materials (including quizzes, progress notes, grades, etc.) on SAKAI.

Progress

PART I: Introduction

Wed. 8/23/2017: Course structure and syllabus

Wed. 8/23/2017 and Fri. 8/25/2017: Definition of computer vision (slides)
Notion of computer vision, well- and ill-posed problems, inverse optics. Short history. Example applications. General pipeline, image acquisition, processing and analysis. Computer graphics.

Mon. 8/28/2017: Human vision system (slides)
Structure of the human eye. Receptors, rods and cones. Information processing, retina, neurons. Image formation, optics of the eye. Higher-level processing, optical illusions.

PART II: Digital image analysis

Wed. 8/30/2017 and Fri. 9/1/2017: Image acquisition (slides)
Measurement of the electromagnetic energy, Reflectance Distribution Function. Digital camera, CCD sensors, performance factors. Pixel, spatial and temporal resolution. Pixels and pixel values, lookup table.

Mon. 9/4/2017: Color imaging (slides)
How we perceive colors? Additive vs. subtractive color mixing. CIE 1931 chromaticity diagram. Tri-chromatic vs. color-opponent theories. Color spaces, RGB, HSV, HSL. Color camera.

Wed. 9/6/2017 and Fri. 9/8/2017: Point operators (slides and MATLAB codes)
Linear, affine and non-linear operators. Histogram equalization. Image intensity quantization, image thresholding, strategies for threshold selection.

Mon. 9/11/2017: Practical 1: Detection of color objects in live stream (Python codes)

Wed. 9/13/2017, Fri. 9/15/2017 and Mon. 9/18/2017: Neighborhood operators (slides and MATLAB codes)
Linear and non-linear filtering, convolution vs. correlation. Fourier transform, convolution theorem.

Wed. 9/20/2017: Edge detection (slides and MATLAB codes)
Detection of discontinuities, polarity and orientation of edges. Importance of zero-crossings, Logan's theorem. Multi-scale analysis, causality. Morphological edge detection. Edge thinning, non-maximum suppression, Canny thinning.

Fri. 9/22/2017 and Mon. 9/25/2017: Segmentation (slides and MATLAB codes)
Detection of lines, Hough Transform for lines and circles, Generalized Hough Transform, RANSAC. Snakes, inner and outer energy, contour evolution. Region-based segmentation, split-and-merge, quad-tree representation. Watersheds.

Wed. 9/27/2017 and Fri. 9/29/2017: Geometric transformations in 2D (slides and MATLAB codes)
Interpolation and decimation. Euclidean, similarity, affine and projective transformations. Properties of 2D transformations. Polar coordinates. Forward vs. inverse warping.

PART III: Pattern recognition

Mon. 10/2/2017 --> Fri. 10/13/2017: Image features (slides, Matlab and Python codes)
Geometric vs. intensity features. Global shape properties, geometric moments, Hu-moments. Wavelets, Gabor wavelets, uncertainty principle, applications. Other feature extractors: DoG, HOG, BSIF, LBP. Detection of keypoints, SIFT, SURF. Image stitching as an example application of sparse features: extraction and matching of keypoints, estimation of 2D geometrical deformation, RANSAC, use of cylindrical and spherical coordinates.

Mon. 10/16/2017 -> Mon. 10/20/2017: Fall break

Mon. 10/23/2017: Practical 2: Object detection and classification (Matlab codes and instructions)

Wed. 10/25/2017 -> Mon. 10/30/2017: Classification (slides)
Bayesian inference in vision, Bayes rule, calculating posterior probabilities given prior knowledge and empirical data, Bayesian classifier, optimal classification error. Development of classifiers based on Bayes rule, QDA, LDA, minimum distance classifier (dMIN), k nearest neighbors classifier (kNN). Linear and non-linear Support Vector Machines (SVM), kernel trick, extentions to multiple-class classification.

Wed. 11/1/2017: Practical 3: Image stitching and image warping (Matlab codes and instructions)

Fri. 11/3/2017: Feature selection (slides)
Selection of feature subsets vs feature space transformations. Sequential Forward and Backward Selection (SFS, SBS), compensating for lack of feature re-evaluation (LRS and BDS methods). Use of mutual information, minimum Redundancy Maximum Relevance (mRMR). Linear transformations, PCA, LDA.

Mon. 11/6/2017 - Fri. 11/17/2017: Deep learning in computer vision (slides)
Elements of neural network models: neuron/node, activation function, layer, multi-layer perceptron. Building prior knowledge into neural networks, receptive fields, weight sharing. Example structures useful in Computer Vision: convolutional neural networks, recurrent neural networks. Challenges in learning of deep structures, hands-on recommendations. Applications of deep learning in object recognition, semantic segmentation, and scene captioning.

Wed. 11/15/2017: Practical 4: CNN-based object recognition (Python codes and instructions).

Mon. 11/20/2017, Mon. 11/27/2017: Performance evaluation (slides)
Point and interval estimation, hypothesis testing. Classification error rates and curves, ROC, DET, CMC, confusion matrix. Efficient use of data samples, training, validation and testing sets, cross validation techniques.

Wed. 11/22/2017 -> Sun. 11/26/2017: Thanksgiving break

Wed. 11/29/2017 -> Mon. 12/4/2017: 3D reconstruction (slides)
Stereo vision as a method of 3D scene reconstruction. Correspondence detection, epipolar lines, epipolar constraint. Stereo image normalization, base distance, dispersion and depth. Solving the correspondence problem, 1D optical flow, pixel block matching, use of dynamic programming for scan-line matching. Other 3D reconstruction methods, laser scanning, time of light, photogrammetry, use of structured light, shape-from-X (shading, texture, focus, deep learning).

Wed. 12/6/2017: Guest lecture: Dr. Daniel Moreira, "Sensitive-Video Analysis"
Abstract: Sensitive videos that may be inadequate to some audiences (e.g., pornography and violence, towards underages) are constantly being shared over the Internet. Employing humans for filtering them is daunting. The huge amount of data and the tediousness of the task ask for computer-aided sensitive-video analysis. In this talk, we will discuss how to tackle this problem in two ways. In the first one (sensitive-video classification), we explore methods to decide whether or not a video contains sensitive material. In the second one (sensitive-content localization), we explore manners to find the moments a video starts and ceases to display sensitive content. For both cases, we will explain in details how we have designed and developed effective and efficient methods, with low-memory footprint, small runtime, and suitability for deployment on mobile devices. We start with a novel Bag-of-Visual-Words-based pipeline for efficient motion-aware sensitive-video classification. Then, we move to a novel high-level multimodal fusion pipeline for sensitive-content localization. Finally, we introduce a novel space-temporal video interest point detector and video content descriptor, which we call Temporal Robust Features (TRoF).

Dr. Daniel Moreira is a Visiting Researcher to the Computer Vision Research Laboratory at ND. For his work "Sensitive-video analysis" Daniel has been awarded the best 2017 computer vision dissertation in Brazil (announced at the XXXVII Congress of the Brazilian Computer Society).