Morning 8:30a-12:30p:

1. Visual Search and Its Applications on Web
2. Tensor Voting: A Perceptual Organization Approach to Computer Vision and Machine Learning
3. Novel Biometrics
4. Generalized PCA
5. Numerical Geometry of Non-Rigid Shapes

Afternoon 1:30p-5:30p:

1. Recognizing and Learning Object Categories: Year 2007
2. Distributed Vision Processing in Smart Camera Networks
3. Feature Extraction and Classification
4. Fundamentals Linking Discrete and Continuous Approaches to Computer Vision - A Topological View

Search has been proven to be a powerful tool and been the most used application in the web. Visual search, on the other hand, is a recently developing application of computer vision and machine learning. Several applications have already shown new ways of traversing and searching the web via visual search. This tutorial describes multiple concepts of visual search, including visual to visual and visual to text search. Visual to visual search uses images to initiate a search and retrieves images similar to the query. Traditionally, the image search problem has been addressed in the context-based image retrieval literature by extracting and matching visual feature sets for pairs of images. However, this approach is insufficient in some cases such as face, text and object recognition where more sophisticated object-based modeling is necessary. In that case, more complex processing including steps such as object detection, alignment/registration and segmentation need to be carried out prior to feature extraction. Visual to text search can be thought as of a reverse mapping of visual features in order to enrich the text indexing on the web. The combination of visual and text search has also been introduced in several applications and will be covered during this lecture. The outline of the lecture will be as follows:

A. Introduction

• What is Visual Search
• Paradigms Of Visual Search (Object Based and Image-Retrieval Based)
• Examples Of Visual Search on the Web

B Steps Of Visual Search

• Detection
• Preprocessing
• Segmentation
• FE
• Recognition/Similarity

C. Important Issues with Visual Search

• Registration
• Learning in High Dimensional Space
• Importance of Context
• Combining features and detectors
• Local features
• Scalability

D. Combining Text and Visual Search

• Text to Visual Search
• Visual to Text Search
• Using Words and Pictures together

Description: Tensor voting is a perceptual organization approach based on the Gestalt principles of proximity and good continuation.It is based on data representation by second-order, symmetric non-negative definite tensors and on information propagation in the form of votes among pairs of neighboring points. The votes are also in the form of second-order tensors and convey the support of the voter for the presence of a structure that passes through the voter and receiver. The analysis of accumulated votes at each point provides estimates of its saliency and orientation as part of a structure such as a surface, curve or junction. Under this approach, many problems can be formulated as the perceptual organization of primitives and the solutions can be found by detecting the most salient structures formed by the primitives. For instance, in the case of stereo vision, the primitives are potential pixel correspondences reconstructed in 3D. The true scene surfaces can be inferred based on the assumption that correct correspondences support each other and form salient surfaces, while erroneous correspondences are scattered and do not form salient structures. Our approach is data-driven, local, does

not include any global computations and requires a minimal number of assumptions. The course will have little overlap with a similar course presented during CVPR 2003. There have been major improvements at the theoretical level, as well as new exciting applications in core computer vision and machine learning problems. It will not be presented as a historical recount of the work and the sequence of the presentation is not in chronological order. The emphasis will be on presenting a unified approach to a wide range of problems in computer vision and machine learning that may seem heterogeneous initially. After an overview of the theory and the introduction of recent enhancements to the framework, the most important among them being an N-D implementation, we will discuss our algorithms for addressing problems in computer vision including: figure completion, binocular stereo, multiple-view stereo, motion analysis, epipolar geometry estimation, texture synthesis, and new results we obtained in the field of instance-based learning, including dimensionality estimation, geodesic distance estimation, and function approximation.

Description:
A) Physiology-Based Face Recognition

• A1) Introduction
• A2) Facial physiology
• A3) Sensing modalities
• A4) Structural and Functional Feature Extraction
• A5) Pattern Matching and Recognition

B) Psycho-Physiology: Biometrics of Hostile Intent

• B1) Introduction
• B2) Intent assessment in context
• B3) Sensing modalities
• B4) Methodologies
• B5) Experimental Analysis

Description: Over the past two decades, we have seen tremendous advances on the simultaneous segmentation and estimation of a collection of models from sample data points, without knowing which points correspond to which model. Most existing segmentation methods treat this problem as "chicken-and-egg", and iterate between model estimation and data segmentation.

This course will show that for a wide variety of data segmentation problems (e.g. mixtures of subspaces), the "chicken-and-egg" dilemma can be tackled using an algebraic geometric technique called Generalized Principal Component Analysis (GPCA). This technique is a natural extension of classical PCA from one to multiple subspaces.

The course will also include several applications of GPCA to computer vision problems such as image/video segmentation, 3-D motion segmentation, and dynamic texture segmentation.

List of topics

I Introduction to Generalized Principal Component Analysis

II Basic GPCA Theory and Algorithms

1. Review of Principal Component Analysis (PCA)
2. Introductory Cases: Line, Plane and Hyperplane Segmentation
3. Segmentation with Known Number of Subspaces
4. Segmentation with Unknown Number of Subspaces

III Advanced Statistical and Algebraic Methods for GPCA

1. (a) Model Selection for Subspace Arrangements
2. (b) Robust Sampling Techniques for Subspace Segmentation
3. (c) Voting Techniques for Subspace Segmentation

IV Applications to Motion and Video Segmentation

1. (a) 2-D and 3-D Motion Segmentation
2. (b) Temporal Video Segmentation
3. (c) Segmentation of Dynamic Textures

V Applications to Image Representation and Segmentation

1. (a) Multi-Scale Hybrid Linear Models for Sparse Image Representation
2. (b) Hybrid Linear Models for Image Segmentation

Description: The short course deals with modern methods of analysis of non-rigid objects, an important emerging field bringing together different disciplines of mathematics and computer science such as differential and metric geometry, numerical analysis, optimization, computer graphics, machine learning, computer vision and computational geometry. The short course objective is to give theoretical and numerical tools for the analysis and comparison of surfaces from the perspective of the recent advances in the field. The first part of the short course will include a brief introduction into topology, metric and Riemannian geometry as well as numerical geometry, numerical analysis and state-of-the-art tools in numerical optimization. The second part is dedicated to the representation of intrinsic geometry of surfaces. We will discuss the notion of isometric embedding, discern between local and global isometries and

study different numerical methods for analysis and comparison of non-rigid surfaces. A major emphasis will be made on multidimensional scaling. We will introduce an axiomatic construction of isometry-invariant distances and discuss their different aspects. The last part of the short course is dedicated to applications. We will see how many important problems can be addressed within the framework of non-rigid surface matching. We will demonstrate an expression-invariant three-dimensional face recognition system based on intrinsic geometric representation of faces. The proposed short course is an abbreviated version of the Advanced Topics graduate course "Analysis of non-rigid surfaces" (236611) taught by us in the spring semester, 2006, at the Department of Computer Science in the Technion - Israel Institute of Technology.

Description:

1. Introduction:
• define the problem of object categorization (OC)
• brief history
• invariance issues in OC
• representation
• learning
• recognition
  
2. Bag of words models:
• model representation
• learning
• recognition
• demo
• all related works
  
3. Part-based models:
• model representation
• learning
• recognition
• demo
• all related works

4. Discriminative models:
• model representation
• learning
• recognition
• demo
• all related works
  
5. Objects and its contexts:
• segmentation based recognition
• context facilitated recognition
• recognition and geometry

Description: Distributed smart cameras combine techniques from computer vision, distributed processing, and embedded computing. Technological advances in the design of sensors and processors have facilitated the development of efficient embedded vision-based techniques. Distributed algorithms can provide more confident deductions about the events of interest or reduce ambiguities in a view caused by occlusion or other factors. Because they operate in real time, a variety of smart environment applications can be enabled based on the development of efficient architectures and algorithms for distributed vision networks. Building upon the premise of distributed vision-based sensing and processing, ambient intelligence can be conceived as electronic

environments that are aware of and responsive to the presence of people. Most application development efforts based on vision have focused towards monitoring scenes and persons. Distributed image sensing networks not only enhance the performance and reliability of such applications, they also enable novel ambient intelligence application areas in which the network provides useful information services to the users by monitoring the events and context they are involved with. This provides a lively field for visionbased research, pushing technology and relevant applications in smart homes, offices, factories, as well as entertainment and gaming application domains.

Description: Fundamentals and research directions of feature extraction algorithms. Feature extraction techniques are essential in many applications in science and engineering, including computer vision. The course will start with a review of unsupervised techniques, and will follow with a description of the most significant supervised methods defined to date. The effects of data noise and the use of linear methods with kernels and non-linear algorithms will be sketched.

List of topics:

• Review of unsupervised feature extraction method (PCA, MDS, ICA, FA, NMF, etc.)
• Supervised feature extraction: discriminant analysis (DA).
• The use of metrics in DA.
  Alternative criteria.
• Effects of data noise in feature extraction.
• Selecting kernels in feature extraction.
• Nonlinear methods.

Description: Continuous and combinatorial methods of image analysis have developed in computer vision largely along separate lines of research. The primary difference between a combinatorial and continuous algorithm for computer vision is whether or not an image is treated as a large set of regular samples approximating a continuum domain or as a set of discrete objects. Although the former approach leads to mathematical exposition and analysis in terms of partial differential equations and the latter to graph theoretic methods, both toolsets for analysis may be derived from the common language of topology. In personal experience with members of the computer vision field, there seems to be a misconception that combinatorial and continuum mechanics are wholly separate from each other and that there can therefore be no meaningful cross-pollination of ideas between continuum and combinatorial algorithms. The primary goal of this short course will be to clarify this confusion and rebuild a common framework from primary principles that accommodates both approaches. The

structure of this short course will be to start from the fundamentals of topology (specifically algebraic topology) and derive both the combinatorial and continuous frameworks in a common setting. The goal is to give attendees working with disparate mathematical tools the ability to translate continuum methods to a combinatorial formulation and vice versa in order to facilitate idea exchange. The latter part of the course will take practical examples from the computer vision literature and show how they fit into this common setting. A running theme of the short course is that neither continuum nor combinatorial methods has any inherit primacy in computer vision and both mathematical (and physical) traditions are rich enough to support most concepts. Therefore, more justification should be given to why a new idea is better expressed in a continuum or combinatorial language. Most of the examples given throughout the short course will be in the area of image segmentation, since this is the presenter's particular area of focus.