View Item

Domain Virtual and Augmented Reality
Domain - extra Image processing and computer vision
Year 2014
Starting October 2014
Status Open
Subject Sparse vs. dense matching. Application to interaction and spatial augmented reality.
Thesis advisor GOUIFFES Michèle
Co-advisors Samia Bouchafa
Laboratory LIMSI AMI
Collaborations LIMSI-CNRS, Orsay
Abstract The thesis will focus on image processing and computer vision applied to mobile Spatial Augmented Reality (implemented on a mobile device), in order to address the three following problems:
• localization of the device by registration with the 3D model of the scene. The surface viewed by the camera of the mobile device is already augmented with textural contents by the global system, and each 2D feature has previously been registered to the 3D model. Therefore, 3D the localization of the device consists in 2D-2D features matching (local 2D to global 2D).
• real-time photometric and geometric adaptation of the projection for enhanced visualization, which requires to account for the Human Visual System.
• egomotion and structure from motion: flow analysis and prediction for motion analysis, sensor fusion: image and accelerometers (when available).

Context The PhD work will be a part of a more global project on Spatial Augmented Reality (SAR, or projector-based AR) and on human-computer interaction using several modules: a global context-aware SAR system, gesture analysis, several mobile devices (when available). SAR consists in adapting a videoprojection to the properties of the surface, relying on smart projectors, i.e. enhanced with sensors to gain information about the environment. These systems can be static or mobile (on smartphones for example). The works lead by the team AMI aim to make the SAR system more reactive by offering more input channels through image analysis techniques (scene analysis and users’ behaviors) and more interactive. Real-time 3D reconstruction and motion analysis will allow users to be recognized by the system. Finally, SAR should become more collaborative and ubiquitous by allowing multiple people to jointly interact through gesture (selecting, displaying or drawing) or through their mobile devices.
Objectives The thesis will focus on feature matching which is a core aspect of several image processing algorithms, the objective is to provide some clear conclusions concerning the comparison between sparse and dense matching methods. Main stages:
1) a theoretical and bibliographic study: analyzing the trends in the domain, determining comparison criteria in terms of noise sensitivity, illumination invariance, errors, complexity.
2) study on several problems among which motion analysis (egomotion), tracking and 3D reconstruction (from stereovision or structure from motion. Examples of methodologies: sparse feature matching using feature graph matching, dense (or semi-dense) matching with cumulative techniques Bouchafa 12 or tensor-based techniques Laguzet 13.
3) A study will be made to automatically switch the type of features and/or the matching method (i.e. sparse/dense feature matching) depending on the knowledge acquired from the context.
4) Experiments in interaction and SAR

Work program The stages of the thesis are the following:
  • Bibliography study concerning sparse and dense features matching in computer vision
  • Graph matching/tracking under geometric changes and large illumination changes
  • Cumulative methods for registration and structure detection
  • Adaptation to the context
  • Experiments
  • Report writting

Extra information Bouchafa12 Samia Bouchafa, Bertrand Zavidovique: c-Velocity: A Flow-Cumulating Uncalibrated Approach for 3D Plane Detection. International Journal of Computer Vision 97(2): 148-166 (2012)
Setkov 13 A. Setkov, M. Gouiffès and C. Jacquemin. Color Invariant Feature Matching For Image Geometric
Correction. ACM Mirage 2013
Laguzet13 Color tracking for contextual switching: Real-time implementation on CPU F.Laguzet, A.Romero, M.Gouiffès, L.Lacassagne, D.Etiemble JRTIP2013
Prerequisite Some knowledge about image processing and 3D geometry are required. The student has to be able to program in C/C++.
Details Download ThesisProposal_LIMSI_Gouiffes - 2014.pdf
Expected funding Institutional funding
Status of funding Expected
user michele.gouiffes
Created Tuesday 28 of May, 2013 11:59:34 CEST
LastModif Monday 16 of June, 2014 12:25:25 CEST
Attachments (1)


DownloadThesisProposal_LIMSI_Gouiffes - 2014.pdf28 May 2013 11:591019338.64 Kb

The original document is available at