Environment mapping and landmarks extraction by passive 3D vision, for robot navigation

Contact: bogdan.stanciulescu at

Position and duration: Postdoctorate – 12 Months full time contract

Starting date: 1st of April 2016

Qualifications and skills: Applicants must have a PhD in the field of computer science, electrical engineering, physics, or any other related field. The candidates need to have a strong background in scene interpretation, particularly in the following fields: 3D environment reconstruction, SLAM, feature extraction, scene recognition, visual object detection. The applicants must have good communication skills, be able to work in a team environment and have fluent English skills. French language knowledge is a plus, but not compulsory.

The application must contain information of research background and work experience, including:

  1. A motivation letter outlining background, experience and interest in the subject.
  2. A detailed CV, including personal contact information and list of publications.


Applications must be submitted by e-mail to bogdan.stanciulescu at with the subject: POSTDOCTORAL POSITION.

The Robotics Laboratory of Mines-ParisTech (CAOR) has developed extensive competences and tools in the field of computer vision and pattern recognition for real-time object detection and classification (people, vehicles, faces, etc). One of the CAOR’s algorithms has been internationally recognised as the 2nd best Pascal VOC challenge 2006.

For its results in real-time object recognition and classification, the CAOR’s has been rewarded the Best Student Paper Award at the International Conference on Control, Automation, Robotics and Vision 2011, and again rewarded the International Joint Conference on Neural Networks 2011 object recognition challenge.

The postdoctoral associate could use the CAOR’s experience in real-time video processing, robust signature extraction from multiple images and machine learning.

Least but no last, the Robotics Lab has acquired a good experience in sensor data fusion for performing indoor SLAM. The Laboratory’s prototype « Corebots » has won 2 times out of 3 the DGA-ANR Carotte competition for mobile robots, by a precise 3D environment mapping and localisation.


SLAM laser mapping by Corebots prototype

This postdoctoral position could extrapolate the acquired experience in laser-SLAM mapping to the new context/camera for the personal assistance robotics.

3D passive vision for mobile robotics

In 3D passive vision, the most known systems are stereovision cameras. In 3D active vision, the most known system families are based on structured light or on the time-of-flight.

These sensors are difficult to use in large areas or in bad visibility conditions, as they are dependent of the reflected beam.

The obstacles’ detection is another field of application of the passive vision algorithms, usually in mixture with machine learning techniques.


a) « Carotte » robot using the stereo-head Blumblebee XB3 ; b) 3D environment mapping at low and high range, performing also obstacle detection (red).

The lighfield cameras are the most promising today sensors for visual SLAM applications.

They are very similar to the stereovision imaging or multi-vision imaging, that reconstruct the 3D environment.

The lightfield 3D reconstruction principle is to observe the different angles of the image of the environment instead of the environment directly. This is done by choosing a sensor which is relatively much bigger than the image of the environment. This will form different point of views on the sensor, of the same (image of the) environment.


a) Mono-vision b) Lightfield

(Source :


c) 3D reconstruction of the object/environment

From the robotics point of view, Raytrix produces an interesting lighfield camera, the Robotics Laboratory is testing now.

Postdoctorate’s objectives

The lightfiled cameras hold a lot of potential for perception applications in robotics. This postdoctoral subject can be divided in two parts, as follows:

A.      3D environment reconstruction by lightfield camera, on a mobile robotised platform

The first objective is to perform a 3D environment mapping using the video data, provided by the lightfield camera. Multiview algorithms for 3D mapping will be explored, in order to extract the spatial model of the environment. A previous step, but an important one, is to perform the camera (auto)calibration, using a known object/shape.

Thanks to several past projects and thesis (Terra Data, Terra Numerica) the Robotics Centre have acquired a good experience in 3D mapping algorithms by lidar and 2D-camera. The postdoctoral candidate could extend this existing algorithmic base to lightfield cameras, or stereovision (if necessary).

B.      Robust landmarks extraction for robot localisation and navigation

The second major part of this postdoctoral will be dedicated to the task of indoor navigation, once the 3D mapping provided.

Visual data segmentation algorithms of the « 2D+Depth » environment map will be used, in order to analyse and extract the obstacles in the 3D environment.

The most robust obstacles with the robot’s ego-motion will be considered as « landmarks »., which will provide the robot coordinates at any moment. A dictionary of these landmarks will be thus created.

Scene recognition could also be performed, through the known landmarks present in the room.

Colour and texture analysis could also be taken into consideration, together with the landmarks detection.


Originality of the topic

This postdoctoral position aims to the study of the lighfield cameras and their 3D environment mapping algorithms, for an assistance robot indoor navigation. This kind of camera performs a complete passive recording of the optical field, phase included, without any wave emission that could limit its range. In holds the advantages of the stereovision, which is its closest sensor, without the drawbacks related to the baseline distance and the size.

Nowadays, inspire of its applicative potential, this kind of sensor is not very well studied. Therefore the challenge is double; we are planning to study its performances in comparison with the existing systems, and in the same time to develop optimal algorithms, adapted to its exploitation.


[Bo07] A. Bosch, A. Zisserman, and X. Munoz. Image Classifcation using Random Forests and Ferns, Proceedings of the International Conference on Computer Vision, Rio de Janeiro, Brazil, 2007, pp. 1-8.

[Ca07] Robert Castle, Darren Gawley, Georg Klein and David Murray. Towards simultaneous recognition, localization and mapping for hand-held and wearable cameras. In Proc. International Conference on Robotics and Automation (ICRA’07, Rome)

[Ca10] A. Cappalunga, S. Cattani, A. Broggi, M. McDaniel, S. Dutta. Real Time 3D Terrain Elevation Mapping Using Ants Optimization Algorithm and Stereo Vision, 2010

[Ch08] “Block-based image de-blurring for digital cameras”, Applicant: NXP B.V., Inventors: A. Chouly, Filing Date: 20 June 2008, Application No: EP08290590.2, NXP Ref: 81048739EP01.

[Ci12] European Project “CityMobil”:

[Cy04] European Project “Cybercars”

[Da07] Andrew J. Davison, Ian D. Reid, Member, IEEE, Nicholas D. Molton, and Olivier Stasse, Member, MonoSLAM: Real-Time Single Camera SLAM. IEEE Transactions on pattern analysis and machine intelligence, Vol. 29, No. 6, June 2007

[Da11] Emmanuel Pierre Aimé D’Angelo, Pierre Vandergheyst, “Method to compensate the effect or the rolling shutter effect”, brevet 348/296; 348/E05.091, 2011

[De12] Défi Carotte, ANR-DGA 2010-2012. Reference :

[En11]   F. Endres. J. Hess, N. Engelhard, J. Sturm, D. Cremers, W. Burgard “An Evaluation of the RGB-D SLAM System”, 2011

[He08] Herbert Bay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool, “SURF: Speeded Up Robust Features”, Computer Vision and Image Understanding (CVIU), Vol. 110, No. 3, pp. 346–359, 2008

[Je06] Jeong-A Im, Dae-Woong Kim, and Ki-Sang Hong, “Digital Video Stabilization Algorithm for CMOS image sensor” ICIP 2006

[Kl07] Georg Klein and David Murray, Parallel Tracking and Mapping for Small AR Workspaces In Proc. International Symposium on Mixed and Augmented Reality (ISMAR’07, Nara) Won the best paper prize

[Kl08] Georg Klein and David Murray – Improving the Agility of Keyframe-based SLAM – In Proc. European Conference on Computer Vision (ECCV’08, Marseille)

[Ku08] K. Kuhnert. “Concept and Implementation of a Software System on the Autonomous Mobile Outdoor Robot” AMOR, 2008

[La05]R. L. Lagendijk and J. Biemond “Basic Methods for Image Restoration and Identification”, Handbook of Image and Video Processing, Elsevier Academic Press, editor Al Bovik, Sd Edition 2005.

[Lee09] T. Lee and T. Höllerer. 2009. “Multithreaded Hybrid Feature Tracking for Markerless Augmented Reality”. IEEE. Transactions on Visualization and Computer Graphics 15, 3 (May. 2009), 355-368.

[M01]  A. Mallet. Localisation d’un robot mobile autonome en milieu naturel, 2001

[Mo08] F. Moutarde, B. Stanciulescu and A. Breheret. “Real-time visual detection of vehicles and pedestrians with new efficient adaBoost features”, 2008.

[OZ07] OZUYSAL, M, FUA, P et LEPETIT, V. Fast “Keypoint Recognition in Ten Lines of Code. Minneapolis”: Conference on Computer Vision and Pattern Recognition, 2007. CCVPR2007.

[St07]: B. Stanciulescu, A. Breheret, F. Moutarde, COGnitive systems with Interactive Sensors 2007, Stanford University California, Etats-Unis, 26 Nov 2007, “Introducing New AdaBoost Features for Real-Time Vehicle Detection”. [ST]

[Tae]     Projet HandyAR – Taehee Lee, Tobias Höllerer –

[Tu08]   Turaga P., Chellappa R., Subrahmanian VS., Udrea O. : “Machine Recognition of Human Activities: A Survey”. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 18, No. 11, November 2008

[Za11] Fatin Zaklouta, Bogdan Stanciulescu, “Real-time traffic sign recognition using spatially weighted HOG trees”, IEEE International Conference on Advanced Robotics (ICAR) 2011, Best Student Paper Award

[Zak11] Fatin Zaklouta, Bogdan Stanciulescu, Omar Hamdoun, “Traffic Sign Classification using K-d trees and Random Forests”, IEEE International Joint Conference on Neural Networks (IJCNN) 201

Comments are closed.