Robot Vision


Dynamic Body VSLAM with Semantic Constraints

Image based reconstruction of urban environments is a challenging problem that deals with optimization of large number of variables, and has several sources of errors like the presence of dynamic objects. Since most large scale approaches make the assumption of observing static scenes, dynamic objects are relegated to the noise modelling section of such systems. This is an approach of convenience since the RANSAC based framework used to compute most multiview geometric quantities for static scenes naturally confine dynamic objects to the class of outlier measurements. However, reconstructing dynamic objects along with the static environment helps us get a complete picture of an urban environment. Such understanding can then be used for important robotic tasks like path planning for autonomous navigation, obstacle tracking and avoidance, and other areas

Videos:

Related Publications:

  1. Semantic Motion Segmentation Using Dense CRF Formulation Dinesh Reddy, Prateek Singhal, K. Madhava Krishna IROS 2015 Download pdf
  2. Dynamic Body VSLAM with Semantic Constraints Dinesh reddy narapureddy, Prateek Singhal, Visesh Chari and K. Madhava Krishna ICVGIP 2014 Download pdf


Monocular Reconstruction of Vehicles

Reasoning about objects in images and videos using 3D representations is re-emerging as a popular paradigm in computer vision. Specifically, in the context of scene understanding for roads, 3D vehicle detection and tracking from monocular videos still needs a lot of attention to enable practical applications.

Project page:


Visual Servoing for Object Category

Traditional visual servoing is able to navigate a robotic system between two views of the same object. However, it is not designed to servo between views of different objects. In this work, we consider a novel problem of servoing any instance (exemplar) of an object category to a desired pose (view) and propose a strategy to accomplish the task. Our approach is based on a discriminative learning framework where the desired pose is estimated using previously seen examples. Specifically, we learn a binary classifier that separates the desired pose from all other poses for that object category. The classification error is then used to control the end-effector so that the desired pose is attained. We present controllers for linear, kernel and exemplar Support Vector Machine (SVM) and empirically discuss their performance in the visual servoing context. To address large intra-category variation in appearance, we propose a modified version of Histogram of Oriented Gradients (HOG) features for visual servoing.

Project page:


Rolling Shutter and Motion Blur Removal for Depth Cameras

Structured light range sensors (SLRS) like the Microsoft Kinect have electronic rolling shutters (ERS). The output of such a sensor while in motion is subject to significant motion blur (MB) and rolling shutter (RS) distortion. Most robotic literature still does not explicitly model this distortion, resulting in inaccurate camera motion estimation. In RGBD cameras, we show via experimentation that the distortion undergone by depth images is different from that of color images and provide a mathematical model for it. We propose an algorithm that rectifies for these RS and MB distortions. To assess the performance of the algorithm we conduct an extensive set of experiments for each step of the pipeline. We assess the performance of our algorithm by comparing the performance of the rectified images on scene-flow and camera pose estimation, and show that with our proposed rectification, the performance improvement is significant.

Related Publications:

  1. Rolling Shutter and Motion Blur Removal for Depth Cameras Siddharth Tourani, Sudhanshu Mittal, Akhil Nagariya, Visesh Chari and K. Madhava Krishna ICRA 2016 Download pdf


Markov Random Field based Small Obstacle Discovery

Small obstacles of the order of 0.5−3cms and homogeneous scenes often pose a problem for indoor mobile robots. These obstacles cannot be clearly distinguished even with the state of the art depth sensors or laser range finders using existing vision based algorithms. With the advent of sophisticated image processing algorithms like SLIC [1] and LSD [9], it is possible to extract rich information from an image which led us to develop a novel architecture to detect very small obstacles on the floor using a monocular camera. This information is further processed using a Markov Random Field based graph cut formalism that precisely segments the floor and detects obstacles which are extremely low. We show robust and accurate obstacle detection and floor segmentation in diverse environments over a large variety of objects found indoors. In our case, low lying obstacles, changing floor patterns and extremely homogeneous environments are properly classified which leads to a drastic decrease in the number of obstacles that may not be classified by existing robotic vision algorithms.

Videos:

Related Publications:

  1. Markov Random Field based Small Obstacle Discovery over Images Suryansh Kumar, M. Siva Karthik and K. Madhava Krishna ICRA 2014 Download pdf
  2. Decision Theoretic Search for Small Objects through Integrating Far and Near Cues M Siva Karthik, Sudhanshu Mittal, Gurshaant Malik and K. Madhava Krishna ECMR 2015 Download pdf


Reactionless Visual Servoing of a Dual-Arm Space Robot

This paper presents a novel visual servoing controller for a satellite mounted dual-arm space robot. The controller is designed to complete the task of servoing the robot’s endeffectors to the desired pose, while regulating orientation of the base-satellite. Task redundancy approach is utilized to coordinate the servoing process and attitude of the base satellite. The visual task is defined as a primary task, while regulating attitude of the base satellite to zero is defined as a secondary task. The secondary task is formulated as an optimization problem in such a way that it does not affect the primary task, and simultaneously minimizes its cost function. A set of numerical experiments are carried out on a dual-arm space robot showing efficacy of the proposed control methodology.

Videos:

Related Publications:

  1. Image Space based Path Planning for Reactionless Manipulation of Redundant Space Robot Rachit Bhargava, P. Mithun, V. V. Anurag, A.H. Abdul Hafez and S. V. Shah. IROS 2016 Download pdf
  2. Reactionless Visual Servoing of a Dual-Arm Space Robot A. H. Abdul Hafez, V. V. Anurag, S. V. Shah, K. Madhava Krishna and C. V. Jawahar. ICRA 2014 Download pdf


Moving object detection from monocular Camera

Detection of moving objects and persons is a key component in mobile robotic perception and understanding of the environment. Robust motion detection algorithms further enable efficient estimation of the state of the environment comprising of stationary and moving objects. Such algorithms find applications in surveillance, intruder detection, person following, human-robot interaction, human augmented mapping and collision avoidance. The objective of this research, is to devise a robust and efficient method for detecting moving objects and person in highly dynamic environments.

Videos:

Related Publications:

  1. Multibody VSLAM with Relative Scale Solution for Curvilinear Motion Reconstruction Rahul Namdev, K. Madhava Krishna and C. V. Jawahar. ICRA 2013 Download pdf
  2. Motion Segmentation of Multiple Objects from a Freely Moving Monocular Camera Rahul Namdev, K Madhava Krishna and C V Jawahar. ICRA 2012 Download pdf
  3. Realtime Multibody Visual SLAM with a Smoothly Moving Monocular Camera Abhijit Kundu, K Madhava Krishna and C V Jawahar. ICCV 2011 Download pdf
  4. Moving Object Detection by Multi-View Geometric Techniques from a Single Camera Mounted Robot Abhijit Kundu, K Madhava Krishna and Jayanthi Sivaswamy. IROS 2009 Download pdf
  5. Tailing Multiple People with a Mobile Robot Using Energy Based Optical Flow Ankur Handa, J Sivaswamy, K Madhava Krishna and Sartaj Singh. Internal technical report Download pdf
  6. Person following by a mobile robot Ankur Handa, J Sivaswamy, K Madhava Krishna and Sartaj Singh. CLAWAR 2008 Download pdf


Vision Based Exploration

This work focuses on autonomously building a map of the indoor environment through exploration, which will also allow the robot to have a higher level understanding (like rooms, corridors, labs etc, which we call as semantics) of the environment. The existing approaches for exploration are based on information theoretic principles and they use range sensors as their only sensing modality. Cameras have become widely used sensor these days because of the rich and dense data they provide. It is possible to glean more information from a camera image than a laser scan. Moreover, they are cheaper and provide semantic content. The question is can we make use of these advantages to explore efficiently? The goal of this work is to try out different methods for exploration based on the semantic clues available in the environment.

Videos:

Related Publications:

  1. A Visual Exploration Algorithm using Semantic Cues that Constructs Image based Hybrid Maps Aravindhan K Krishnan and K Madhava Krishna.IROS 2010 Download pdf
  2. Image Based Exploration for Indoor Environments using Local Features Aravindhan K Krishnan, K Madhava Krishna and Supreeth Achar.AAMAS 2010 Download pdf


Convex Optimization based Mapping for VSLAM

This research aims at a novel real-time algorithm to sequentially solve the triangulation problem. The problem addressed is estimation of 3D point coordinates given its images and the matrices of the respective cameras used in the imaging process. The algorithm has direct application to real time systems like visual SLAM. The application of the proposed algorithm to the mapping problem in visual SLAM is demonstrated. The application of the proposed method to mapping in visual SLAM outperforms the state of the art mapping methods.

Related Publications:

  1. On-line Convex Optimization based Solution for Mapping in VSLAM A H Abdul Hafez, S Bhuvanagiri, K Madhava Krishna and C V Jawahar.IROS 2008 Download pdf