Skip to content

Ideas page for MRPT Google Summer of Code 2016

Jump to bottom Edit New page
Jose Luis Blanco-Claraco edited this page May 29, 2016 · 53 revisions

Table of contents

Important information for Students

  • GSoC2016 website. It is fundamental to read all rules and documents from Google before writing a proposal.
  • GSoC mailing list: You can ask general questions here.
  • GSoC 2016 timeline
    • 29th Feb 2016: MRPT got accepted as a GSOC 2016 organization!
    • 14th to 25th Mar 2016: Students can apply in this period only!

Being accepted as a GSoC student is a quite competitive process. Please, if you wish to submit a proposal, consider becoming familiar with the involved technologies first. Simply copying and pasting from this page will not be accepted.

How you will be evaluated if you are an accepted student

Student projects will be paid only if:

  • Midterm (20-27 June 2016): A pull request is requested that...
    • Builds, ideally in travis-ci
    • Has, at least, stubbed out new functionality
    • Code has appropriate Doxygen documentation
    • Has a stubbed out example/tutorial/ROS launch file that builds/runs without errors
    • It observes the recommendations in "How to contribute", which include C++ style suggestions.
  • End of summer (23-29 August 2016):
    • A complete pull request that builds, full Doxygen documentation, unit test if applicable, complete functionality.
    • A video (e.g. on YouTube) demonstrating your code.

Students interested in applying

  • You must already be proficient in C++.
  • Take your time to learn about MRPT. Try watching some YouTube videos, reading tutorials, downloading it and launching demo applications, for example. Getting familiar with ROS and MRPT ROS packages would be a plus.
  • Take a look at the projects in the "GSoC ideas page". Discuss those of your interest, or your own ideas, in the MRPT forum/mailing list.
  • Read carefully about Google's student eligibility rules.
  • Sign up in GSoC and post your project proposal to the MRPT Organization. Your proposal should include:
    • A coding portfolio of past/current projects.
    • Your GitHub/GitLab/... username.
    • A detailed description of your project proposal, including initial plans for research directions and a list of key and optional features to be coded as time permits.
    • Will you be working full-time on the project during the summer? Will you have other commitments too (a second job, classes, etc)?
    • Why do you feel motivated to undertake this particular project? :-)

Rules say that all communications between students and mentors should happen in public: please, use the MRPT mailing list or GitHub comments in pull-requests or commits whenever possible, before and during GSoC.

Ideas list for GSoC 2016

MRPT provides developers with portable and well-tested applications and C++ libraries covering data structures and algorithms employed in common robotics research areas. ROS (Robot Operating System), supported by the OSRF, provides libraries and tools to help software developers create robot applications. It provides hardware abstraction, device drivers, libraries, visualizers, message-passing, package management, and more.

At present (Feb-2016), only a minimal part of MRPT functionality is available to ROS users, hence that many of the project ideas are aimed at developing MRPT-powered ROS packages.

List of potential mentors (pairing of projects to mentors will be done when Google decides the number of slots assigned to MRPT):

Project #1: Robust SLAM/localization with artificial visual landmarks and stereo vision

  • Brief description: Design a set of "QR code-like" bidimensional markers suitable for recognition from several meters away, write (or integrate existing? AprilTags?) C++ code to detect and unequivocally identify them by some unique ID numbers in stereo images. A set of non-aligned points (a bidi array of corners?) must be detected with (sub)pixel accuracy to enable the optimization algorithm to converge to a non-degenerate solution. With all these, you can write code for automatically reconstruct the 3D pose (SE3) of all markers and the exact trajectory of the camera. This is called Bundle Adjustment or SLAM, and consists in solving an optimization problem, most of its pieces already implemented in MRPT. Also, once a map of markers is generated, one can run the optimization for the camera poses only, allowing accurate and robust localization, with applications to autonomous vehicle navigation, augmented reality, etc.

  • Expected results:

  • A new C++ class in mrpt-detectors for QR-like markers detection. When finished, its integration into OpenCV will be considered.

  • Generic new C++ classes in mrpt-slam for SLAM and localization with visual markers.

  • A working application capable of running the new methods on live or offline stereo visual datasets.

  • The method must work for professional stereo cameras (e.g. Bumblebee2). The organization will provide the student with stereo images of the markers designed by him/her.

  • The method may optionally also work with a pair of consumer-grade webcams, in the case the student wants to print his/her own markers for testing at home.

  • Knowledge prerequisites: C++, computer vision, OpenCV, numerical optimization.

  • Difficulty level: Middle

Project #2: Integrate MRPT reactive navigators into ROS move_base

mrpt_reactivenav2d

  • Brief description: Use ROS pluginlib to integrate MRPT TP-Space reactive navigation for autonomous vehicles/robots as a local_planner in move_base. Further reading:
  • Expected results: Being able to launch a navigation command in ROS' RViz so it is firstly processed by existing ROS global planners and the path followed locally by MRPT methods. Writing a good tutorial on ROS Wiki about the process would be also a great outcome.
  • Knowledge prerequisites: C++, GNU/Linux, vehicle kinematics.
  • Difficulty level: Middle.

Project #3: ROS package for 2D/3D range-bearing EKF SLAM/Localization

3d-ekf-slam-snapshot

Useful for features-based mapping and localization: LIDAR-based corners, visual features from stereo images, etc.

  • Brief description: MRPT features C++ classes, a command-line application and a GUI application implementing localization and SLAM for mobile robots (or hand-held cameras/sensors) and features-based maps. Nothing of which is exposed in ROS, and to the best of our knowledge, there exist no other ROS nodes exposing such sensor-agnostic, generic algorithms.
  • Expected results: ROS messages and nodes for 2D/3D SLAM and localization. Perhaps different ROS nodes will have to be written for each type of problem. Tutorial and/or demo based on simulator data. Optionally, use real data from LIDAR corner detectors, feature points from stereo camera, etc. to show the algorithm working in real-world conditions.
  • Knowledge prerequisites: C++, GNU/Linux.
  • Difficulty level: Middle.

Project #4: ROS package(s) for versatile RBPF-based SLAM

ro-slam

  • Brief description: MRPT features RBPF-SLAM with different kinds of map models and sensors. This project consists in creating ROS nodes for some practical cases:
    • LIDARs + occupancy grid maps.
    • Range-only sensors + beacon maps.
  • Expected results: Working ROS node(s) for, at least:
    • SLAM with LIDAR and 2D grid maps / point clouds (user-selectable)
    • SLAM with range-only sensors (new ROS messages may have to be defined)
    • Localization with range-only / RFID sensors and odometry. All those nodes must be proven to work with simulated data and, where possible, real datasets.
  • Knowledge prerequisites: C++, GNU/Linux.
  • Difficulty level: Middle / Middle-high.

Project #5: Extend existing ROS packages for grabbing and playing back MRPT datasets

mrpt-datasets

  • Brief description: Many datasets are available in MRPT Rawlog format or can be imported to it (e.g. Carmen logs). ROS log format (rosbag) is incompatible, so run-time or off-line conversion tools are required if Rawlog datasets want to be used in a ROS application.
    • mrpt_rawlog ROS wiki page of existing packages. Very limited set of sensors are implemented at present.
  • Expected results: Being able to view all/most sensory data from a complex Rawlog dataset, e.g. Málaga Urban Dataset, in ROS RViz. It may be done via run-time on-the-fly conversion (easier) or by a new command-line tool capable of converting among rawlog and rosbag formats (harder).
  • Knowledge prerequisites: C++, GNU/Linux.
  • Difficulty level: Low / Middle-Low.

Project #6: New GUI app to visualize and edit robot-made maps

  • Brief description:

    • During 2005-2008 there existed a complex GUI application with this purpose called SimpleMapsViewer, but it was Windows-only (written in Borland C++ Builder!) so it was dropped when MRPT became crossplatform.
    • It is strongly recommended to have a look at a working snapshot of the old app (from 2008!) to get an idea of the expected basic functionality: SimpleMapsViewer-MRPT-0.5.3-win32.zip Use: Extract the ZIP, execute SimpleMapsViewer.exe and load the example map file localization_demo.simplemap.
    • At present, the only similar app in MRPT is observations2map, which allows generating metric map but without a GUI and without edit possibilities.

  • Expected results: A new GUI app should be able to load recent versions of MRPT simple maps as generated from ICP-SLAM or RBPF-SLAM. The app must allow inspecting the raw observations (at least, 2D LIDAR scans and monocular and stereo images) and generate one or more metric maps according to user-given parameters, e.g. 2D point clouds, 2D occupancy grid maps, a 3D octomap, etc.

  • Knowledge prerequisites: C++, GUI design (Qt or wxWidgets).

  • Difficulty level: Middle

Project #7: Port mrpt::gui GUI classes from wxWidgets to Qt

mrpt_screenshot_example_opengl_objects_demo

  • Brief description: MRPT offers C++ classes under mrpt::gui for easily prototyping command-line apps capable of 2D and 3D rendering, without having to worry about message loops, multithreading, etc. All this runs behind scenes. At present, MRPT classes built upon wxWidgets GUI components. However, there exist other nice Qt alternatives, especially for 2D plots, so porting the code to Qt would be a great improvement.
  • Expected results: mrpt::gui should build with wxWidgets or Qt, and the user API should be unaffected by the underlying GUI library. All existing examples and apps should compile with the new Qt GUI and run without problems.
  • Knowledge prerequisites: C++, Qt.
  • Difficulty level: Easy

Project #8: Algorithm improvements to KD-Tree module: applications to ICP

mrpt_icp_demo

  • Brief description: nanoflann is a children project from MRPT, born as a smaller and faster version of the widely-used (OpenCV, PCL) FLANN C++ library for kd-tree neighbor search. nanoflann is especially efficient for finding nearest neighbors in 2D/3D point clouds, an operation at the heart of most ICP algorithms for robot SLAM and localization. A key feature missing from nanoflann is the ability to support dynamic point clouds: at present, adding just 1 point to a pointcloud of 1e6 points would force rebuilding the kd-tree index, a costly operation. The goal of this project would be, among other minor improvements, the introduction of two dynamic strategies: (a) rebuild threshold, and (b) the logarithm method. In theory, (b) is better, but both should be implemented and benchmarked, among them, and other kd-tree C++ libraries, so we could be an informed decision about which is the most efficient implementation we should use in MRPT and other robotics software. In the way, nanoflann could be renamed something else, like libkdt or anything even more creative!
  • Expected results: An improved version of nanoflann, capable of handling dynamic datasets efficiently. MRPT ICP, perhaps after modifications, should build against the new kd-tree code. The results of the above mentioned benchmark, along with run-time improvements of ICP runs over public datasets of 3D point clouds.
  • Knowledge prerequisites: C++, algorithms complexity analysis.
  • Difficulty level: Middle.

Project #9: Implement a generic graph-slam engine, particularized for 2D/3D ICP-based graph-SLAM

mrpt-graph-slam

  • Brief description: The goal is writing a C++ class capable of processing a sequence of observations (odometry, LIDAR scans, images, etc.), decide how often to insert new keyframes (robot poses) in a graph-SLAM map, launch loop-closure detectors (based on ICP, gridmap matching, or visual clues) and optimize the global map. It should be capable of handling small and mid-sized SLAM problems, so it would become an invaluable tool for practical roboticists.

Such a complete SLAM system would comprise two parts, which should ideally be user-selectable independently:

  1. SLAM frontend: For each kind of sensory data, process it and extract a discrete set of "observations". The frontend must also face the difficult problem of detecting loop closures (refer to SLAM literature for the meaning of this term). Naturally, loop closure detection algorithm is strongly dependent on the nature of sensors, so different methods may be selected by the user. It is foreseeable that these methods will have a few magic parameters and thresholds, the number of such should be reduced as much as possible. (The GSoC must implement at least one LC method, e.g. based on grid-map matching, ICP-based for 2D scans, or based on BoW or alike for similar image query). Note that it is not mandatory to use the same kind of sensory information for defining relative SE(2)/SE(3) poses among neighbors in the graph (e.g. 2D ICP) than for loop closure (e.g. GPS, visual images), although the framework should be robust enough to cope with 1, 2 or 3 types of simultaneous sensors installed on the robot. For multiple sensors, think about the implications of observations not being sampled at exactly the same instants. If necessary, it will be fine to assume that the robot/vehicle moves at a relatively low speed. Checkout the C++ class mrpt::obs::CSensoryFrame just in case it is useful here. Examples of "observations" extracted from "raw sensory data":
  • A front-end for 2D laser scanners may perform ICP to other scans -> observation = relative 2D/3D poses to nearby nodes (The GSoC code should implement at least this one).
  • A front-end for 2D scanners may extract LIDAR features -> observation = relative (range-bearing) landmarks with a descriptor. LIDAR features:
  • Stereo images, BRIEF detector -> observation = relative position of 3D features with descriptors.
  • etc...
  1. SLAM backend: Accept definitions of new robot key-frames/nodes, optionally also landmarks (e.g. the LIDAR ICP frontend does not require landmarks) and constraint edges among them. The API should be general so we can plug in here one or another SLAM backend implementation easily. Supported backends:
  • MRPT mrpt::graphslam Levenberg-Marquardt graph-SLAM optimizer (at minimum this one must be supported so the entire framework does not require more 3rd party dependencies).
  • Optionally, other 3rd party solvers: g2o, iSAM2, etc.

Resources: An attempt to create such an engine was already made in 2014 but it was abandoned. The branch was deleted but is still reachable here. Take a look at the files libs/graphslam/include/mrpt/graphslam/GraphSlamEngine.h, libs/graphslam/include/mrpt/graphslam/deciders.h, libs/graphslam/include/mrpt/graphslam/f2f_match_icp_2d.h,... to get an idea of one possible implementation.

Start studying the "C++ code" section in http://www.mrpt.org/Graph-SLAM_maps Ideally, the framework should use the current graph SLAM implementation in mrpt::graphslam. Using g2o may be acceptable as well, but it would introduce new dependencies into MRPT and might complicate building it under Windows, which is a problem for MRPT where we try to support both GNU/Linux and Windows.

Project #10: PTG visual editor

(New idea for next GSoC 2017?)

New project template:

  • Brief description:
  • Expected results:
  • Knowledge prerequisites:
  • Difficulty level:
Add a custom footer
Add a custom sidebar
Clone this wiki locally