Skip to content

Ideas page for MRPT Google Summer of Code 2017

Jump to bottom Edit New page
Jose Luis Blanco-Claraco edited this page Mar 26, 2017 · 31 revisions

Table of contents

Important information for Students

  • GSoC2017 website. It is fundamental to read all rules and documents from Google before writing a proposal.
  • GSoC mailing list: You can ask general questions here.
  • GSoC 2017 timeline
  • March 20 16:00 UTC-April 3 16:00 UTC: Students can apply in this period only!

Being accepted as a GSoC student is a quite competitive process. Please, if you wish to submit a proposal, consider becoming familiar with the involved technologies first. Simply copying and pasting from this page will not be accepted.

How you will be evaluated if you are an accepted student

Student projects will be paid only if:

  • 1st phase (June 26-30, 2017): A pull request is requested that...
    • Builds, ideally in travis-ci
    • Has, at least, stubbed out new functionality
  • 2nd phase (July 24-28, 2017): A pull request is requested that...
    • Same as above, plus:
    • Code has appropriate Doxygen documentation
    • Has a stubbed out example/tutorial/ROS launch file that builds/runs without errors
    • It observes the recommendations in "How to contribute", which include C++ style suggestions.
  • Final phase (August 21-29, 2017):
    • A complete pull request that builds, full Doxygen documentation, unit test if applicable, complete functionality.
    • A video (e.g. on YouTube) demonstrating your code.

Students interested in applying

  • You must already be proficient in C++.
  • Take your time to learn about MRPT. Try watching some YouTube videos, reading tutorials, downloading it and launching demo applications, for example. Getting familiar with ROS and MRPT ROS packages would be a plus.
  • Take a look at the projects in the "GSoC ideas page". Discuss those of your interest, or your own ideas, in the MRPT forum/mailing list.
  • Read carefully about Google's student eligibility rules.
  • Sign up in GSoC and post your project proposal to the MRPT Organization. Your proposal should include:
    • A coding portfolio of past/current projects.
    • Your GitHub/GitLab/... username.
    • A detailed description of your project proposal, including initial plans for research directions and a list of key and optional features to be coded as time permits.
    • Will you be working full-time on the project during the summer? Will you have other commitments too (a second job, classes, etc)?
    • Why do you feel motivated to undertake this particular project? :-)

Rules say that all communications between students and mentors should happen in public: please, use the MRPT mailing list or GitHub comments in pull-requests or commits whenever possible, before and during GSoC.

Update: We have created a slack channel for answering questions and providing feedback during the GSoC period. You can sign up using https://mrpt-gsoc.herokuapp.com/

Ideas list for GSoC 2017

MRPT provides developers with portable and well-tested applications and C++ libraries covering data structures and algorithms employed in common robotics research areas. ROS (Robot Operating System), supported by the OSRF, provides libraries and tools to help software developers create robot applications. It provides hardware abstraction, device drivers, libraries, visualizers, message-passing, package management, and more.

List of potential mentors (pairing of projects to mentors will be done when Google decides the number of slots assigned to MRPT):

Project #1: Web-based UI

  • Brief description: Robots are quickly becoming part of the Internet of Things. Naturally, a web-based UI is ideal for an Internet connected robot. You will be providing a web-based UI for MRPT. The backend RESTful web service must be able to interact with a native MRPT application and the frontend must be able to run in a Chrome browser. You will use a MVC design pattern, so that new web applications can be easily developed and maintained. A native C++ WebSocket library such as libwebsockets should be used to efficiently stream real-time data from the MRPT app to the browser.

  • Expected results: At least two applications. The applications may be web-based clones similar to a current wxwidget applications. At least one application must be able to render robot poses, observations and maps in 3D using real-time data.

  • Knowledge prerequisites: C++, WebSockets, REST, MVC, JavaScript(e.g. AngularJS, socket.io), WebGL

  • Difficulty level: Middle

Project #2: Algorithm improvements to KD-Tree module: applications to ICP

mrpt_icp_demo

  • Brief description: nanoflann is a children project from MRPT, born as a smaller and faster version of the widely-used (OpenCV, PCL) FLANN C++ library for kd-tree neighbor search. nanoflann is especially efficient for finding nearest neighbors in 2D/3D point clouds, an operation at the heart of most ICP algorithms for robot SLAM and localization. A key feature missing from nanoflann is the ability to support dynamic point clouds: at present, adding just 1 point to a pointcloud of 1e6 points would force rebuilding the kd-tree index, a costly operation. The goal of this project would be, among other minor improvements, the introduction of two dynamic strategies: (a) rebuild threshold, and (b) the logarithm method. In theory, (b) is better, but both should be implemented and benchmarked, among them, and other kd-tree C++ libraries, so we could be an informed decision about which is the most efficient implementation we should use in MRPT and other robotics software. In the way, nanoflann could be renamed something else, like libkdt or anything even more creative!
  • Expected results: An improved version of nanoflann, capable of handling dynamic datasets efficiently. MRPT ICP, perhaps after modifications, should build against the new kd-tree code. The results of the above mentioned benchmark, along with run-time improvements of ICP runs over public datasets of 3D point clouds.
  • Knowledge prerequisites: C++, algorithms complexity analysis.
  • Difficulty level: Middle.

Project #3: Continue the development of mvsim

MvSim intro

  • Brief description: mvsim is a children project of MRPT that allows real-time dynamic simulation of multiple vehicles. MultiVehicle simulator (mvsim) is a lightweight, realistic dynamical simulator for 2D ("2.5D") vehicles and robots. It is tailored to analysis of vehicle dynamics, wheel-ground contact forces and accurate simulation of typical robot sensors (e.g. laser scanners). It includes a C++ library mvsim, a standalone app and a ROS node.

  • Expected results:

    • Improve the existing prototype vehicle models to have more realistic kinematics and dynamics. A central point is fully understanding the forces involved in the vehicle propulsion, tire-ground interaction, etc.
    • Maintenance of documentation explaining the Mechanical/Physics models, the existing (and newly created) APIs, etc.
    • Integration with MATLAB (via MEX and Matlab classes).
    • Simple C++ and/or MATLAB examples demonstrating different interesting behavior: vehicle slippage, slipping angle while maneuvering, a vehicle pushing a heavy block, etc. (Many of these are already there, but should be sorted, documented, etc.)
    • (Optional) Create new vehicle models, e.g. a tractor.

  • Pointers to main source code locations where lies the current implementation of the vehicle/wheels dynamics equations:

  • Knowledge prerequisites: C++, Dynamics/Mechanics.

  • Difficulty level: Middle.

Project #4: New GUI app to visualize and edit robot-made maps

  • Brief description:

    • During 2005-2008 there existed a complex GUI application with this purpose called SimpleMapsViewer, but it was Windows-only (written in Borland C++ Builder!) so it was dropped when MRPT became crossplatform.
    • It is strongly recommended to have a look at a working snapshot of the old app (from 2008!) to get an idea of the expected basic functionality: SimpleMapsViewer-MRPT-0.5.3-win32.zip Use: Extract the ZIP, execute SimpleMapsViewer.exe and load the example map file localization_demo.simplemap.
    • At present, the only similar app in MRPT is observations2map, which allows generating metric map but without a GUI and without edit possibilities.

  • Expected results: A new GUI app should be able to load recent versions of MRPT simple maps as generated from ICP-SLAM or RBPF-SLAM. The app must allow inspecting the raw observations (at least, 2D LIDAR scans and monocular and stereo images) and generate one or more metric maps according to user-given parameters, e.g. 2D point clouds, 2D occupancy grid maps, a 3D octomap, etc.

  • Knowledge prerequisites: C++, GUI design (Qt or wxWidgets).

  • Difficulty level: Middle

Project #5: Robust SLAM/localization with artificial visual landmarks and stereo vision

  • Brief description: Design a set of "QR code-like" bidimensional markers suitable for recognition from several meters away, write (or integrate existing? AprilTags?) C++ code to detect and unequivocally identify them by some unique ID numbers in stereo images. A set of non-aligned points (a bidi array of corners?) must be detected with (sub)pixel accuracy to enable the optimization algorithm to converge to a non-degenerate solution. With all these, you can write code for automatically reconstruct the 3D pose (SE3) of all markers and the exact trajectory of the camera. This is called Bundle Adjustment or SLAM, and consists in solving an optimization problem, most of its pieces already implemented in MRPT. Also, once a map of markers is generated, one can run the optimization for the camera poses only, allowing accurate and robust localization, with applications to autonomous vehicle navigation, augmented reality, etc.

  • Expected results:

  • A new C++ class in mrpt-detectors for QR-like markers detection. When finished, its integration into OpenCV will be considered.

  • Generic new C++ classes in mrpt-slam for SLAM and localization with visual markers.

  • A working application capable of running the new methods on live or offline stereo visual datasets.

  • The method must work for professional stereo cameras (e.g. Bumblebee2). The organization will provide the student with stereo images of the markers designed by him/her.

  • The method may optionally also work with a pair of consumer-grade webcams, in the case the student wants to print his/her own markers for testing at home.

  • Knowledge prerequisites: C++, computer vision, OpenCV, numerical optimization.

  • Difficulty level: Middle

Project #6: Port mrpt::gui GUI classes from wxWidgets to Qt

mrpt_screenshot_example_opengl_objects_demo

  • Brief description: MRPT offers C++ classes under mrpt::gui for easily prototyping command-line apps capable of 2D and 3D rendering, without having to worry about message loops, multithreading, etc. All this runs behind scenes. At present, MRPT classes built upon wxWidgets GUI components. However, there exist other nice Qt alternatives, especially for 2D plots, so porting the code to Qt would be a great improvement.
  • Expected results: mrpt::gui should build with wxWidgets or Qt, and the user API should be unaffected by the underlying GUI library. All existing examples and apps should compile with the new Qt GUI and run without problems.
  • Knowledge prerequisites: C++, Qt.
  • Difficulty level: Easy

Mini-project #7: Proposal to port existing website to a static generator system

(A proposal must comprise two "mini" projects to be eligible)

  • Brief description: Propose a plan to port http://www.mrpt.org/ from WordPress to an appropriate modern static-page generator (e.g. Pelican). This includes writing small plugins (Python?) or templates to handle special pages with a particular, normalized form (e.g. datasets).

  • Expected results:

  • Knowledge prerequisites: Python, HTML5, CSS.

  • Difficulty level: Middle-Low.

Project #8: Implement a state-of-the-art graph-slam solver, SE-Sync

  • Brief description: The goal is writing a novel C++ class for optimizing graph-SLAM maps (a SLAM backend). We in MRPT do our best to bring to life cutting-edge advancements in the field of SLAM. In this case, we seek to implement a state-of-the-art solver, SE-Sync: This is the first SLAM solver providing fast and global convergence. Yes, you read well, global, no need to initialize at all and yet you will reach the best possible solution for your map. The implemented solver should fit into the current graph-slam engine framework, bringing immediately the aforementioned advantages to the resolution of large-scale SLAM problems. Because of this, the output of this project should become an invaluable tool for practical roboticists.

A more or less detailed roadmap for the development of the desired solver follows:

  1. Getting familiar with the theoretical approach. The solver to implement, SE-Sync, is a very recently proposed approach proposed at WAFR16, where it obtained the best paper award. In arxiv you may find the conference paper as well as an extended technical report with the approach. A video recording of the presentation at the conference is available as well. Note the theory behind the solver relies on Riemannian geometry and related techniques, BUT you need not to be scared: The hardest part occurs behind the scenes so you will not need to know to much about the topic of Riemannian optimization (just the basics). In addition, you can find a Matlab implementation to play with.

  2. Once you have a look at the materials above and get an idea of what we are heading for, you should definitely be thrilled about the potential outcome of this project. We are :) So next comes porting this exciting stuff into a clean and efficient C++ implementation the MRPT community can benefit of seamlessly. The main required steps to achieve this are:

  3. Create a novel CPoseSync class that fits the generative model exploited in SE-Sync. Difficulty: Easy

  4. Implement the new optimization method, mrpt::graphslam::optimize_graph_Sync, akin to the current mrpt::graphslam::optimize_graph_spa_levmarq, with the SE-Sync approach. This is where the magic should happen: Given a graph consisting of a mrpt::graphs::CNetworkOfPoses with poses of the newly defined CPoseSync type, the method should follow the SE-Sync algorithm to solve the globally optimal estimate of the map. Some very useful hints are:

    • The Matlab implementation of SE-Sync strongly relies on the Manopt toolbox for applying Riemannian optimization behind the scenes, with a minimal understanding of what's going on.
    • Luckily, the corresponding recent library ROPTLIB provides the same functionality in C++. The most appropriate approach we expect you to take is, then, to adapt the Matlab code of SE-Sync into C++ by means of ROPTLIB, and then the additional dependency on ROPTLIB should be safely introduced into MRPT as a result. This will require you to get a minimal user-level understanding of these two libraries (Manopt and ROPTLIB), which come with plenty of working examples. Additional links with related resources will be given below. Difficulty: Medium/high

  5. To make this solver usable in a practical framework, provide a new specialization of mrpt::graphslam::optimizers::CGraphSlamOptimizer, mrpt::graphslam::optimizers::CSyncGSO, akin to mrpt::graphslam::optimizers::CLevMarqGSO, that interfaces the solver with the graph-slam engine (mrpt::graphslam::CGraphSlamEngine). Difficulty: Easy

Resources: Some useful resources in the different directions of the project follow:

Start studying the "C++ code" section of Graph-SLAM in MRPT, and have a thorough look into mrpt::graphslam::optimize_graph_spa_levmarq to gain insight into the common classes and syntax used by MRPT in the context of Graph-SLAM. Once you get this reference, familiarize yourself with the SE-Sync work and specially with the Matlab implementation of SE-Sync, planning how to port the key ingredients of SE-Sync into the MRPT framework. Finally, once the structure of the new MRPT method has been designed, we will implement the necessary auxiliar classes within MRPT, add the dependency ROPTLIB and put all the pieces together.

  • Expected results: The new method should be able to solve test Graph-SLAM instances by calling the graph-slam application on text graph files (there is plenty of these available on the Internet). The new method should be also eligible from the graph-slam engine aplication, making it useful for more complex and reallistic datsets provided in the MRPT rawlog format.

Note that the code above should be templatized for handling both 2D and 3D cases at once. Also note that, in order to keep the integration within MRPT and dependency issues simpler, the present project should be addressed as a novel repository (within the MRPT organization, such as the SRBA module).

The outcome of the proposed project should constitute the first certifiably globally optimal C++ solver for Graph-Solver, and as such has a high potential for publication of the results in renowned robotics conferences, in case that is a plus for you :)

IMPORTANT NOTE: Given the complexity of this topic, we ask potential candidates to provide some real C++ examples (e.g. via GitHub, GitLab repo) that build cleanly and solve some related but simpler problems involved in this project. Please follow the guidelines in this document.

  • Knowledge prerequisites: C++, graphSLAM
  • Difficulty level: Middle/High

Project #9: GUI app to benchmark image feature detectors and descriptors:

  • Brief description: Computer vision-based applications for robotics typically rely on extracting meaningful features from images (usually keypoints) that are later used to perform visual odometry, SLAM, place recognition, etc. These features are generally augmented with some sort of local descriptor that identifies them. There is a large bunch of methods for detecting/describing such features, each one of them with special properties that make them more suitable for certain applications/datasets (e.g. SIFT, SURF, FAST, ORB, AKAZE, LATCH, LSD/BLD (lines), etc.). Some of the detectors already provide descriptors but some of them do not, so different combinations of detectors and descriptors can be used. Most of these methods are already implemented in OpenCV.

  • Expected results: A new GUI application that provides a benchmark for such detectors and descriptors, so that an user can open a rawlog file with image observations (or a folder with images) and then select a certain detector/descriptor that is applied to either a certain image (or pair of images if it is a stereo dataset) or the whole dataset. A set of metrics regarding their performance must be provided as the outcome, e.g.:

  • (Detectors): Repeatability, dispersion on the image, computational cost, number of found points, etc.
  • (Descriptors): Percentage of matches, descriptor distance between close matches, false positives/negatives, computational cost, etc.

The app should allow the user to tune the parameters/thresholds for the detectors and descriptors, as well as to provide some basic options such as activate/deactivate non-maximal suppression, activate/deactivate robust stereo matching, image decimation, forward and backward step-by-step play of the images and features to allow the user to inspect the evolution of the performance, application of the detector/descriptor to certain images, etc.

It should be easy to add new detectors/descriptors in the future.

  • Knowledge prerequisites: C++, OpenCV, computer vision, GUI design (Qt or wxWidgets)
  • Difficulty level: Middle

Mini-project #10: Extend existing ROS packages for grabbing and playing back MRPT datasets

mrpt-datasets

(A proposal must comprise two "mini" projects to be eligible)

  • Brief description: Many datasets are available in MRPT Rawlog format or can be imported to it (e.g. Carmen logs). ROS log format (rosbag) is incompatible, so run-time or off-line conversion tools are required if Rawlog datasets want to be used in a ROS application.
    • mrpt_rawlog ROS wiki page of existing packages. Very limited set of sensors are implemented at present.
  • Expected results: Being able to view all/most sensory data from a complex Rawlog dataset, e.g. Málaga Urban Dataset, in ROS RViz. It may be done via run-time on-the-fly conversion (easier) or by a new command-line tool capable of converting among rawlog and rosbag formats (harder).
  • Knowledge prerequisites: C++, GNU/Linux.
  • Difficulty level: Low / Middle-Low.

Project #11: GUI app for extrinsic calibration of visual and range sensors:

  • Brief description: Using a combination of several sensors in a robotic platform is a common approach to obtain relevant information of the robot environment. This information is put into a common frame of reference by means of the extrinsic calibration of the sensor system, i.e. the relative 3D poses of the different sensors. The objective of this project is to build a GUI application to estimate the extrinsic calibration of different types of sensor systems from their own readings (the observation of the scene provides useful information to find the extrinsic calibration). In order to obtain the extrinsic calibration, a cost function is usually minimized depending on geometric restrictions using scene features (like lines and planes) or sensor ego-motion (the motion of each sensor is estimated independently and their trajectories are matched to obtain the extrinsic calibration), as described in the papers by Brookshire and Teller and Taylor and Nieto.

  • Expected results: A new GUI application that opens a rawlog file containing a sequence of observations from several sensors (e.g.: a set of range cameras, a set of 2D range sensors (LiDARs), or a RGB and a range camera). The implementation of these examples using both scene features and ego-motion is required in this project. The app must show a 3D reconstruction of the scene using the computed calibration (when possible). The app must also allow the user to switch the calibration method, when several calibration methods can be applied to a particular sensor system, to compare visually their results.

  • Knowledge prerequisites: C++, OpenCV, computer vision, GUI design (Qt or wxWidgets)

  • Difficulty level: Middle

New project template:

  • Brief description:
  • Expected results:
  • Knowledge prerequisites:
  • Difficulty level:
Add a custom footer
Add a custom sidebar
Clone this wiki locally