The main idea behind this project is to create an Unity application that lets the user experience a virtual bike tour recorded using a 360 camera. The user will use an indoor bike setup that will control the video speed based on effort and compete with other players or AI opponents. Essentially, it would resemble a classic track racing game, but with bikes!
The project drew inspiration from the Rouvy project. Some related projects available are:
Aside from the 360 video, other data will be included in the application, such as bio-metric indexes from external sensors (heart-rate), GPS coordinates and elevation, and other measured or estimated quantities (time, speed, track progress, etc), that will be displayed in the user interface.
Here’s a short demo on the current progress of the project: VIDEO LINK
- [X] In the application, a 360 video has been imported and projected on the inner surface of a 3D sphere. The main camera has been placed on the center of that sphere which can be dragged with the mouse (for now).
- [X] The video playback is controlled by pressing the spacebar key and played at the target FPS set before the play-mode. A progress bar illustrates the current time and duration of the video.
- [X] A minimap is displayed with the GPS track projected onto the actual world map, which in turn is assembled with the map tiles from the OpenStreetMap (OSM) project. Based on the recorded GPS coordinates (
.gpxfile format), map tiles are downloaded at the beginning of play-mode and assembled into a UnityTileMap. The GPS track is rendered as aLineRenderercomponent on top of the tilemap. A second minimap camera projects its contents with aRenderTextureonto theUI Canvas.
Assets ├── Materials # Standard Material directory. ├── Player # Contains a biker model and animation ├── Plugins # External unmanaged (C++) Plugins (.dll or .so files) ├── Prefabs # Prefab dir. Contains Minimap and VideoPlayer prefabs ├── Resources # Data file. │ ├── GPS_Traces # GPX data files │ └── Video # 360 Video files ├── Scenes # Standard scene directory ├── Scripts # Scripts directory │ ├── ExternalLibs # 3rd party C# libraries │ │ ├── AwesomeTiles # Lib to manage OSM tiles │ │ └── GPX # Lib to manage .gpx data files │ ├── Player # Player related scripts (WIP) │ ├── UI # UI related scripts │ └── Video # Video related scripts └── Textures # Textures
… in no particular order.
- [ ] Show list of other players (chart).
- [ ] Show Statistics/Measurements (Time, Speed, Km, Heart-rate, etc..)
- [ ] Combine track progress-bar with challenge-level/elevation data.
- [ ] Percentage elevation signs shown on road (hard - optional, requires computer vision methods).
- [ ] Have different play-modes: 3D Simulation view / 360 POV Video view / Location in Map view.
- [ ] Add 3D player (biker) animation from Infinity Bike project
- [ ] Add AI Opponents (simple case - let them chase way-points in map).
- [ ] Implement a frame interpolation method for real-time usage.
- [ ] Have minimap camera follow the player during play-mode (GTA-style minimap).
- [ ] Have analogue input that controls the video speed.
- [ ] Integrate VR setup for the 360 video navigation.
- [ ] (Fill this list....)
Here’s a list of some resources that might be useful to this project:
- Bike Tour 360 Videos
- GPS Data
- Infinity Bike Project
- Multi-player tutorial
- 3D Terrain with Google Maps
- Blender plugin
- NASA Earth Data
There are currently some issues with the project that I haven’t been able to solve. The major bugs that need to be solved immediately are:
- Frame Interpolation: My approach on frame interpolation, i.e., to create
an external plugin in (C++ code) and import the
.dll(or.so) file in Unity, has the disadvantage that adds overhead on run-time (x10 times slower that the original performance in a native application). - Downloading OSM tilemaps on application startup may not always work correctly. Perhaps, because the images are downloaded asynchronously and are not available from the start, causes the rest of the application to crush because of missing information. Starting the application over and over again, until all tilemaps are downloaded, solves the issue (since they are load from file). But this is not the desired behavior.
- NOTE: The project was developed in a Linux OS, so compatibility with a Windows setup is uncertain. For example, the prefabs will probably have missing components when loaded on Windows.