Skip to content

Commit

Permalink
Modularize, add pjmsg_mcap backend
Browse files Browse the repository at this point in the history
  • Loading branch information
asherikov committed Jan 9, 2025
1 parent b32b913 commit 3674946
Show file tree
Hide file tree
Showing 95 changed files with 23,895 additions and 47 deletions.
4 changes: 3 additions & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,4 +37,6 @@ jobs:
- run: make dep_install PKG=intrometry
- run: make intrometry BUILD_PROFILE=scan_build
- run: make intrometry
- run: make test PKG=intrometry
- run: make test PKG=intrometry_frontend
- run: make test PKG=intrometry_pjmsg_mcap
- run: make test PKG=intrometry_pjmsg_topic
38 changes: 38 additions & 0 deletions .utils/make/intrometry.mk
Original file line number Diff line number Diff line change
@@ -1,5 +1,43 @@
FIND_SOURCES=find ./test/ ./src ./include/ -iname "*.h" -or -iname "*.cpp"

# 0ebaf69 = releases/cpp/v1.4.1
MCAP_SHA=0ebaf69efa2dd5dbe33a882ce51caa8451d518d2
MCAP_DIR=pjmsg_mcap/src/3rdparty/mcap

# 2.2.6
FASTCDR_SHA=9149259a1eb54a9b1104a9f622f89b00a66b6011
FASTCDR_DIR=pjmsg_mcap/src/3rdparty/fastcdr


format:
${FIND_SOURCES} | xargs ${CLANG_FORMAT} -verbose -i

update_mcap:
-git remote add mcap https://github.com/foxglove/mcap.git --no-tags
git fetch --depth=1 mcap ${MCAP_SHA}
git rm --ignore-unmatch -rf ${MCAP_DIR}
rm -rf ${MCAP_DIR}
env GIT_LFS_SKIP_SMUDGE=1 git read-tree --prefix=${MCAP_DIR} -u ${MCAP_SHA}
find ${MCAP_DIR} \
-not -wholename "${MCAP_DIR}/cpp/mcap*" \
-not -wholename "${MCAP_DIR}/cpp" \
-not -wholename "${MCAP_DIR}" \
| xargs rm -rf

update_fastcdr:
-git remote add fastcdr https://github.com/eProsima/Fast-CDR.git --no-tags
git fetch --depth=1 fastcdr ${FASTCDR_SHA}
git rm --ignore-unmatch -rf ${FASTCDR_DIR}
rm -rf ${FASTCDR_DIR}
env GIT_LFS_SKIP_SMUDGE=1 git read-tree --prefix=${FASTCDR_DIR} -u ${FASTCDR_SHA}
find ${FASTCDR_DIR} \
-not -wholename "${FASTCDR_DIR}/cmake/common/check_configuration.cmake" \
-not -wholename "${FASTCDR_DIR}/cmake/common" \
-not -wholename "${FASTCDR_DIR}/cmake/packaging*" \
-not -wholename "${FASTCDR_DIR}/cmake" \
-not -wholename "${FASTCDR_DIR}/include*" \
-not -wholename "${FASTCDR_DIR}/src*" \
-not -wholename "${FASTCDR_DIR}/LICENSE" \
-not -wholename "${FASTCDR_DIR}" \
| xargs rm -rf
cp ${FASTCDR_DIR}/../CMakeLists.txt.fastcdr ${FASTCDR_DIR}/CMakeLists.txt
94 changes: 69 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,27 @@ library <https://github.com/asherikov/ariles> instead of macro.
Doxygen documentation: <https://asherikov.github.io/intrometry/doxygen/group__API.html>


Design
------

- Intrometry consist of multiple modules: one frontend and multiple backends.
Frontend defines common API, and backends implement various storage options.
Typically application code should depend on a single backend, explicit
dependency on the frontend is not needed in this case.

- `intrometry` tries to interfere as little as possible with the execution of
other code, in particular this means that:
- `intrometry` would rather lose data than block calling thread for more
time than is needed for copying it;
- `intrometry` would rather lose data than run out of memory;
- `intrometry` ignores failures, e.g., due to failed initialization, which
allows to disable it by simply providing an empty id.

- Frontend is meant to be agnostic of any possible data storage or transmission
dependencies, in particular ROS. Backends may depend on various libraries in
order to handle data.


Comparison
----------

Expand Down Expand Up @@ -58,7 +79,7 @@ class ArilesDebug : public ariles2::DefaultBase
#include ARILES2_INITIALIZE
}
...
intrometry::Sink sink;
intrometry::pjmsg_topic::Sink sink;
ArilesDebug debug;
...
sink.initialize("my_sink");
Expand Down Expand Up @@ -99,41 +120,64 @@ Key features:
- automatic generation of variable names and support for various types suchs as
`Eigen` matrices, stl vectors, maps, etc is provided by `ariles`.

Methods:

- `initialize()`, `assign()`, and `retract()` methods are "heavy" and are meant
to be used sparingly.
- `write()` is a "light" method that should be suitable for soft real time
applications.


Backends
--------

### `pjmsg_topic`

Creates a dedicated ROS2 node and spawns a publishing thread that takes care of
sending data using `plotjuggler_msgs` at a given frequency. Recorded ROS bags
can be viewed with `PlotJuggler` <https://plotjuggler.io/>. Keep in mind that
`PlotJuggler` has a flaw that may result in a collision of metric names
<https://github.com/facontidavide/PlotJuggler/pull/339> -- `intrometry` makes
an effort to avoid this, but it is still possible.

### `pjmsg_mcap`

Serializes metrics to `plotjuggler_msgs` and writes them directly to `mcap`
files. All serialization logic and schemas are compiled in, so this backend
does NOT depend on any ROS components. The resulting files can also be viewed
by `PlotJuggler`.


Using backends with cmake
-------------------------

```
find_package(intrometry_pjmsg_mcap REQUIRED)
target_link_libraries(my_library intrometry::pjmsg_mcap)
```


Dependencies
------------

### Frontend
- `C++17` compatible compiler
- `cmake`
- `ariles` (`ariles2_namevalue2_ws`) <https://github.com/asherikov/ariles/tree/pkg_ws_2>
- `thread_supervisor` <https://github.com/asherikov/thread_supervisor>
- `rclcpp` / `plotjuggler_msgs`
- `ariles` (`ariles2_core_ws`) <https://github.com/asherikov/ariles/tree/pkg_ws_2>

### Backends

Design
------
`pjmsg_topic`

- Sink creates a dedicated ROS2 node and spawns a publishing thread that
takes care of sending data using `plotjuggler_msgs` at a given frequency.
Recorded ROS bags can be viewed with `PlotJuggler` <https://plotjuggler.io/>.
Keep in mind that `PlotJuggler` has a flaw that may result in a collision of
metric names <https://github.com/facontidavide/PlotJuggler/pull/339> --
`intrometry` makes an effort to avoid this, but it is still possible.
- `thread_supervisor` <https://github.com/asherikov/thread_supervisor>
- `rclcpp` / `plotjuggler_msgs`
- `ariles` (`ariles2_namevalue2_ws`) <https://github.com/asherikov/ariles/tree/pkg_ws_2>

- `intrometry` tries to interfere as little as possible with the execution of
other code, in particular this means that:
- `intrometry` would rather lose data than block calling thread for more
time than is needed for copying it;
- `intrometry` ignores failures, e.g., due to failed initialization, which
allows to disable it by simply providing an empty id.
`pjmsg_mcap`

- `thread_supervisor` <https://github.com/asherikov/thread_supervisor>
- `ariles` (`ariles2_namevalue2_ws`) <https://github.com/asherikov/ariles/tree/pkg_ws_2>

- API:
- Public API is ROS2 / `plotjuggler_msgs` agnostic, other backends may
be implemented in the future.
- `initialize()`, `assign()`, and `retract()` methods are "heavy" and are
meant to be used sparingly.
- `write()` is a "light" method that should be suitable for soft real time
applications.

TODO
====
Expand Down
7 changes: 4 additions & 3 deletions api/CMakeLists.txt → frontend/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
cmake_minimum_required(VERSION 3.8)
project(intrometry_api VERSION 1.0.0 LANGUAGES CXX)
project(intrometry_frontend VERSION 1.0.0 LANGUAGES CXX)

if(CCWS_CLANG_TIDY)
set(CMAKE_CXX_CLANG_TIDY "${CCWS_CLANG_TIDY}" CACHE STRING "" FORCE)
Expand All @@ -25,8 +25,9 @@ find_package(ariles2-core REQUIRED)


add_library(${PROJECT_NAME} INTERFACE)
set_target_properties(${PROJECT_NAME} PROPERTIES EXPORT_NAME frontend)
target_link_libraries(${PROJECT_NAME}
INTERFACE ariles2-core
INTERFACE ariles2::core
)
target_include_directories(${PROJECT_NAME} INTERFACE
$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/include>
Expand Down Expand Up @@ -60,7 +61,7 @@ export(EXPORT ${PROJECT_NAME}

install(EXPORT ${PROJECT_NAME}
FILE ${PROJECT_NAME}Targets.cmake
NAMESPACE ${PROJECT_NAME}::
NAMESPACE intrometry::
DESTINATION share/${PROJECT_NAME}/
)

Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion api/package.xml → frontend/package.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0"?>
<package format="2">
<name>intrometry_api</name>
<name>intrometry_frontend</name>
<version>1.0.0</version>
<description>Inner telemetry collector</description>
<maintainer email="[email protected]">Alexander Sherikov</maintainer>
Expand Down
109 changes: 109 additions & 0 deletions pjmsg_mcap/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
cmake_minimum_required(VERSION 3.8)
project(intrometry_pjmsg_mcap VERSION 1.0.0 LANGUAGES CXX)

if(CCWS_CLANG_TIDY)
set(CMAKE_CXX_CLANG_TIDY "${CCWS_CLANG_TIDY}" CACHE STRING "" FORCE)
endif()
if(CCWS_CXX_FLAGS)
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CCWS_CXX_FLAGS}")
set (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${CCWS_LINKER_FLAGS}")
else()
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_VERBOSE_MAKEFILE ON)

if(NOT CMAKE_CXX_STANDARD)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
endif()

if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
add_compile_options(-Wall -Wextra -Wpedantic -Werror)
endif()
endif()

find_package(intrometry_frontend REQUIRED)
find_package(thread_supervisor REQUIRED)
find_package(ariles2-namevalue2 REQUIRED)

add_subdirectory(src/3rdparty/)


add_library(${PROJECT_NAME} SHARED
src/intrometry.cpp
)
set_target_properties(${PROJECT_NAME} PROPERTIES EXPORT_NAME pjmsg_mcap)
target_link_libraries(${PROJECT_NAME}
PUBLIC intrometry::frontend
PRIVATE ariles2::namevalue2
PRIVATE thread_supervisor::thread_supervisor
PRIVATE fastcdr
)
target_include_directories(${PROJECT_NAME} PUBLIC
$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:include>
)
target_include_directories(${PROJECT_NAME}
SYSTEM PRIVATE src/3rdparty/mcap/cpp/mcap/include/
)
set_property(TARGET ${PROJECT_NAME} PROPERTY INTERFACE_${PROJECT_NAME}_MAJOR_VERSION ${PROJECT_VERSION_MAJOR})
set_property(TARGET ${PROJECT_NAME} APPEND PROPERTY COMPATIBLE_INTERFACE_STRING ${PROJECT_VERSION_MAJOR})

install(
TARGETS ${PROJECT_NAME} EXPORT ${PROJECT_NAME}
INCLUDES DESTINATION include
)

install(TARGETS ${PROJECT_NAME}
ARCHIVE DESTINATION lib
LIBRARY DESTINATION lib
RUNTIME DESTINATION bin
)

install(DIRECTORY include/intrometry
DESTINATION include
FILES_MATCHING PATTERN "*.h"
)


if(NOT DEFINED BUILD_TESTING OR BUILD_TESTING)
enable_testing()
add_subdirectory(test)
endif()


# ---
# cmake package stuff
export(EXPORT ${PROJECT_NAME}
FILE "${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Targets.cmake"
NAMESPACE ${PROJECT_NAME}::
)

install(EXPORT ${PROJECT_NAME}
FILE ${PROJECT_NAME}Targets.cmake
NAMESPACE intrometry::
DESTINATION share/${PROJECT_NAME}/
)


include(CMakePackageConfigHelpers)

write_basic_package_version_file(
"${PROJECT_BINARY_DIR}/${PROJECT_NAME}ConfigVersion.cmake"
VERSION ${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR}.${PROJECT_VERSION_PATCH}
COMPATIBILITY SameMajorVersion
)
file(
WRITE
"${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake"
"include(\"\${CMAKE_CURRENT_LIST_DIR}/${PROJECT_NAME}Targets.cmake\")\n"
"include(CMakeFindDependencyMacro)\n"
"find_dependency(ariles2_namevalue2_ws)"
)

install(
FILES
"${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake"
"${PROJECT_BINARY_DIR}/${PROJECT_NAME}ConfigVersion.cmake"
DESTINATION share/${PROJECT_NAME}/
)
# ---
58 changes: 58 additions & 0 deletions pjmsg_mcap/include/intrometry/pjmsg_mcap/sink.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
/**
@file
@author Alexander Sherikov
@copyright 2024 Alexander Sherikov. Licensed under the Apache License,
Version 2.0. (see LICENSE or http://www.apache.org/licenses/LICENSE-2.0)
@brief Sink class.
*/

#pragma once

#include "intrometry/api/sink.h"


namespace intrometry::pjmsg_mcap
{
namespace sink
{
class Parameters
{
public:
/**
* publish rate (system clock),
* data written at higher rate is going to overwrite unpublished data
*/
std::size_t rate_;
/// id of the sink, disables publishing if empty
std::string id_;

public:
Parameters(const std::string &id = ""); // NOLINT
Parameters(const char *id = ""); // NOLINT

Parameters &rate(const std::size_t value);
Parameters &id(const std::string &value);
};

class Implementation;
} // namespace sink


/**
* @brief Publish data.
*
* @ingroup API
*/
class Sink : public SinkPIMPLBase<sink::Parameters, sink::Implementation>
{
public:
using SinkPIMPLBase::SinkPIMPLBase;
~Sink();

bool initialize();
void assign(const ariles2::DefaultBase &source, const Source::Parameters &parameters = Source::Parameters());
void retract(const ariles2::DefaultBase &source);
void write(const ariles2::DefaultBase &source, const uint64_t timestamp = 0);
};
} // namespace intrometry::pjmsg_mcap
Loading

0 comments on commit 3674946

Please sign in to comment.