phmap::parallel_flat_hash_map to manage lidar pointcloud?

[maomao0220]

Hello, when using std::unordered_map to store lidar point clouds, it is very time-consuming and CPU-intensive for traversing, inserting, and deleting. Now I want to use phmap::parallel_flat_hash_map storage and manage the point cloud data of LiDAR, how can I use it? Is there a guidance document for storing lidar data? At present, it mainly uses functions such as traversing the stored historical point cloud points, deleting a point, inserting a frame point cloud, and clearing.

[greg7mdp]

Well, you can use phmap::flat_hash_map in the same way you currently use std::unordered_map, and it will already be faster and use less memory.

However, the big gain is to use many threads and use phmap::parallel_flat_hash_map with internal mutexes.

If you have a working example of a C++ program you can share that implements one of the algorithms you are interested in, I can make further suggestions.

[maomao0220]

#pragma once
#include "iostream"
#include "unordered_map"
#include
#include
#include
#include <parallel_hashmap/phmap.h>
#include <shared_mutex>

#define HASH_PRIME 116101
#define MAX_N 201326611

class Hash_3D_key {
public:
int64_t x, y, z;

Hash_3D_key(int64_t vx = 0, int64_t vy = 0, int64_t vz = 0) :
	x(vx), y(vy), z(vz) {}

bool operator==(const Hash_3D_key &other) const { return (x == other.x && y == other.y && z == other.z); }

};

// Hash value
namespace std {
template <>
struct hash<Hash_3D_key> {
int64_t operator()(const Hash_3D_key &s) const {
using std::hash;
using std::size_t;
return ((((s.z) * HASH_PRIME) % MAX_N + (s.y)) * HASH_PRIME) % MAX_N + (s.x);
}
};
}

template <typename data_type = float, typename T = void *>
struct Hash_map_3d {
std::mutex m_map_mutex;
double N = 1;
// using hash_3d_T = std::unordered_map<Hash_3D_key, T>;
// using hash_3d_T = phmap::flat_hash_map<Hash_3D_key, T>;
// using hash_3d_T = phmap::parallel_flat_hash_map<Hash_3D_key, T>;

using hash_3d_T = phmap::parallel_flat_hash_map<Hash_3D_key, T, phmap::priv::hash_default_hash<Hash_3D_key>, phmap::priv::hash_default_eq<Hash_3D_key>, phmap::priv::Allocator<phmap::priv::Pair<Hash_3D_key, T>>, 8, std::shared_mutex>; // std::shared_mutex std::mutex

hash_3d_T m_map_3d_hash_map;
Hash_map_3d() {
	if (std::is_same<float, data_type>::value) {
		N = 1e5; 
	} else if (std::is_same<double, data_type>::value) {
		N = 1e7;
	}
}

void reserve(long size) {
	m_map_3d_hash_map.reserve(size);
}

void insert(const data_type &x, const data_type &y, const data_type &z, const T &target) { m_map_3d_hash_map[Hash_3D_key(x * N, y * N, z * N)] = target; }

bool if_exist(const data_type &x, const data_type &y, const data_type &z) {
	if (m_map_3d_hash_map.find(Hash_3D_key(x * N, y * N, z * N)) == m_map_3d_hash_map.end()) {
		return false;
	}
	return true;
}

void clear() { m_map_3d_hash_map.clear(); }

void all_data(std::vector<T> &all_data) {
	all_data.clear();
	all_data.reserve(m_map_3d_hash_map.size());
	for (auto it = m_map_3d_hash_map.begin(); it != m_map_3d_hash_map.end(); it++) {
		all_data.push_back(it->second);
	}
}

void erase_data_out_of_range(const data_type &x_min, const data_type &x_max, const data_type &y_min, const data_type &y_max, const data_type &z_min, const data_type &z_max) {
	for (auto it = m_map_3d_hash_map.begin(); it != m_map_3d_hash_map.end();) {
		if (it->first.x < x_min * N || it->first.x > x_max * N || it->first.y < y_min * N || it->first.y > y_max * N) {
			it = m_map_3d_hash_map.erase(it);
		} else {
			it++;
		}
	}
}

void erase_data(const data_type &x, const data_type &y, const data_type &z) {
	m_map_3d_hash_map.erase(Hash_3D_key(x * N, y * N, z * N));
}

int total_size() { return m_map_3d_hash_map.size(); }

};

My relevant project code looks like above: the above code is developed based on std::unordered_map。Now change std::unordered_map to phmap::p arallel_flat_hash_map, how should the corresponding implementation of the reserve() insert() if_exist() clear all_data() erase_data_out_of_range() and erase_data() functions be rewritten? Looking forward to your reply, thank you

[maomao0220]

The main function to be realized is: based on ROS, receive real-time point cloud data of each frame, insert it into the point cloud map that m_map_3d_hash_map maintain, and output the inserted point cloud map in real time @greg7mdp

[greg7mdp]

@maomao0220 please include a piece of code that I can compile by itself and I'll look into it. The code you posted above has a lot of issues like empty #include statements and others.