Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[struct_pb]improve performance #775

Merged
merged 3 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
cmake_minimum_required(VERSION 3.15)
project(yaLanTingLibs
VERSION 0.3.7
VERSION 0.3.8
DESCRIPTION "yaLanTingLibs"
HOMEPAGE_URL "https://github.com/alibaba/yalantinglibs"
LANGUAGES CXX
Expand Down
24 changes: 16 additions & 8 deletions include/ylt/standalone/iguana/pb_reader.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -72,15 +72,23 @@ IGUANA_INLINE void from_pb_impl(T& val, std::string_view& pb_str,
throw std::invalid_argument(
"Invalid fixed int value: too few bytes.");
}
using item_type = typename T::value_type;
size_t start = pb_str.size();
if constexpr (is_fixed_v<item_type>) {
int num = size / sizeof(item_type);
int old_size = val.size();
detail::resize(val, old_size + num);
std::memcpy(val.data() + old_size, pb_str.data(), size);
pb_str = pb_str.substr(size);
}
else {
size_t start = pb_str.size();

while (!pb_str.empty()) {
item_type item;
from_pb_impl(item, pb_str);
val.push_back(std::move(item));
if (start - pb_str.size() == size) {
break;
while (!pb_str.empty()) {
item_type item;
from_pb_impl(item, pb_str);
val.push_back(std::move(item));
if (start - pb_str.size() == size) {
break;
}
}
}
}
Expand Down
2 changes: 1 addition & 1 deletion include/ylt/version.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@
// YLT_VERSION % 100 is the sub-minor version
// YLT_VERSION / 100 % 1000 is the minor version
// YLT_VERSION / 100000 is the major version
#define YLT_VERSION 307 // 0.3.7
#define YLT_VERSION 308 // 0.3.8
2 changes: 2 additions & 0 deletions src/struct_pb/examples/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,9 @@ struct person {
std::string name;
int age;
};
#if __cplusplus >= 202002L
YLT_REFL(person, id, name, age);
#endif

int main() {
nest v{"Hi", my_struct{1, false, {3}}, 5};
Expand Down
55 changes: 22 additions & 33 deletions website/docs/zh/struct_pb/struct_pb_intro.md
Original file line number Diff line number Diff line change
@@ -1,55 +1,48 @@
# struct_pb 简介

struct_pb 是基于C++17 开发的高性能、易用、header only的protobuf格式序列化/反序列化库。
struct_pb 是基于C++17/C++20 开发的高性能、易用、header only的protobuf格式序列化/反序列化库。

## 动机
不再依赖proto文件去定义dsl message,而是通过C++ 结构体去定义需要序列化/反序列化的对象;因为没有protoc文件所以也不再依赖protoc去生成代码。通过C++17去实现可以做很多性能优化,从而获得更好的性能,比如可以支持反序列化时对字符串的零拷贝、尽可能内联和编译期计算以及字符串非memset的resize等。
不再依赖proto文件去定义dsl message,而是通过C++ 结构体去定义需要序列化/反序列化的对象;因为没有protoc文件所以也不再依赖protoc去生成代码。通过C++17/C++20去实现可以做很多性能优化,从而获得更好的性能,比如可以支持反序列化时对字符串的零拷贝、尽可能内联和编译期计算以及字符串非memset的resize等。

## 例子

### 定义结构体
```cpp
#include <ylt/struct_pb.hpp>

struct my_struct : struct_pb::base_impl<my_struct> {
int x;
bool y;
struct_pb::fixed64_t z;
};
YLT_REFL(my_struct, x, y, z);

struct nest : struct_pb::base_impl<nest> {
struct person {
int id;
std::string name;
my_struct value;
int var;
int age;
};
YLT_REFL(nest, name, value, var);
#if __cplusplus >= 202002L
YLT_REFL(person, id, name, age);
#endif
```

如果使用C++20标准,结构体为aggregate类型,且编译器版本为gcc11+, clang13+ 则不需要定义额外的宏YLT_REFL。

### 序列化
```cpp
int main() {
nest v{"Hi", {1, false, {3}}, 5}, v2{};
std::string s;
struct_pb::to_pb(v, s);
struct_pb::from_pb(v2, s);
assert(v.var == v2.var);
assert(v.value.y == v2.value.y);
assert(v.value.z == v2.value.z);
person p{1, "tom", 22};
std::string str;
struct_pb::to_pb(p, str);

person p1;
struct_pb::from_pb(p1, str);
assert(p.age == p1.age);
assert(p.name == p1.name);
assert(p.id == p1.id);
}
```
上面的这个结构体如果对应到protobuf的proto文件则是:
```
message my_struct {
int32 optional_int32 = 1;
bool optional_int64 = 2;
sfixed64 z = 3;
}

message nest {
string name = 1;
my_struct value = 2;
int32 var = 3;
int32 id = 1;
string name = 2;
int32 age = 3;
}
```

Expand Down Expand Up @@ -156,12 +149,8 @@ oneof -> `std::variant <...>`

## 约束
- 目前还只支持proto3,不支持proto2;
- 目前还没支持反射;
- 还没支持unkonwn字段;
- struct_pb 结构体必须派生于base_impl

## roadmap
- 支持proto2;
- 支持反射;
- 支持unkonwn字段;
- 去除struct_pb 结构体必须派生于base_impl的约束;
Loading