Skip to content

Commit

Permalink
[struct_pb]improve performance (#775)
Browse files Browse the repository at this point in the history
  • Loading branch information
qicosmos authored Sep 18, 2024
1 parent 78a71ab commit cc48815
Show file tree
Hide file tree
Showing 5 changed files with 42 additions and 43 deletions.
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
cmake_minimum_required(VERSION 3.15)
project(yaLanTingLibs
VERSION 0.3.7
VERSION 0.3.8
DESCRIPTION "yaLanTingLibs"
HOMEPAGE_URL "https://github.com/alibaba/yalantinglibs"
LANGUAGES CXX
Expand Down
24 changes: 16 additions & 8 deletions include/ylt/standalone/iguana/pb_reader.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -72,15 +72,23 @@ IGUANA_INLINE void from_pb_impl(T& val, std::string_view& pb_str,
throw std::invalid_argument(
"Invalid fixed int value: too few bytes.");
}
using item_type = typename T::value_type;
size_t start = pb_str.size();
if constexpr (is_fixed_v<item_type>) {
int num = size / sizeof(item_type);
int old_size = val.size();
detail::resize(val, old_size + num);
std::memcpy(val.data() + old_size, pb_str.data(), size);
pb_str = pb_str.substr(size);
}
else {
size_t start = pb_str.size();

while (!pb_str.empty()) {
item_type item;
from_pb_impl(item, pb_str);
val.push_back(std::move(item));
if (start - pb_str.size() == size) {
break;
while (!pb_str.empty()) {
item_type item;
from_pb_impl(item, pb_str);
val.push_back(std::move(item));
if (start - pb_str.size() == size) {
break;
}
}
}
}
Expand Down
2 changes: 1 addition & 1 deletion include/ylt/version.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@
// YLT_VERSION % 100 is the sub-minor version
// YLT_VERSION / 100 % 1000 is the minor version
// YLT_VERSION / 100000 is the major version
#define YLT_VERSION 307 // 0.3.7
#define YLT_VERSION 308 // 0.3.8
2 changes: 2 additions & 0 deletions src/struct_pb/examples/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,9 @@ struct person {
std::string name;
int age;
};
#if __cplusplus >= 202002L
YLT_REFL(person, id, name, age);
#endif

int main() {
nest v{"Hi", my_struct{1, false, {3}}, 5};
Expand Down
55 changes: 22 additions & 33 deletions website/docs/zh/struct_pb/struct_pb_intro.md
Original file line number Diff line number Diff line change
@@ -1,55 +1,48 @@
# struct_pb 简介

struct_pb 是基于C++17 开发的高性能、易用、header only的protobuf格式序列化/反序列化库。
struct_pb 是基于C++17/C++20 开发的高性能、易用、header only的protobuf格式序列化/反序列化库。

## 动机
不再依赖proto文件去定义dsl message,而是通过C++ 结构体去定义需要序列化/反序列化的对象;因为没有protoc文件所以也不再依赖protoc去生成代码。通过C++17去实现可以做很多性能优化,从而获得更好的性能,比如可以支持反序列化时对字符串的零拷贝、尽可能内联和编译期计算以及字符串非memset的resize等。
不再依赖proto文件去定义dsl message,而是通过C++ 结构体去定义需要序列化/反序列化的对象;因为没有protoc文件所以也不再依赖protoc去生成代码。通过C++17/C++20去实现可以做很多性能优化,从而获得更好的性能,比如可以支持反序列化时对字符串的零拷贝、尽可能内联和编译期计算以及字符串非memset的resize等。

## 例子

### 定义结构体
```cpp
#include <ylt/struct_pb.hpp>

struct my_struct : struct_pb::base_impl<my_struct> {
int x;
bool y;
struct_pb::fixed64_t z;
};
YLT_REFL(my_struct, x, y, z);

struct nest : struct_pb::base_impl<nest> {
struct person {
int id;
std::string name;
my_struct value;
int var;
int age;
};
YLT_REFL(nest, name, value, var);
#if __cplusplus >= 202002L
YLT_REFL(person, id, name, age);
#endif
```
如果使用C++20标准,结构体为aggregate类型,且编译器版本为gcc11+, clang13+ 则不需要定义额外的宏YLT_REFL。
### 序列化
```cpp
int main() {
nest v{"Hi", {1, false, {3}}, 5}, v2{};
std::string s;
struct_pb::to_pb(v, s);
struct_pb::from_pb(v2, s);
assert(v.var == v2.var);
assert(v.value.y == v2.value.y);
assert(v.value.z == v2.value.z);
person p{1, "tom", 22};
std::string str;
struct_pb::to_pb(p, str);
person p1;
struct_pb::from_pb(p1, str);
assert(p.age == p1.age);
assert(p.name == p1.name);
assert(p.id == p1.id);
}
```
上面的这个结构体如果对应到protobuf的proto文件则是:
```
message my_struct {
int32 optional_int32 = 1;
bool optional_int64 = 2;
sfixed64 z = 3;
}
message nest {
string name = 1;
my_struct value = 2;
int32 var = 3;
int32 id = 1;
string name = 2;
int32 age = 3;
}
```

Expand Down Expand Up @@ -156,12 +149,8 @@ oneof -> `std::variant <...>`

## 约束
- 目前还只支持proto3,不支持proto2;
- 目前还没支持反射;
- 还没支持unkonwn字段;
- struct_pb 结构体必须派生于base_impl

## roadmap
- 支持proto2;
- 支持反射;
- 支持unkonwn字段;
- 去除struct_pb 结构体必须派生于base_impl的约束;

0 comments on commit cc48815

Please sign in to comment.