| title | Protobuf IDL Support |
|---|---|
| sidebar_position | 5 |
| id | protobuf-idl |
| license | Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. |
This page explains how Apache Fory works with Protocol Buffers (.proto) schemas,
how protobuf concepts map to Fory, and how to use protobuf-only Fory extension options.
- Choosing protobuf vs Fory for your use case
- Syntax and semantic differences that matter during adoption
- Supported Fory extension options in protobuf files
- Practical transition patterns from protobuf to Fory
| Situation | Recommended Format |
|---|---|
| You are building gRPC APIs and rely on protobuf tooling | Protocol Buffers |
| You need maximum object-graph performance and ref tracking | Fory |
| You need circular/shared references in serialized data | Fory |
| You need strong unknown-field behavior for wire compatibility | Protocol Buffers |
| You need native structs/classes instead of protobuf wrappers | Fory |
| Aspect | Protocol Buffers | Fory |
|---|---|---|
| Primary purpose | RPC/message contracts | High-performance object serialization |
| Encoding model | Tag-length-value | Fory binary protocol |
| Reference tracking | Not built-in | First-class (ref) |
| Circular refs | Not supported | Supported |
| Unknown fields | Preserved | Not preserved |
| Generated types | Protobuf-specific model types | Native language constructs |
| gRPC ecosystem | Native | Java/Python/Go/Rust/C#/Dart/Scala/Kotlin/JavaScript service codegen |
Fory can generate Java, Python, Go, Rust, C#, Dart, Scala, Kotlin, and
JavaScript gRPC service companions with --grpc. JavaScript browser clients are
generated with --grpc-web. Those services use normal gRPC transports but serialize request
and response payloads with Fory rather than protobuf. For broad gRPC ecosystem
tooling, schema reflection, and protobuf-native interceptors, protobuf remains
the mature/default choice.
- Idiomatic generated code: Fory IDL generates language-idiomatic classes and structs that can be used directly as domain objects.
- Faster serialization: In Fory benchmarks, Fory can be around 10x faster than protobuf for object serialization workloads.
- Better graph modeling: Shared and circular references are first-class features instead of application-level ID-link workarounds.
See benchmark details under Performance References.
Protocol Buffers
syntax = "proto3";
package example.models;
option java_package = "com.example.models";
option go_package = "example.com/models";Fory
package example.models;Fory uses one package namespace for cross-language registration. Language-specific package placement is still configurable in code generation.
Protocol Buffers
message User {
string id = 1;
string name = 2;
optional string email = 3;
int32 age = 4;
repeated string tags = 5;
map<string, string> metadata = 6;
}
enum Status {
STATUS_UNSPECIFIED = 0;
STATUS_ACTIVE = 1;
}Fory
message User [id=101] {
string id = 1;
string name = 2;
optional string email = 3;
int32 age = 4;
list<string> tags = 5;
map<string, string> metadata = 6;
}
enum Status [id=102] {
UNKNOWN = 0;
ACTIVE = 1;
}Key differences:
- Fory can assign stable type IDs directly (
[id=...]). - Fory uses
list<T>(withrepeated Tas alias). - Enum naming conventions are language-driven instead of protobuf prefix style.
Protobuf oneof is translated to a nested Fory union plus an optional field
referencing that union.
Protocol Buffers
message Event {
oneof payload {
string text = 1;
int32 number = 2;
}
}Fory-style shape after translation
message Event {
union payload {
string text = 1;
int32 number = 2;
}
optional payload payload = 1;
}Notes:
- Union case IDs are derived from the original
oneoffield numbers. - The synthetic union field uses the smallest
oneofcase number.
Protobuf imports are supported. Common well-known types map directly:
google.protobuf.Timestamp->timestampgoogle.protobuf.Duration->durationgoogle.protobuf.Any->any
| Protobuf Type | Fory Mapping |
|---|---|
bool |
bool |
int32, uint32 |
variable-length 32-bit integer kinds |
sint32 |
zigzag 32-bit integer |
int64, uint64 |
variable-length 64-bit integer kinds |
sint64 |
zigzag 64-bit integer |
fixed32, fixed64 |
fixed-width unsigned integer kinds |
sfixed32, sfixed64 |
fixed-width signed integer kinds |
float, double |
float32, float64 |
string, bytes |
string, bytes |
repeated T |
list<T> |
map<K, V> |
map<K, V> |
optional T |
optional T |
oneof |
union + optional union reference field |
int64 [(fory).type = "tagged int64"] |
tagged int64 encoding |
uint64 [(fory).type = "tagged uint64"] |
tagged uint64 encoding |
Fory-specific options in .proto use the (fory). prefix.
option (fory).enable_auto_type_id = true;
message TreeNode {
TreeNode parent = 1 [(fory).weak_ref = true];
repeated TreeNode children = 2 [(fory).ref = true];
}| Option | Type | Description |
|---|---|---|
(fory).use_record_for_java_message |
bool | Generate Java records for all messages in this file |
(fory).polymorphism |
bool | Enable polymorphic serialization metadata by default |
(fory).enable_auto_type_id |
bool | Auto-generate type IDs when omitted (compiler default is true) |
(fory).evolving |
bool | Default schema-evolution behavior for messages |
(fory).go_nested_type_style |
string | Go nested naming style: underscore (default) or camelcase |
(fory).swift_namespace_style |
string | Swift namespace style: enum (default) or flatten; applies only when package is non-empty |
| Option | Applies To | Type | Description |
|---|---|---|---|
(fory).id |
message, enum | int | Explicit type ID for registration |
(fory).alias |
message, enum | string | Alternate name used for auto-ID hashing |
(fory).evolving |
message | bool | Override file-level evolution setting |
(fory).use_record_for_java |
message | bool | Generate Java record for this message |
(fory).deprecated |
message, enum | bool | Mark type as deprecated |
(fory).namespace |
message | string | Override default package-based namespace |
| Option | Type | Description |
|---|---|---|
(fory).ref |
bool | Enable reference tracking for this field |
(fory).nullable |
bool | Treat field as nullable (optional) |
(fory).weak_ref |
bool | Generate weak pointer semantics (C++/Rust codegen) |
(fory).thread_safe_pointer |
bool | Rust ref carrier selection; default true uses Arc/ArcWeak, explicit false uses Rc/RcWeak |
(fory).deprecated |
bool | Mark field as deprecated |
(fory).type |
string | Primitive override for tagged 64-bit integer encoding |
Reference option behavior:
weak_ref = trueimplies ref tracking.- For
repeatedfields,(fory).ref = trueapplies to list elements. - For
map<K, V>fields,(fory).ref = trueapplies to map values. weak_refandthread_safe_pointerare codegen hints for C++/Rust.thread_safe_pointerdefaults totrue; it changes only the generated Rust pointer carrier and does not change the wire format.- In Rust codegen,
(fory).weak_ref = trueusesArcWeakby default and switches toRcWeakonly when(fory).thread_safe_pointer = false.
message Graph {
Node root = 1 [(fory).ref = true];
repeated Node nodes = 2 [(fory).ref = true];
map<string, Node> cache = 3 [(fory).ref = true];
Node parent = 4 [(fory).weak_ref = true];
Node local = 5 [(fory).ref = true, (fory).thread_safe_pointer = false];
}Protobuf itself does not preserve shared/cyclic object graphs. With Fory protobuf extensions, you can opt into graph semantics.
Without Fory ref options (protobuf-style IDs):
message TreeNode {
string id = 1;
string parent_id = 2;
repeated string child_ids = 3;
}With Fory ref options (object graph):
message TreeNode {
TreeNode parent = 1 [(fory).weak_ref = true];
repeated TreeNode children = 2 [(fory).ref = true];
}- Keep package names stable.
- Replace
repeated Twithlist<T>(or keeprepeatedalias). - Add explicit
[id=...]where you need stable numeric registration.
oneof->union+ optional union field.- Map protobuf well-known types to Fory primitives (
timestamp,duration,any).
Where protobuf used manual ID links for object graphs, switch to Fory ref
modifiers (and optional ref(weak=true) where needed).
Replace protobuf generation steps with the Fory compiler invocation for target languages.
For supported service outputs, add --grpc to emit gRPC companion code:
foryc api.proto --java_out=./generated/java --python_out=./generated/python --go_out=./generated/go --rust_out=./generated/rust --csharp_out=./generated/csharp --dart_out=./generated/dart --scala_out=./generated/scala --kotlin_out=./generated/kotlin --javascript_out=./generated/javascript --grpcGenerated Java service files compile against grpc-java, generated Python service
modules use grpc.aio by default, generated Rust service files import tonic
and bytes, generated Go service files import grpc-go, generated JavaScript
Node.js service files import @grpc/grpc-js, generated C# service files import
Grpc.Core.Api types, generated Dart service files import package:grpc,
generated Scala service files compile against grpc-java, and generated Kotlin
service files compile against grpc-java and grpc-kotlin. Add those dependencies
in your application build; Fory packages do not add gRPC as a hard dependency.
Use --grpc-python-mode=sync for sync Python grpcio companions. Use
--grpc-web with JavaScript output to generate browser clients that import
grpc-web.
Protobuf oneof fields are translated to Fory union fields inside request and
response messages. Direct union RPC request or response types are not part of
normal protobuf RPC syntax.
For staged transitions, keep both formats in parallel and verify payload-level parity with integration tests.
You can run protobuf and Fory in parallel during a staged transition:
public byte[] serialize(Object obj, Format format) {
if (format == Format.PROTOBUF) {
return ((MessageLite) obj).toByteArray();
}
return fory.serialize(obj);
}Use translators at service boundaries while internal object-graph heavy paths migrate first.
- Benchmarks: https://fory.apache.org/docs/benchmarks/
- Benchmark code: https://github.com/apache/fory/tree/main/benchmarks
Use protobuf when your primary concern is API contracts and gRPC ecosystem integration. Use Fory when object-graph performance, native models, and reference semantics are the primary concern.