-
Notifications
You must be signed in to change notification settings - Fork 4
repo: Add READMEs #2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,36 @@ | ||||||
| # Envoy Modules | ||||||
|
|
||||||
| This repository hosts buildable out-of-tree extensions for [Envoy Proxy](https://www.envoyproxy.io/). These modules extend Envoy's functionality without requiring modifications to the core Envoy codebase. | ||||||
|
|
||||||
| ## Module Types | ||||||
|
|
||||||
| Envoy supports three primary approaches for building extensions: | ||||||
|
|
||||||
| ### [Dynamic Modules](./dynamic/) | ||||||
| Native extensions written in C++, Rust, or Go that are compiled as shared libraries and loaded dynamically at runtime. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
AFAIK, the |
||||||
|
|
||||||
| ### [Go Extensions](./go/) | ||||||
| Extensions written using the Envoy Go SDK, providing additional features and a Go-native development experience. | ||||||
|
|
||||||
| ### [WebAssembly (Wasm) Modules](./wasm/) | ||||||
| Cross-platform extensions compiled from C++, Rust, or other languages to WebAssembly bytecode. | ||||||
|
|
||||||
| ## Getting Started | ||||||
|
|
||||||
| Each module type has its own development workflow and requirements. Please refer to the README in each directory for specific instructions: | ||||||
|
|
||||||
| - [Dynamic Modules Documentation](./dynamic/README.md) | ||||||
| - [Go Extensions Documentation](./go/README.md) | ||||||
| - [Wasm Modules Documentation](./wasm/README.md) | ||||||
|
|
||||||
| ## Contributing | ||||||
|
|
||||||
| Contributions are welcome! Please ensure your modules: | ||||||
| - Follow Envoy's extension best practices | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what is the best practices? |
||||||
| - Include comprehensive documentation | ||||||
| - Provide example configurations | ||||||
| - Include tests where applicable | ||||||
|
|
||||||
| ## License | ||||||
|
|
||||||
| This repository is licensed under the same terms as Envoy Proxy. See [LICENSE](./LICENSE) for details. | ||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,81 @@ | ||||||
| # Dynamic Modules | ||||||
|
|
||||||
| Dynamic modules are native extensions for Envoy that are compiled as shared libraries and loaded at runtime. These modules can be written in C++, Rust, or Go. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. May be remove the |
||||||
|
|
||||||
| ## Overview | ||||||
|
|
||||||
| Dynamic modules provide the highest performance and most direct integration with Envoy's core APIs. They are loaded as `.so` (Linux), `.dylib` (macOS), or `.dll` (Windows) files. | ||||||
|
|
||||||
| ## Supported Languages | ||||||
|
|
||||||
| ### C++ | ||||||
| - Direct access to Envoy's C++ APIs | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Note, dynamic module couldn't access Envoy C++ API directly. The dynamic modules are build based a C API (https://github.com/envoyproxy/envoy/blob/main/source/extensions/dynamic_modules/abi.h) that exposed by the Envoy. It has very big difference with the actual C++ API that used by our actual native extension (Like https://github.com/envoyproxy/envoy/blob/main/envoy/http/filter.h) |
||||||
| - Best performance characteristics | ||||||
| - Requires matching Envoy ABI version | ||||||
|
|
||||||
| ### Rust | ||||||
| - Safe systems programming | ||||||
| - Can use FFI to interact with Envoy's C++ APIs | ||||||
| - Growing ecosystem of Envoy Rust bindings | ||||||
|
|
||||||
| ### Go | ||||||
| - Uses cgo for C++ interop | ||||||
| - Note: For pure Go extensions, consider using the [Go SDK](../go/) | ||||||
|
|
||||||
| ## Building Dynamic Modules | ||||||
|
|
||||||
| ### Prerequisites | ||||||
| - Matching Envoy version and build environment | ||||||
| - Bazel or CMake build system | ||||||
| - C++ toolchain (for C++ modules) | ||||||
| - Rust toolchain (for Rust modules) | ||||||
| - Go toolchain with cgo support (for Go modules) | ||||||
|
|
||||||
| ### Build Process | ||||||
| 1. Match your build environment to the target Envoy version | ||||||
| 2. Implement the required extension interfaces | ||||||
| 3. Build as a shared library | ||||||
| 4. Configure Envoy to load the module at runtime | ||||||
|
|
||||||
| ## Module Structure | ||||||
|
|
||||||
| ``` | ||||||
| dynamic/ | ||||||
| ├── filters/ # HTTP/Network filters | ||||||
| ├── access_loggers/ # Access logging extensions | ||||||
| ├── tracers/ # Distributed tracing implementations | ||||||
| └── stats_sinks/ # Statistics output extensions | ||||||
| ``` | ||||||
|
|
||||||
| ## Configuration | ||||||
|
|
||||||
| Dynamic modules are configured in Envoy using the `envoy.extensions.common.dynamic_modules` extension: | ||||||
|
|
||||||
| ```yaml | ||||||
| static_resources: | ||||||
| listeners: | ||||||
| - filter_chains: | ||||||
| - filters: | ||||||
| - name: envoy.filters.http.dynamic_module | ||||||
| typed_config: | ||||||
| "@type": type.googleapis.com/envoy.extensions.filters.http.dynamic_module.v3.DynamicModule | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Is this the right type, from https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/dynamic_modules/v3/dynamic_modules.proto#extensions-filters-http-dynamic-modules-v3-dynamicmodulefilter ? |
||||||
| library_path: "/path/to/module.so" | ||||||
| library_id: "my_module" | ||||||
| ``` | ||||||
|
|
||||||
| ## Best Practices | ||||||
|
|
||||||
| 1. **Version Compatibility**: Ensure your module is built against the same Envoy version it will run with | ||||||
| 2. **Error Handling**: Implement robust error handling as crashes will take down the entire Envoy process | ||||||
| 3. **Resource Management**: Properly manage memory and other resources | ||||||
| 4. **Thread Safety**: Ensure your code is thread-safe as Envoy uses multiple worker threads | ||||||
|
|
||||||
| ## Examples | ||||||
|
|
||||||
| See the `examples/` subdirectory for sample implementations in each language. | ||||||
|
|
||||||
| ## Debugging | ||||||
|
|
||||||
| - Use `ldd` (Linux) or `otool -L` (macOS) to verify library dependencies | ||||||
| - Enable debug logging in Envoy to see module loading information | ||||||
| - Consider using AddressSanitizer or other debugging tools during development | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,145 @@ | ||
| # Go Extensions | ||
|
|
||
| Go extensions for Envoy are built using the official Envoy Go SDK, providing a Go-native development experience with additional features beyond what's available in other extension mechanisms. | ||
|
|
||
| ## Overview | ||
|
|
||
| The Envoy Go SDK enables writing Envoy extensions in pure Go, offering: | ||
| - Familiar Go development patterns | ||
| - Access to Go's standard library and ecosystem | ||
| - Simplified memory management with garbage collection | ||
| - Built-in concurrency primitives | ||
|
|
||
| ## Features | ||
|
|
||
| ### Unique Capabilities | ||
| - Direct manipulation of headers and body | ||
| - Async processing with goroutines | ||
| - Access to external services during request processing | ||
| - Dynamic configuration updates | ||
| - Native Go testing frameworks | ||
|
|
||
| ### Supported Extension Points | ||
| - HTTP filters | ||
| - Network filters | ||
| - Access loggers | ||
| - Custom protocols | ||
|
|
||
| ## Getting Started | ||
|
|
||
| ### Prerequisites | ||
| - Go 1.19 or later | ||
| - Envoy built with Go support enabled | ||
| - Basic understanding of Envoy's filter chain | ||
|
|
||
| ### Installation | ||
|
|
||
| ```bash | ||
| go get github.com/envoyproxy/go-control-plane | ||
| ``` | ||
|
|
||
| ## Project Structure | ||
|
|
||
| ``` | ||
| go/ | ||
| ├── http/ # HTTP filter implementations | ||
| │ ├── auth/ # Authentication filters | ||
| │ ├── ratelimit/ # Rate limiting filters | ||
| │ └── transform/ # Request/response transformation | ||
| ├── network/ # Network filter implementations | ||
| └── common/ # Shared utilities and helpers | ||
| ``` | ||
|
|
||
| ## Example HTTP Filter | ||
|
|
||
| ```go | ||
| package main | ||
|
|
||
| import ( | ||
| "github.com/envoyproxy/envoy/contrib/golang/common/go/api" | ||
| ) | ||
|
|
||
| type myFilter struct { | ||
| api.PassThroughStreamFilter | ||
| config *filterConfig | ||
| } | ||
|
|
||
| func (f *myFilter) DecodeHeaders(headers api.RequestHeaderMap, endStream bool) api.StatusType { | ||
| // Process request headers | ||
| headers.Set("x-custom-header", "processed-by-go") | ||
| return api.Continue | ||
| } | ||
|
|
||
| func (f *myFilter) EncodeHeaders(headers api.ResponseHeaderMap, endStream bool) api.StatusType { | ||
| // Process response headers | ||
| return api.Continue | ||
| } | ||
| ``` | ||
|
|
||
| ## Configuration | ||
|
|
||
| Go extensions are configured using the `envoy.filters.http.golang` filter: | ||
|
|
||
| ```yaml | ||
| http_filters: | ||
| - name: envoy.filters.http.golang | ||
| typed_config: | ||
| "@type": type.googleapis.com/envoy.extensions.filters.http.golang.v3alpha.Config | ||
| library_id: "my-go-filter" | ||
| library_path: "/path/to/filter.so" | ||
| plugin_config: | ||
| "@type": type.googleapis.com/xds.type.v3.TypedStruct | ||
| value: | ||
| key: "value" | ||
| ``` | ||
|
|
||
| ## Building | ||
|
|
||
| ### Build Command | ||
| ```bash | ||
| go build -buildmode=c-shared -o filter.so . | ||
| ``` | ||
|
|
||
| ### Build Flags | ||
| - `-buildmode=c-shared`: Required for Envoy integration | ||
| - `-trimpath`: Recommended for production builds | ||
| - `-ldflags="-s -w"`: Reduce binary size | ||
|
|
||
| ## Performance Considerations | ||
|
|
||
| 1. **Garbage Collection**: Be mindful of GC pauses in latency-sensitive paths | ||
| 2. **Goroutine Usage**: Avoid spawning excessive goroutines per request | ||
| 3. **Memory Allocation**: Reuse objects where possible | ||
| 4. **Blocking Operations**: Use async patterns for I/O operations | ||
|
|
||
| ## Testing | ||
|
|
||
| ```go | ||
| func TestFilter(t *testing.T) { | ||
| // Use the test harness provided by the SDK | ||
| harness := test.NewFilterTestHarness() | ||
| // Configure and run tests | ||
| } | ||
| ``` | ||
|
|
||
| ## Best Practices | ||
|
|
||
| 1. **Error Handling**: Always handle errors gracefully | ||
| 2. **Logging**: Use structured logging for debugging | ||
| 3. **Configuration**: Validate configuration during initialization | ||
| 4. **Resource Cleanup**: Implement proper cleanup in filter destructors | ||
| 5. **Panics**: Avoid panics - they will crash the Envoy process | ||
|
|
||
| ## Examples | ||
|
|
||
| See the `examples/` directory for complete working examples including: | ||
| - JWT authentication filter | ||
| - Request routing based on custom logic | ||
| - Response caching | ||
| - Custom metrics collection | ||
|
|
||
| ## Resources | ||
|
|
||
| - [Envoy Go SDK Documentation](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/golang_filter) | ||
| - [Go Control Plane](https://github.com/envoyproxy/go-control-plane) | ||
| - [Example Filters](https://github.com/envoyproxy/envoy/tree/main/examples/golang) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.