|
| 1 | +# EMQX Enterprise Version 6 |
| 2 | + |
| 3 | +## 6.0.0 |
| 4 | + |
| 5 | +### Highlights |
| 6 | + |
| 7 | +- **Major Version Release**: EMQX Enterprise 6.0.0 is the first release of the EMQX Enterprise version 6 series, bringing significant architectural improvements and new capabilities. |
| 8 | + |
| 9 | +- **Enhanced AWS Integration**: Support for Instance Metadata Service v2 APIs from EC2 instances when using S3 or S3Tables data integration. This enables seamless access to S3 buckets without manual AWS credential configuration, leveraging IAM roles for better security. |
| 10 | + |
| 11 | +- **Elixir Support**: All packages now ship with Elixir support through the Mix build system, opening EMQX to the Elixir community and enabling better tooling with IEx console. |
| 12 | + |
| 13 | +- **New Data Integrations**: |
| 14 | + - BigQuery Connector and Action for appending data to Google BigQuery |
| 15 | + - Snowpipe Streaming upload mode for Snowflake Action (preview feature) |
| 16 | + - Parquet format support for S3Tables Action |
| 17 | + |
| 18 | +- **Durable Storage Optimizations**: Significant RAM usage and storage efficiency improvements with new RocksDB configuration options and ASN1 serialization schema as default. |
| 19 | + |
| 20 | +- **Enhanced LDAP Support**: LDAP authorization now supports extended ACL rules in JSON format, and LDAP authentication can fetch ACL rules directly from LDAP with client-side caching. |
| 21 | + |
| 22 | +- **Improved Tracing**: Configurable limits for maximum traces (`trace.max_traces`) and trace file sizes (`trace.max_file_size`), with optimized implementation to prevent atom leaks. |
| 23 | + |
| 24 | +- **Cluster Management**: New `cluster.description` configuration option allows users to set and display custom cluster descriptions in the EMQX Dashboard. |
| 25 | + |
| 26 | +### Enhancements |
| 27 | + |
| 28 | +#### Rule Engine |
| 29 | + |
| 30 | +- [#15631](https://github.com/emqx/emqx/pull/15631) A new API endpoint is added to list all models available for an AI provider. |
| 31 | + |
| 32 | +- [#15467](https://github.com/emqx/emqx/pull/15467) Expose transport options for AI Completion Providers. |
| 33 | + These options allow configuring connection timeouts and maximum connections to an AI Completion Provider. |
| 34 | + |
| 35 | +#### Data Integration |
| 36 | + |
| 37 | +- [#15635](https://github.com/emqx/emqx/pull/15635) Added new `key` and `tag` template fields for the RocketMQ Action which sets the message's key and tag, respectively. Also, added a new `key_dispatch` value for the `strategy` field. |
| 38 | + |
| 39 | +- [#15621](https://github.com/emqx/emqx/pull/15621) Now, `access_key_id` and `secret_access_key` are optional fields for the S3Tables Connector. If omitted, they'll be obtained from the Instance Metadata Service v2 APIs from the EC2 instance where EMQX is deployed. |
| 40 | + |
| 41 | +- [#15542](https://github.com/emqx/emqx/pull/15542) Upgraded our `erlcoud` library to `3.8.3.0`. This allows one to setup a S3 Connector without specifying Access Key Id and Secret Access Key, so long as the EC2 instance EMQX is running in has the correct IAM permissions to read/write to the configured bucket(s). |
| 42 | + |
| 43 | +- [#15418](https://github.com/emqx/emqx/pull/15418) Added new Connector and Action that appends data to BigQuery. |
| 44 | + |
| 45 | +- [#15401](https://github.com/emqx/emqx/pull/15401) Added support for Snowpipe Streaming upload mode for Snowflake Action. Note: this is currently a preview feature by Snowflake, and support for it is available to all accounts on AWS only. |
| 46 | + |
| 47 | +- [#15387](https://github.com/emqx/emqx/pull/15387) Improved Kinesis Producer Connector and Action health checks to mitigate the occurrence of rate limiting when calling `ListStreams` and `DescribeStream` APIs. Now, we limit the the calls per Connector to such APIs to 5/s and 10/s, respectively. If a Connector or Action cannot call their health check API before timing out, they will simply maintain their current status. If they receive a throttling response (e.g.: `LimitExceededException`), they will also maintain their current status. |
| 48 | + |
| 49 | + Introduced a new `resource_opts.health_check_interval_jitter` configuration to add an uniform random delay to `resource_opts.health_check_interval`, so that multiple Actions under the same Connector will seldom run their health checks simultaneously. |
| 50 | + |
| 51 | +- [#15371](https://github.com/emqx/emqx/pull/15371) Added `tags` fields to the return of `GET /actions_summary` and to the fallback actions returned in `GET /actions/:id`. |
| 52 | + |
| 53 | +- [#15360](https://github.com/emqx/emqx/pull/15360) Added support for writing data files in Parquet format for S3Tables Action. |
| 54 | + |
| 55 | +- [#15176](https://github.com/emqx/emqx/pull/15176) Upgrade the GreptimeDB connector client and support an optional new parameter `ttl` to set the default time-to-live for automatically created tables. |
| 56 | + |
| 57 | +- [#15585](https://github.com/emqx/emqx/pull/15585) Updated our `brod` client to version 4.4.4. This expands the supported Kafka API ranges, in particular due to the `JoinGroups` API `v0`-`v1` being deprecated. |
| 58 | + |
| 59 | +- [#15628](https://github.com/emqx/emqx/pull/15628) Remove HStreamDB data integration. |
| 60 | + |
| 61 | +#### Access Control |
| 62 | + |
| 63 | +- [#15349](https://github.com/emqx/emqx/pull/15349) Optimize external resource management for authentication and authorization. Previously, EMQX could remain connected to a resource configured for a disabled authentication or authorization provider. |
| 64 | + |
| 65 | +- [#15294](https://github.com/emqx/emqx/pull/15294) Enhance LDAP authentication and authorization. |
| 66 | + LDAP authorization now supports extended ACL rules in JSON format. |
| 67 | + LDAP authenticaton now can fetch ACL rules from LDAP. These rules are cached in the client's metadata, so authorization is performed without additional LDAP queries. |
| 68 | + |
| 69 | +#### Smart Data Hub |
| 70 | + |
| 71 | +- [#15525](https://github.com/emqx/emqx/pull/15525) Now, when attempting to remove an Internal Schema which is being referenced by either a Schema Validation or a Message Transformation, the removal will be denied. |
| 72 | + |
| 73 | +#### Durable Storage |
| 74 | + |
| 75 | +- [#15463](https://github.com/emqx/emqx/pull/15463) Durable storage RAM usage and storage efficiency optimizations. |
| 76 | + |
| 77 | + 1. Added the following configuration parameters for the durable storage: |
| 78 | + |
| 79 | + - `durable_storage.messages.rocksdb.write_buffer_size`: RocksDB memtable size per shard. |
| 80 | + - `durable_storage.messages.rocksdb.cache_size`: RocksDB block size per shard. |
| 81 | + - `durable_storage.messages.rocksdb.max_open_files`: Limits the number of file descriptors used by RocksDB per shard. |
| 82 | + - `durable_storage.messages.layout.wildcard_thresholds`: Allows to tune wildcard thresholds for the `wildcard_optimized_v2` storage layout |
| 83 | + |
| 84 | + 2. The default `serialization_schema` for the messages has been changed to `asn1`. |
| 85 | + |
| 86 | +#### Observability |
| 87 | + |
| 88 | +- [#15594](https://github.com/emqx/emqx/pull/15594) Exposed maximum number of traces allowed to exist in the cluster simultaneously as a configuration option `trace.max_traces`. This limit does not apply to node-local traces managed through `emqx ctl trace`. |
| 89 | + |
| 90 | + Optimized tracing implementation to eliminate potential atom leaks per created trace. |
| 91 | + |
| 92 | +- [#15556](https://github.com/emqx/emqx/pull/15556) Exposed maximum trace file size limit per each individual trace as a configuraion option `trace.max_file_size`. |
| 93 | + |
| 94 | +- [#15364](https://github.com/emqx/emqx/pull/15364) Add HTTP header configuration items to the OpenTelemetry integration to adapt to collectors with HTTP authentication. |
| 95 | + |
| 96 | +#### Deployment |
| 97 | + |
| 98 | +- [#15484](https://github.com/emqx/emqx/pull/15484) Changed our build system so that all packages are build with [Elixir](https://elixir-lang.org/)'s [Mix](https://hexdocs.pm/elixir/introduction-to-mix.html), so that all now ship with Elixir support. This opens up EMQX to Elixir community and allows us to use Elixir dependencies when required, besides enabling use of [IEx](https://hexdocs.pm/iex/IEx.html) as a better EMQX console. |
| 99 | + |
| 100 | +- [#15399](https://github.com/emqx/emqx/pull/15399) Now, `node_dump` will export the current system configuration in HOCON format with redacted secrets. |
| 101 | + |
| 102 | +#### Clustering |
| 103 | + |
| 104 | +- [#15600](https://github.com/emqx/emqx/pull/15600) Added a new `cluster.description` configuration that allows one to add a description to the EMQX cluster. This description can be changed via `PUT /cluster`, and viewed in the `GET /cluster` response. |
| 105 | + |
| 106 | +#### Performance |
| 107 | + |
| 108 | +- [#15536](https://github.com/emqx/emqx/pull/15536) Disable the `node.global_gc_interval` configuration by default. |
| 109 | + |
| 110 | +- [#15539](https://github.com/emqx/emqx/pull/15539) Optimize Erlang VM parameters. |
| 111 | + |
| 112 | + - Increase the buffer sizes for distributed channels to 32MB to avoid `busy_dist_port` alarms during intensive Mnesia operations: `+zdbbl 32768` |
| 113 | + - Disable scheduler busy-waiting to reduce CPU usage observed by the operating system: `+sbwt none +sbwtdcpu none +sbwtdio none` |
| 114 | + - Set scheduler binding type to `db` to reduce message latency: `+stbt db` |
| 115 | + |
| 116 | +- [#15451](https://github.com/emqx/emqx/pull/15451) Introduce experimental `socket` backend for TCP listeners, designed to improve message processing latency and reduce compute resource usage. This can be enabled via the new `tcp_backend` listener option. |
| 117 | + |
| 118 | +### Bug Fixes |
| 119 | + |
| 120 | +#### Data Integration |
| 121 | + |
| 122 | +- [#15647](https://github.com/emqx/emqx/pull/15647) Previously, if the user of a MongoDB Connector did not have sufficient privileges to perform `find` queries in a `foo` collection, it would be considered disconnected. This has been fixed. |
| 123 | + |
| 124 | +- [#15603](https://github.com/emqx/emqx/pull/15603) Fixed an issue with the MQTT bridge when a stale connection was displayed as `Connected' and the connection was not re-established. |
| 125 | + |
| 126 | + |
| 127 | +- [#15522](https://github.com/emqx/emqx/pull/15522) Fixed an issue where Snowflake Connector would fail to start correctly is `username` was not provided. |
| 128 | + |
| 129 | +- [#15476](https://github.com/emqx/emqx/pull/15476) When most of the Actions that use aggregated mode (Azure Blob Storage, Snowflake, S3Tables) had a delivery that failed, the following log would be printed: |
| 130 | + |
| 131 | + ``` |
| 132 | + "emqx_connector_aggreg_delivery:format_status/1 crashed" |
| 133 | + ``` |
| 134 | + |
| 135 | + This has been fixed, and more information about the delivery process will now be logged. |
| 136 | + |
| 137 | +- [#15394](https://github.com/emqx/emqx/pull/15394) Fixed a very rare race condition in which Action metrics could end up in an inconsistent state. |
| 138 | + |
| 139 | +- [#15383](https://github.com/emqx/emqx/pull/15383) Fix a potential resource leak in MQTT bridge when the bridge fails to start. Previously, the topic index table was not properly cleaned up when the bridge failed to start. |
| 140 | + |
| 141 | +#### Rule Engine |
| 142 | + |
| 143 | +- [#15569](https://github.com/emqx/emqx/pull/15569) Fixed an issue where a Republish Rule Action could fail if the template for `direct_dispatch` was an empty string or resolve to a non-boolean value. Now, if such situations occur, the default value, `false`, will be used. |
| 144 | + |
| 145 | +#### Core MQTT Functionalities |
| 146 | + |
| 147 | +- [#15518](https://github.com/emqx/emqx/pull/15518) Resolve a race condition that may lead to accumulating inconsistencies in the routing table and shared subscriptions state in the cluster when a large number of shared subscribers disconnect simultaneously. |
| 148 | + |
| 149 | +- [#15416](https://github.com/emqx/emqx/pull/15416) Fixed occasional warning-level log events and crashes during session expiration of WebSocket connections, introduced by recent WebSocket performance improvements. These had no impact on broker capacity, but produced log entries like the following: |
| 150 | + * `error: {function_clause,[{gen_tcp,send,[closed,[]],[{file,“gen_tcp.erl”},{line,966}]},{cowboy_websocket_linger,commands,3,[{file,“cowboy_websocket_linger.erl”},{line,665}]},...` |
| 151 | + * `message: {tcp,#Port<0.364>,<<136,130,...>>}, msg: emqx_session_mem_unknown_message` |
| 152 | + |
| 153 | +- [#15396](https://github.com/emqx/emqx/pull/15396) Removed redundant cleanup operations for shared subscriptions of disconnected clients, which were prone to crashes under high disconnect volume, resulting in potential inconsistencies in the global broker state. |
| 154 | + |
| 155 | +- [#15361](https://github.com/emqx/emqx/pull/15361) Fixed a function clause error when parsing a malformed `User-Property` pair where the pair length is wrong (too short). |
| 156 | + |
| 157 | +#### Access Control |
| 158 | + |
| 159 | +- [#15489](https://github.com/emqx/emqx/pull/15489) Fix OIDC issuer scheme validation. |
| 160 | + |
| 161 | + The previously unsupported issuer URLs listed below are now supported. |
| 162 | + |
| 163 | + - `https://xxxxxxxx:8443/webman/sso/.well-known/openid-configuration` |
| 164 | + - `hostname` |
| 165 | + |
| 166 | +#### Gateway |
| 167 | + |
| 168 | +- [#15342](https://github.com/emqx/emqx/pull/15342) Fixed NATS gateway crash when clientinfo override templates contain undefined packet fields by returning empty binary instead of undefined atom. |
| 169 | + |
| 170 | +#### Security |
| 171 | + |
| 172 | +- [#15581](https://github.com/emqx/emqx/pull/15581) Upgrade OTP version from 26.2.5.2 to 26.2.5.14 |
| 173 | + |
| 174 | + This upgrade includes two TLS-related fixes relevant to EMQX: |
| 175 | + |
| 176 | + - Fixed a crash in TLS connections caused by a race condition during certificate renewal. |
| 177 | + - Added support for RSA certificates signed with PSS parameters. Previously TLS handshake may fail with `invalid_signature`. |
| 178 | + |
| 179 | +#### Deployment |
| 180 | + |
| 181 | +- [#15580](https://github.com/emqx/emqx/pull/15580) Add emqxLicenseSecretRef variable to EMQX Enterprise helm chart, allowing users to specify a Kubernetes secret containing the EMQX license key. This fixes the issue with defunct emqxLicenseSecretName variable. |
| 182 | + |
| 183 | +- [#15553](https://github.com/emqx/emqx/pull/15553) Fixes an issue with helm chart when all nodes except one will be crashing if the chart is deployed with default values. |
| 184 | + |
| 185 | +#### HTTP Server |
| 186 | + |
| 187 | +- [#15547](https://github.com/emqx/emqx/pull/15547) Fixed error when an HTTP request with a large body is sent. |
0 commit comments