Releases: Azure/azure-event-hubs-spark
azure-eventhubs-databricks_2.11:3.4.0
A new package is available that is compatible with Databricks Runtime Version 3.4. Internals of the EventHubsSource were changed to cast strings and maps to the internal types used in Structured Streaming. In our case we cast strings to UTF8String
and maps to ArrayBasedMapData
. This should avoid any casting issues users experience in their streaming jobs.
<groupdId>com.microsoft.azure</groupId>
<artifactId>azure-eventhubs-databricks_2.11</artifactId>
<version>3.4.0</version>
<!--
3.4 to signify the compatible Databricks version.
We'll use the final digit for any patches to this release.
-->
azure-eventhubs-spark_2.11:2.1.6
This is the final planned release supporting Spark 2.1. Development efforts will now be focused on Spark 2.2 and future releases. In this release, we:
- Updated naming conventions to align with other artifacts in the EventHubs ecosystem.
- Added URI support (in addition namespace name) to allow use with non-public clouds running EventHubs.
azure-eventhubs-databricks_2.11:3.3.0
A new package is available that is compatible with Databricks Runtime Version 3.3. Internals of the EventHubSource were changed to accommodate the version of Spark used by Databricks. No other changes were made.
<groupdId>com.microsoft.azure</groupId>
<artifactId>azure-eventhubs-databricks_2.11</artifactId>
<version>3.3.0</version>
<!-- 3.3 to signify the compatible Databricks version.
We'll use the final digit for future releases involving this Databricks Runtime
-->
spark-streaming-eventhubs_2.11:2.1.5
This release includes the following changes to the spark-eventhubs library:
- GetPartitionRuntimeInfo API call is made every batch to ensure RDDs are note made with stale data
- Metadata progress files have been added for faster initialization
- Recever-based streams are no longer available
- Restful EventHubs Client has been removed
- Upgraded to version 0.15.1 of the Event Hubs Java Client (see https://github.com/Azure/azure-event-hubs-java/releases)
- Spark streaming jobs can now shutdown gracefully
- Additional traces have been added
- scalafmt has been adopted
- misc. code clean up
- repo improvements (re-written README, Contributor's Guide, Issue and PR templates, etc)
The corresponding Maven package can be found here:
http://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cspark-streaming-eventhubs_2.11%7C2.1.5%7C
This works with Spark 2.1.x and Spark 2.0.x. Open an issue if you have any questions/concerns/feedback of any kind! 👍
Spark Streaming Eventhubs Receiver v1.6.0
Supports:
- Azure Eventhubs Client 0.8.1 (https://mvnrepository.com/artifact/com.microsoft.azure/azure-eventhubs/0.8.1)
- Spark 1.6.x
- Java 8
- Scala 2.10
Spark Streaming Eventhubs Receiver v1.1.0
Merge pull request #7 from ppatierno/master Added the credits parameter handling as "eventhubs.receiver.credits"
Spark Streaming Eventhubs Receiver v1.0.0
Merge pull request #5 from wmeints/api-improvements Add implicit conversion methods