-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
31 additions
and
20 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,51 +1,62 @@ | ||
# pyrkakfa | ||
# pyrkakfa: Python Wrapper for rdfkafka (Rust) | ||
|
||
[![PyPI version](https://badge.fury.io/py/pyrkafka.svg)](https://pypi.org/project/pyrkafka/) | ||
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT) | ||
|
||
## Description | ||
## Python Library for Apache Kafka and RDF Data Processing | ||
|
||
`rdfkafka` is a Rust library for working with Kafka in Python. It provides a high-level interface for producing and consuming RDF data using the Kafka messaging system. | ||
`pyrkafka` is a Python wrapper around the `rdfkafka` library, which is written in Rust. It simplifies the integration with Apache Kafka for processing RDF (Resource Description Framework) data. `pyrkafka` provides a high-level Pythonic API for producing and consuming RDF messages, supporting various serialization formats such as JSON, Avro, and Protobuf. With `pyrkafka`, you can efficiently and scalably process RDF data in your Python applications, leveraging the performance and safety of Rust. | ||
|
||
## Features | ||
## Key Features of pyrkafka | ||
|
||
- Easy integration with Kafka for RDF data processing. | ||
- High-level API for producing and consuming RDF messages. | ||
- Support for various serialization formats (e.g., JSON, Avro, Protobuf). | ||
- Efficient and scalable processing of RDF data. | ||
- **Pythonic Interface to rdfkafka**: `pyrkafka` provides a Pythonic interface to the `rdfkafka` Rust library, making it easy to integrate Apache Kafka with Python applications. | ||
- **High-Level API for RDF Messages**: `pyrkafka` offers a high-level API for producing and consuming RDF messages in Python. | ||
- **Support for Various Serialization Formats**: `pyrkafka` supports various serialization formats, including JSON, Avro, and Protobuf. | ||
- **Efficient RDF Data Processing**: `pyrkafka` enables efficient and scalable processing of RDF data, leveraging the performance of Rust. | ||
|
||
## Installation | ||
|
||
You can install `rdfkafka` using pip: | ||
``` | ||
pip install rdfkafka | ||
You can install `pyrkakfa` using pip: | ||
```bash | ||
pip install pyrkakfa | ||
``` | ||
|
||
## Producer Example | ||
```python | ||
from rdfkafka import KafkaProducer | ||
# Getting Started with pyrkafka | ||
|
||
## Sending Messages to Kafka with pyrkafka | ||
Here's a simple example of how to send a message to a Kafka topic using `pyrkafka`: | ||
|
||
```python | ||
from pyrkakfa import KafkaProducer | ||
|
||
# Initialize a producer | ||
producer = KafkaProducer(bootstrap_servers='localhost:9092') | ||
|
||
# Specify the topic | ||
topic = 'my_topic' | ||
|
||
# Create a message | ||
message = 'Hello, Kafka!' | ||
producer.send(topic, message.encode()) | ||
|
||
# Send the message | ||
producer.send(topic, message.encode()) | ||
``` | ||
|
||
##### Note: producer gets closed when dropped | ||
|
||
## Consumer Example | ||
## Consuming Messages from Kafka with pyrkafka | ||
Here's how to consume messages from a Kafka topic using `pyrkafka`: | ||
```python | ||
from rdfkafka import KafkaConsumer | ||
from pyrkakfa import KafkaConsumer | ||
|
||
# Initialize a consumer | ||
consumer = KafkaConsumer(bootstrap_servers='localhost:9092') | ||
topic = 'my_topic' | ||
|
||
# Specify the topic | ||
topic = 'my_topic' | ||
|
||
# Subscribe to the topic | ||
consumer.subscribe(topic) | ||
|
||
# Listen for messages | ||
for message in consumer: | ||
print(message.value.decode()) | ||
``` |