Support for Apache-Ignite

[matsvitt]

Hello everyone,

if anyone is interested in accessing Apache Ignite datasets… this is now possible (however, early alpha - you’ve been warned).
Please include the latest version (which is 0.5.6 as of today) of the sqlalchemy-ignitex project
In your superset setup. It is installable via pipy via:

pip install sqlalchemy-ignitex

The code for this package is also on github (https://github.com/matsvitt/sqlalchemy-ignite/). It is a fork based on the original sqlalchemy-ignite project (kudos to predictiveworks). I just added a few things in order to make it work again.

Please let me know, if it is useful. I would be glad to support.

[fivetran-fedorabramov]

Hey! Trying to connect my Ignite to superset.
Database itself works fine.
Added sqlalchemy-ignitex to docker/requirements-local.txt in the superset dist.
Looks like superset successfully installs your package but when I'm trying to connect my database to superset, I have an error.
Could you help me please? What I'm doing wrong?

[matsvitt]

Hey, I thought for ages, I am the only one interested in connecting Superset to Ignite :) I need to build a short example (please give me some time)… However: My Connect6ion looks like this:  So you might adjust the protocol to igniteworks Best Matthias

…

On 24. Jan 2025, at 14:46, Fedor Abramov ***@***.***> wrote: Hey! Trying to connect my Ignite to superset. Database itself working fine. Added sqlalchemy-ignitex to docker/requirements-local.txt in the superset. Looks like superset successfully installs your package but when I'm trying to connect my database to superset, I have an error. Could you help me please? What I'm doing wrong? Screenshot.2025-01-24.at.16.41.17.png (view on web) <https://github.com/user-attachments/assets/c89d739e-8621-46fe-b67e-7e1fdf4c9230> Screenshot.2025-01-24.at.16.43.42.png (view on web) <https://github.com/user-attachments/assets/355cb5d7-1a46-40bf-b8da-103f7104fca5> Screenshot.2025-01-24.at.16.44.20.png (view on web) <https://github.com/user-attachments/assets/cdf931ba-d1ae-45e0-a6c2-5af2776e94d7> — Reply to this email directly, view it on GitHub <#29340 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGVDPCD3KF256KHJYNNYJT2MI72RAVCNFSM6AAAAABVZW7LMCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCOJUGM2DONA>. You are receiving this because you authored the thread.

[fivetran-fedorabramov]

WOW! Changed protocol to igniteworks and it realy works! Thank you so much!

[matsvitt]

Glad to hear that :)

…

On 24. Jan 2025, at 15:47, Fedor Abramov ***@***.***> wrote: WOW! Changed protocol to igniteworks and it realy works! Thank you so much! — Reply to this email directly, view it on GitHub <#29340 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGVDPFYFO37QJUUFQIAXKD2MJHBRAVCNFSM6AAAAABVZW7LMCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCOJUGQYTKOI>. You are receiving this because you authored the thread.

[lomumve]

hello, I add requirements-local.txt under docker folder and add sqlalchemy-ignitex==0.5.6 in this file
after this. I run docker compose down -v and then docker compose -f docker-compose-non-dev.yml up
but when I try to connect with igniteworks://localhost:10800, it shows error
2025-02-14 02:50:43,058:ERROR:pyignite.connection:Failed to perform handshake, connection to node(address=localhost, port=10800) with protocol context ProtocolContext(version=(1, 7, 0), features=4) failed: [Errno 111] Connection refused
superset_app | Traceback (most recent call last):
superset_app | File "/app/.venv/lib/python3.11/site-packages/pyignite/connection/connection.py", line 238, in connect
superset_app | result = self._connect_version()
superset_app | ^^^^^^^^^^^^^^^^^^^^^^^
superset_app | File "/app/.venv/lib/python3.11/site-packages/pyignite/connection/connection.py", line 268, in _connect_version
superset_app | self._socket.connect((self.host, self.port))
and

Do you know if there is any possible reason for this. Thanks a lot

[matsvitt]

Hi there, I assume you have an Apache Ignite instance running on port 10800 on localhost? Best Matthias

…

On 14. Feb 2025, at 03:54, 憨憨 ***@***.***> wrote: hello, I add requirements-local.txt under docker folder and add sqlalchemy-ignitex==0.5.6 in this file after this. I run docker compose down -v and then docker compose -f docker-compose-non-dev.yml up but when I try to connect with igniteworks://localhost:10800, it shows error 2025-02-14 02:50:43,058:ERROR:pyignite.connection:Failed to perform handshake, connection to node(address=localhost, port=10800) with protocol context ProtocolContext(version=(1, 7, 0), features=4) failed: [Errno 111] Connection refused superset_app | Traceback (most recent call last): superset_app | File "/app/.venv/lib/python3.11/site-packages/pyignite/connection/connection.py", line 238, in connect superset_app | result = self._connect_version() superset_app | ^^^^^^^^^^^^^^^^^^^^^^^ superset_app | File "/app/.venv/lib/python3.11/site-packages/pyignite/connection/connection.py", line 268, in _connect_version superset_app | self._socket.connect((self.host, self.port)) and image.png (view on web) <https://github.com/user-attachments/assets/5628b2a1-497c-4d38-a5c9-9e104a362fec> Do you know if there is any possible reason for this. Thanks a lot — Reply to this email directly, view it on GitHub <#29340 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGVDPGCZ6OQQNMQ5B4LVCD2PVLHNAVCNFSM6AAAAABVZW7LMCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMJZGUZDKNI>. You are receiving this because you authored the thread.

[lomumve]

Hi Matsvitt,
Yes, My workflows are:

1. Download Supperset code from github
2. Add requirements-local.txt under docker folder and add sqlalchemy-ignitex==0.5.6 in this file
3. Run Docker Compose down -v
4. Run Docker Compose -f docker-compose-non-dev.yml up
5. Forward remote ignite db to my local env with command kubectl --kubeconfig dev.yaml -n namespace -dev port-forward service/grid-web 10800:10800
6. And then try to connect in Supperset under Other type with connection as igniteworks://localhost:10800

I tested with below python code could connect success:

from sqlalchemy import create_engine

engine = create_engine("igniteworks://localhost:10800")
connection = engine.connect()

[matsvitt]

are you able to connect to ignite using a python script running locally?

…

On 16. Feb 2025, at 13:16, 憨憨 ***@***.***> wrote: Hi Matsvitt, Yes, My workflows are: Download Supperset code from github Add requirements-local.txt under docker folder and add sqlalchemy-ignitex==0.5.6 in this file Run Docker Compose down -v Run Docker Compose -f docker-compose-non-dev.yml up Forward remote ignite db to my local env with command kubectl --kubeconfig dev.yaml -n namespace -dev port-forward service/grid-web 10800:10800 And then try to connect in Supperset under Other type with connection as igniteworks://localhost:10800 I tested with below python code could connect success: from sqlalchemy import create_engine engine = create_engine("igniteworks://localhost:10800") connection = engine.connect() — Reply to this email directly, view it on GitHub <#29340 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGVDPFPPXTPJIDB7VKU3732QB6TFAVCNFSM6AAAAABVZW7LMCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMRRGQ4TCNI>. You are receiving this because you authored the thread.