Support for Starburst Trino

[shivaellur1980]

Dear experts, do we have any support for starburst Trino integration for great expectations. In absence of great expectation integration, we are using Trino in-build function but it has numerous limitations associated with it. It would be great if you can suggest way forward in it.

[rubnsbarbosa]

@shivaellur1980 Great Expectations relies on the library trino.sqlalchemy.dialect to connect to your data, but you will need to install the package trino containing this library.

• customize your trino datasource configuration with your credentials

host = "YOUR_HOST"
port = "YOUR_PORT"
username = "YOUR_USERNAME"
password = "YOUR_PASSWORD"

# Trino has a concept of catalog which maps to database
database = "YOUR_TRINO_CATALOG"
schema_name = "YOUR_SCHEMA"

# A table that you would like to add initially as a Data Asset
table_name = "YOUR_TABLE_NAME"

• great expectations datasource configuration

trino_yaml = f"""
name: {datasource_name}
class_name: Datasource
execution_engine:
  class_name: SqlAlchemyExecutionEngine
  credentials:
    host: {host}
    port: '{port}'
    username: {username}
    password: {password}
    database: {database}
    drivername: trino
data_connectors:
  default_runtime_data_connector_name:
    class_name: RuntimeDataConnector
    batch_identifiers:
      - default_identifier_name
  default_inferred_data_connector_name:
    class_name: InferredAssetSqlDataConnector
    include_schema_name: True
    introspection_directives:
      schema_name: {schema_name}
  default_configured_data_connector_name:
    class_name: ConfiguredAssetSqlDataConnector
    assets:
      {table_name}:
        class_name: Asset
        schema_name: {schema_name}
"""

• save the datasource configuration into gx/great_expectations.yml file. Note that overwrite_existing defaults to False, but you may change it to True because you wish to overwrite, after overwrite it and see the trino datasource configuration in your gx/great_expectations.yml file you may change it to False again.

sanitize_yaml_and_save_datasource(context, trino_yaml, overwrite_existing=False)