add swift connector example #438
Conversation
fivetran-surabhisingh
added
the
hackathon
🧹 Python Code Quality Check📎 Download full report from workflow artifacts. 📌 Only Python files changed in this PR were checked. This comment is auto-updated with every commit. |
fivetran-sahilkhirwal
requested review from
Copilot,
fivetran-sahilkhirwal and
manjutapali
There was a problem hiding this comment.
Pull Request Overview
This PR adds a new Swift connector for fetching payment data via SWIFT API. The connector is a minimal implementation showing API integration setup.
- Adds a new Swift connector under
connectors/swift/ - Implements basic transaction fetching from a SWIFT API endpoint
- Uses a decorator-based connector pattern
| """ | ||
|
|
||
| import requests | ||
| from fivetran_connector_sdk import connector, config, state, records, log, schema |
There was a problem hiding this comment.
BLOCKER: Incorrect SDK imports. The Fivetran Connector SDK uses a different import pattern. Use these exact imports instead: from fivetran_connector_sdk import Connector, from fivetran_connector_sdk import Logging as log, and from fivetran_connector_sdk import Operations as op. The decorator-based pattern with @connector, config, state, records, and schema imports does not exist in the SDK.
| CONFIG = config.Config( | ||
| base_url=config.StringField(description="SWIFT API base URL"), | ||
| api_key=config.SecretField(description="SWIFT API key") | ||
| ) |
There was a problem hiding this comment.
BLOCKER: This configuration pattern is not supported by the Fivetran Connector SDK. Configuration should be defined in a configuration.json file which the SDK automatically validates. Remove this CONFIG object and create a configuration.json file instead with the required fields.
| SCHEMA = schema.Schema( | ||
| name="swift_transactions", | ||
| columns={ | ||
| "transaction_id": schema.StringColumn(), | ||
| "amount": schema.StringColumn(), | ||
| "currency": schema.StringColumn(), | ||
| "timestamp": schema.StringColumn(), | ||
| } | ||
| ) |
There was a problem hiding this comment.
BLOCKER: This schema definition pattern is not supported. The SDK requires a schema() function that returns a list of table dictionaries. Replace this with a proper schema(configuration: dict) function following the template format.
| @connector( | ||
| name="SwiftConnector", | ||
| version="0.1.0", | ||
| config=CONFIG, | ||
| schema=SCHEMA, | ||
| ) | ||
| def run_connector(ctx: state.Context): |
There was a problem hiding this comment.
BLOCKER: The decorator-based @connector pattern is not part of the Fivetran Connector SDK. The SDK requires an update(configuration: dict, state: dict) function and a Connector object initialized as connector = Connector(update=update, schema=schema). Remove the decorator and implement the required functions.
| Fetches payment data via SWIFT API (mock endpoint for base setup). | ||
| """ | ||
|
|
||
| import requests |
There was a problem hiding this comment.
Missing import comment. Add a comment explaining the purpose of this import, e.g., # For making HTTP API requests (provided by SDK runtime).
| import requests | |
| import requests # For making HTTP API requests (provided by SDK runtime) |
| for tx in response.json().get("data", []): | ||
| records.write("swift_transactions", tx) | ||
|
|
||
| return ctx.update_state({"last_sync": "now"}) |
There was a problem hiding this comment.
BLOCKER: Incorrect state management pattern. The SDK uses op.checkpoint(state) to save state, and the update function should not return state. Replace with proper checkpointing using op.checkpoint() and include the required checkpoint comment.
| """ | ||
| Swift connector for Fivetran Connector SDK. | ||
| Fetches payment data via SWIFT API (mock endpoint for base setup). | ||
| """ |
There was a problem hiding this comment.
Missing required structure: The connector is missing the mandatory main block for debugging. Add the following at the end: if __name__ == '__main__': block with connector.debug(configuration=configuration) as shown in the template.
| for tx in response.json().get("data", []): | ||
| records.write("swift_transactions", tx) | ||
|
|
||
| return ctx.update_state({"last_sync": "now"}) |
There was a problem hiding this comment.
Using string literal 'now' as a timestamp value is incorrect. Use a proper ISO 8601 timestamp format like datetime.datetime.now().isoformat() or a Unix timestamp.
| response = requests.get(f"{ctx.config.base_url}/transactions", headers=headers) | ||
| response.raise_for_status() |
There was a problem hiding this comment.
Missing specific exception handling. Generic exceptions from raise_for_status() should be caught and handled specifically (e.g., distinguish between 4xx client errors that shouldn't be retried vs 5xx server errors that should). Implement proper error handling with specific exception types.
| """ | ||
|
|
||
| import requests | ||
| from fivetran_connector_sdk import connector, config, state, records, log, schema |
There was a problem hiding this comment.
Import of 'log' is not used.
| from fivetran_connector_sdk import connector, config, state, records, log, schema | |
| from fivetran_connector_sdk import connector, config, state, records, schema |
1 participant
Jira ticket
Closes
<ADD TICKET LINK HERE, EACH PR MUST BE LINKED TO A JIRA TICKET>Description of Change
<MENTION A SHORT DESCRIPTION OF YOUR CHANGES HERE>Testing
<MENTION ABOUT YOUR TESTING DETAILS HERE, ATTACH SCREENSHOTS IF NEEDED (WITHOUT PII)>Checklist
Some tips and links to help validate your PR:
fivetran debugcommand.