Skip to content

add gusto connector example #442

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

add gusto connector example #442

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions connectors/gusto/connector.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
"""
Gusto connector for Fivetran Connector SDK.
Fetches HR and payroll data from Gusto API.
"""
Comment on lines +1 to +4
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Module docstring should include a link to technical reference documentation. Expected format: Line 1 should be a one-line description, followed by reference links. Example from template: \"\"\"ADD ONE LINE DESCRIPTION OF YOUR CONNECTOR HERE.\\nSee the Technical Reference documentation (https://fivetran.com/docs/connectors/connector-sdk/technical-reference#update)\\nand the Best Practices documentation (https://fivetran.com/docs/connectors/connector-sdk/best-practices) for details\"\"\"

Copilot generated this review using guidance from repository custom instructions.

import requests
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing comment explaining the import. Every import must have an inline comment. Note that requests is provided by the SDK runtime and should not be included in requirements.txt. Add comment like: # For making HTTP API requests to Gusto (provided by SDK runtime)

Copilot generated this review using guidance from repository custom instructions.
from fivetran_connector_sdk import connector, config, state, records, log, schema
Comment on lines +6 to +7
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKER: Incorrect SDK imports. The Fivetran Connector SDK does not export connector, config, state, records, or schema modules in this way. The correct imports should be: from fivetran_connector_sdk import Connector, from fivetran_connector_sdk import Logging as log, and from fivetran_connector_sdk import Operations as op. See template_example_connector/connector.py for the correct pattern.

Copilot generated this review using guidance from repository custom instructions.
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import of 'log' is not used.

Suggested change
from fivetran_connector_sdk import connector, config, state, records, log, schema
from fivetran_connector_sdk import connector, config, state, records, schema

Copilot uses AI. Check for mistakes.

CONFIG = config.Config(
Comment on lines +1 to +9
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing required structure. The connector is missing: (1) import json for configuration file reading, (2) schema(configuration: dict) function with required docstring, (3) update(configuration: dict, state: dict) function with required docstring, (4) Connector initialization: connector = Connector(update=update, schema=schema), (5) Main block for local debugging with if __name__ == '__main__': connector.debug(). See template_example_connector/connector.py lines 7-166 for the complete required structure.

Copilot generated this review using guidance from repository custom instructions.
base_url=config.StringField(default="https://api.gusto.com"),
access_token=config.SecretField(description="OAuth2 access token")
)

SCHEMA = schema.Schema(
name="gusto_employees",
columns={
"id": schema.StringColumn(),
"first_name": schema.StringColumn(),
"last_name": schema.StringColumn(),
"email": schema.StringColumn(),
"status": schema.StringColumn(),
}
)

@connector(
name="GustoConnector",
version="0.1.0",
config=CONFIG,
schema=SCHEMA,
)
def run_connector(ctx: state.Context):
Comment on lines +9 to +31
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKER: The entire structure is incompatible with Fivetran Connector SDK. The SDK does not use decorators, Config classes, or Schema classes. Instead, it requires: (1) A schema(configuration: dict) function that returns a list of table dictionaries, (2) An update(configuration: dict, state: dict) function for data fetching, (3) A Connector object initialized with these functions: connector = Connector(update=update, schema=schema). Configuration should be provided via configuration.json file. See template_example_connector/connector.py for the correct implementation pattern.

Copilot generated this review using guidance from repository custom instructions.
headers = {"Authorization": f"Bearer {ctx.config.access_token}"}
response = requests.get(f"{ctx.config.base_url}/v1/employees", headers=headers)
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing retry logic for API request. Network calls must implement retry logic with exponential backoff for transient failures (network errors, rate limits, 5xx errors). Limit to 3-5 retry attempts and log each retry. Example: for attempt in range(__MAX_RETRIES): try: response = requests.get(...); break except (requests.Timeout, requests.ConnectionError) as e: if attempt == __MAX_RETRIES - 1: raise; sleep_time = min(60, 2 ** attempt); time.sleep(sleep_time)

Copilot generated this review using guidance from repository custom instructions.
response.raise_for_status()

for emp in response.json():
Comment on lines +33 to +36
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential memory issue: response.json() loads the entire response into memory. For large employee datasets, this could cause memory overflow. If the Gusto API supports pagination, implement it to process data in chunks. If pagination is not available and datasets are guaranteed to be small, add a comment explaining why full materialization is acceptable.

Copilot generated this review using guidance from repository custom instructions.
records.write("gusto_employees", emp)
Comment on lines +33 to +37
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKER: Incorrect operations usage. The SDK does not have a records.write() function. Instead, use op.upsert(table='gusto_employees', data=emp) after importing from fivetran_connector_sdk import Operations as op. Each upsert call must be preceded by a comment explaining its purpose.

Copilot generated this review using guidance from repository custom instructions.

return ctx.update_state({"last_sync": "now"})
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKER: Incorrect state management. The SDK does not use ctx.update_state(). Instead, state should be updated via op.checkpoint(state) where state is a dictionary. The update function should not return anything. State checkpointing must occur after successful data operations with the required comment: # Save the progress by checkpointing the state...

Copilot generated this review using guidance from repository custom instructions.
Loading