refactor: move source implementation in Invoke() function to Source

[Yuan325]

Move source-related queries from Invoke() function into Source.

This PR addresses the following sources:

• dataplex
• http
• serverlessspark

This is an effort to generalizing tools to work with any Source that implements a specific interface. This will provide a better segregation of the roles for Tools vs Source.

Tool's role will be limited to the following:

• Resolve any pre-implementation steps or parameters (e.g. template parameters)
• Retrieving Source
• Calling the source's implementation

[gemini-code-assist]

Summary of Changes

Hello @Yuan325, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant refactoring effort aimed at enhancing the modularity and maintainability of the codebase. By relocating the core logic for interacting with external services from the Tool layer to the Source layer, the PR establishes a clearer division of responsibilities. This change ensures that Tools primarily handle input processing and orchestration, while Sources are solely responsible for executing specific API operations and data handling. This architectural improvement promotes a more robust and scalable design, making it easier to manage and extend integrations with various data platforms and services.

Highlights

• Architectural Refactoring: Moved source-related query implementations from the Invoke() function within various tools directly into their respective Source implementations. This significantly improves the separation of concerns, making Tools responsible for parameter resolution and delegating API calls to Sources.
• Dataplex Source Enhancements: The dataplex.go source now includes dedicated methods for LookupEntry, SearchAspectTypes, and SearchEntries, encapsulating the logic for interacting with the Dataplex API, including retry mechanisms for aspect type lookups.
• HTTP Source Streamlining: The http.go source now features a RunRequest method that centralizes the execution of HTTP requests, handling response reading, status code validation, and JSON unmarshaling, removing this logic from the http tool's Invoke() function.
• Serverless Spark Source Expansion: The serverlessspark.go source has been extended with methods for CancelOperation, CreateBatch, ListBatches, and GetBatch, providing a comprehensive interface for managing Dataproc Serverless batches directly within the source implementation. Associated URL generation utilities were also moved into this source package.
• Tool Simplification: Corresponding Invoke() methods in dataplexlookupentry, dataplexsearchaspecttypes, dataplexsearchentries, http, serverlesssparkcancelbatch, serverlesssparkcreatebatch, serverlesssparkgetbatch, and serverlesssparklistbatches tools have been simplified to call the new methods on their respective Source interfaces, reducing complexity and improving maintainability.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request is a well-executed refactoring that moves source-specific logic from tools into the source implementations. This improves separation of concerns and makes the tools more generic. The changes for dataplex, http, and serverlessspark sources are clear and follow the intended pattern. I've identified a few areas for improvement: a bug in the retry backoff logic, some dead code, and several unused methods in interfaces that can be cleaned up. Overall, this is a great step towards a more maintainable and extensible architecture.