Skip to content

Introduce a new feature to establish base configurator for Profiler Assessment #1981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 18 commits into from
Oct 6, 2025
Merged

Introduce a new feature to establish base configurator for Profiler Assessment #1981

merged 18 commits into from
Oct 6, 2025

Conversation

@sundarshankar89
Copy link
Collaborator

@sundarshankar89 sundarshankar89 commented Sep 2, 2025 •
edited
Loading

Changes

What does this PR do?

  • Introduces a new feature to establish a base configurator for Profiler Assessment within the project.

Relevant implementation details

Caveats/things to watch out for when reviewing:

Linked issues

Resolves #..

Functionality

  • added relevant user documentation
  • added new CLI command
  • modified existing command: databricks labs lakebridge ...
  • ... +add your own

Tests

  • manually tested
  • added unit tests
  • added integration tests

@sundarshankar89 sundarshankar89 requested a review from a team as a code owner September 2, 2025 12:28
@sundarshankar89 sundarshankar89 changed the title Add a new Feature to establish base configurator for Profiler Assessment Add a new feature to establish base configurator for Profiler Assessment Sep 2, 2025
@github-actions
Copy link

github-actions bot commented Sep 2, 2025 •
edited
Loading

✅ 27/27 passed, 2 flaky, 1m33s total

Flaky tests:

  • 🤪 test_transpiles_informatica_with_sparksql (10.438s)
  • 🤪 test_transpile_sql_file (9.478s)

Running from acceptance #2498

@sundarshankar89 sundarshankar89 changed the title Add a new feature to establish base configurator for Profiler Assessment Introduce a new feature to establish base configurator for Profiler Assessment Sep 2, 2025
@sundarshankar89
Copy link
Collaborator Author

Merged #1940 in this

co-authored by: @goodwillpunning

m-abulazm
m-abulazm reviewed Sep 3, 2025
View reviewed changes
src/databricks/labs/lakebridge/assessments/pipeline.py
logger.info(f"Creating a virtual environment for Python script execution: ${venv_dir}")
# Install dependencies in the virtual environment
# Define the paths to the virtual environment's Python and pip executables
if sys.platform == "win32":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to standardize this with databricks.labs.lakebridge.install.WheelInstaller._create_venv
please add a TODO for later

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • create_env
  • activate env
  • _install_dependencies
  • _run_python_script

are all generic and can be extracted out to a common utility class

m-abulazm
m-abulazm requested changes Sep 3, 2025
View reviewed changes
src/databricks/labs/lakebridge/connections/database_manager.py
"snowflake": SnowflakeConnector,
"mssql": MSSQLConnector,
"tsql": MSSQLConnector,
"synapse": MSSQLConnector,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this intentional?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

@sundarshankar89 sundarshankar89 self-assigned this Sep 15, 2025
@sundarshankar89 sundarshankar89 added the feat/profiler Issues related to profilers label Sep 15, 2025
m-abulazm
m-abulazm approved these changes Sep 19, 2025
View reviewed changes
Copy link
Contributor

@m-abulazm m-abulazm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gueniai gueniai mentioned this pull request Oct 1, 2025
Closed
7 tasks
@sundarshankar89
Copy link
Collaborator Author

@gueniai, this PR is waiting on the new Blueprint release.

@sundarshankar89
Copy link
Collaborator Author

the format fix for this PR is being addressed by #2080

@sundarshankar89 sundarshankar89 changed the base branch from main to bump/blueprint-0.11.4 October 6, 2025 12:02
@sundarshankar89 sundarshankar89 added stacked PR Should be reviewed, but not merged and removed do-not-merge stacked PR Should be reviewed, but not merged labels Oct 6, 2025
Base automatically changed from bump/blueprint-0.11.4 to main October 6, 2025 16:43
@sundarshankar89 sundarshankar89 added this pull request to the merge queue Oct 6, 2025
Merged via the queue into main with commit c112a30 Oct 6, 2025
9 checks passed
@sundarshankar89 sundarshankar89 deleted the feature/configure-assessment branch October 6, 2025 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gueniai gueniai gueniai approved these changes

@m-abulazm m-abulazm m-abulazm approved these changes

@goodwillpunning goodwillpunning Awaiting requested review from goodwillpunning

Assignees

@sundarshankar89 sundarshankar89

Labels

feat/profiler Issues related to profilers

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sundarshankar89 @gueniai @m-abulazm