Skip to content

Conversation

@sundarshankar89
Copy link
Collaborator

@sundarshankar89 sundarshankar89 commented Sep 15, 2025

Changes

What does this PR do?

  • Introduces the Profiler Skeleton for the Lakebridge project.
  • Adds initial Profiler class with supporting utilities and constants.
  • Implements core logic for profiling supported source technologies with placeholder support for MSSQL and Synapse.
  • Sets up the structure for profiling pipelines, including config file handling and extraction logic.

Relevant implementation details

Caveats/things to watch out for when reviewing:

Linked issues

Resolves #..

Functionality

  • added relevant user documentation
  • added new CLI command
  • modified existing command: databricks labs lakebridge ...
  • ... +add your own

Tests

  • manually tested
  • added unit tests
  • added integration tests

@sundarshankar89 sundarshankar89 requested a review from a team as a code owner September 15, 2025 08:37
@sundarshankar89 sundarshankar89 self-assigned this Sep 15, 2025
@sundarshankar89 sundarshankar89 added feat/profiler Issues related to profilers stacked PR Should be reviewed, but not merged labels Sep 15, 2025
@github-actions
Copy link

github-actions bot commented Sep 15, 2025

✅ 44/44 passed, 6 flaky, 3m11s total

Flaky tests:

  • 🤪 test_validate_non_empty_tables (8ms)
  • 🤪 test_transpiles_informatica_to_sparksql_non_interactive[False] (14.771s)
  • 🤪 test_transpile_teradata_sql_non_interactive[True] (15.311s)
  • 🤪 test_transpile_teradata_sql (17.027s)
  • 🤪 test_transpile_teradata_sql_non_interactive[False] (8.235s)
  • 🤪 test_transpiles_informatica_to_sparksql (9.921s)

Running from acceptance #2630

@sundarshankar89 sundarshankar89 changed the base branch from main to feature/configure-assessment September 15, 2025 08:46
Copy link
Contributor

@m-abulazm m-abulazm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I proposed two small refactoring.

  1. use better name for the constant PLATFORM_TO_PIPELINE_CFG
  2. inject the config in the constructor so we can get rid of patching in the tests

Copy link
Contributor

@m-abulazm m-abulazm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Base automatically changed from feature/synapse_profiler_scripts to main October 14, 2025 05:54
@sundarshankar89 sundarshankar89 added this pull request to the merge queue Oct 14, 2025
Merged via the queue into main with commit 9f20d55 Oct 14, 2025
9 checks passed
@sundarshankar89 sundarshankar89 deleted the feature/add_profiler_skeleton branch October 14, 2025 06:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat/profiler Issues related to profilers stacked PR Should be reviewed, but not merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants