diff --git a/.changes/header.tpl.md b/.changes/header.tpl.md new file mode 100644 index 00000000..825c32f0 --- /dev/null +++ b/.changes/header.tpl.md @@ -0,0 +1 @@ +# Changelog diff --git a/.changes/unreleased/added-20260203-163010.yaml b/.changes/unreleased/added-20260203-163010.yaml new file mode 100644 index 00000000..60a71cbd --- /dev/null +++ b/.changes/unreleased/added-20260203-163010.yaml @@ -0,0 +1,8 @@ +kind: added +body: Support parallelize deployments within a given item type +time: 2026-02-03T16:30:10.310834+02:00 +custom: + Author: mdrakiburrahman + AuthorLink: https://github.com/mdrakiburrahman + Issue: "719" + IssueLink: https://github.com/microsoft/fabric-cicd/issues/719 diff --git a/.changes/unreleased/docs-20260127-110059.yaml b/.changes/unreleased/docs-20260127-110059.yaml new file mode 100644 index 00000000..a94f47fc --- /dev/null +++ b/.changes/unreleased/docs-20260127-110059.yaml @@ -0,0 +1,8 @@ +kind: docs +body: Fix troubleshooting docs +time: 2026-01-27T11:00:59.989726+02:00 +custom: + Author: shirasassoon + AuthorLink: https://github.com/shirasassoon + Issue: "747" + IssueLink: https://github.com/microsoft/fabric-cicd/issues/747 diff --git a/.changes/unreleased/fixed-20260128-102844.yaml b/.changes/unreleased/fixed-20260128-102844.yaml new file mode 100644 index 00000000..f668e33e --- /dev/null +++ b/.changes/unreleased/fixed-20260128-102844.yaml @@ -0,0 +1,8 @@ +kind: fixed +body: Remove OrgApp item type support +time: 2026-01-28T10:28:44.435609802Z +custom: + Author: shirasassoon + AuthorLink: https://github.com/shirasassoon + Issue: "758" + IssueLink: https://github.com/microsoft/fabric-cicd/issues/758 diff --git a/.changes/unreleased/fixed-20260202-115910.yaml b/.changes/unreleased/fixed-20260202-115910.yaml new file mode 100644 index 00000000..215d5353 --- /dev/null +++ b/.changes/unreleased/fixed-20260202-115910.yaml @@ -0,0 +1,8 @@ +kind: fixed +body: Improve environment-mapping behavior in optional config fields +time: 2026-02-02T11:59:10.371508+02:00 +custom: + Author: shirasassoon + AuthorLink: https://github.com/shirasassoon + Issue: "716" + IssueLink: https://github.com/microsoft/fabric-cicd/issues/716 diff --git a/.changes/v0.1.0.md b/.changes/v0.1.0.md new file mode 100644 index 00000000..c1fac11a --- /dev/null +++ b/.changes/v0.1.0.md @@ -0,0 +1,9 @@ +## [v0.1.0](https://pypi.org/project/fabric-cicd/0.1.0) - January 23, 2025 + +### โœจ New Functionality + +- Initial public preview release +- Supports Notebook, Pipeline, Semantic Model, Report, and Environment deployments +- Supports User and System Identity authentication +- Released to PyPi +- Onboarded to Github Pages diff --git a/.changes/v0.1.1.md b/.changes/v0.1.1.md new file mode 100644 index 00000000..3cbf40f5 --- /dev/null +++ b/.changes/v0.1.1.md @@ -0,0 +1,5 @@ +## [v0.1.1](https://pypi.org/project/fabric-cicd/0.1.1) - January 23, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix Environment stuck in publish ([#51](https://github.com/microsoft/fabric-cicd/issues/51)) diff --git a/.changes/v0.1.10.md b/.changes/v0.1.10.md new file mode 100644 index 00000000..5dac74e4 --- /dev/null +++ b/.changes/v0.1.10.md @@ -0,0 +1,13 @@ +## [v0.1.10](https://pypi.org/project/fabric-cicd/0.1.10) - March 19, 2025 + +### โœจ New Functionality + +- DataPipeline SPN Support ([#133](https://github.com/microsoft/fabric-cicd/issues/133)) + +### ๐Ÿ”ง Bug Fix + +- Workspace ID replacement in data pipelines ([#164](https://github.com/microsoft/fabric-cicd/issues/164)) + +### ๐Ÿ“ Documentation Update + +- Sample for passing in arguments from Azure DevOps Pipelines diff --git a/.changes/v0.1.11.md b/.changes/v0.1.11.md new file mode 100644 index 00000000..63363bbe --- /dev/null +++ b/.changes/v0.1.11.md @@ -0,0 +1,14 @@ +## [v0.1.11](https://pypi.org/project/fabric-cicd/0.1.11) - March 25, 2025 + +### โš ๏ธ Breaking Change + +- Parameterization refactor introducing a new parameter file structure and parameter file validation functionality ([#113](https://github.com/microsoft/fabric-cicd/issues/113)) + +### โœจ New Functionality + +- Support regex for publish exclusion ([#121](https://github.com/microsoft/fabric-cicd/issues/121)) +- Override max retries via constants ([#146](https://github.com/microsoft/fabric-cicd/issues/146)) + +### ๐Ÿ“ Documentation Update + +- Update to [parameterization](https://microsoft.github.io/fabric-cicd/latest/how_to/parameterization/) docs diff --git a/.changes/v0.1.12.md b/.changes/v0.1.12.md new file mode 100644 index 00000000..663443eb --- /dev/null +++ b/.changes/v0.1.12.md @@ -0,0 +1,8 @@ +## [v0.1.12](https://pypi.org/project/fabric-cicd/0.1.12) - March 27, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix constant overwrite failures ([#190](https://github.com/microsoft/fabric-cicd/issues/190)) +- Fix bug where all workspace ids were not being replaced ([#186](https://github.com/microsoft/fabric-cicd/issues/186)) +- Fix type hints for older versions of Python ([#156](https://github.com/microsoft/fabric-cicd/issues/156)) +- Fix accepted item types constant in pre-build diff --git a/.changes/v0.1.13.md b/.changes/v0.1.13.md new file mode 100644 index 00000000..85724787 --- /dev/null +++ b/.changes/v0.1.13.md @@ -0,0 +1,21 @@ +## [v0.1.13](https://pypi.org/project/fabric-cicd/0.1.13) - April 07, 2025 + +### โœจ New Functionality + +- Added support for Lakehouse Shortcuts +- New `enable_environment_variable_replacement` feature flag ([#160](https://github.com/microsoft/fabric-cicd/issues/160)) + +### ๐Ÿ†• New Items Support + +- Onboard Workspace Folders ([#81](https://github.com/microsoft/fabric-cicd/issues/81)) +- Onboard Variable Library item type ([#206](https://github.com/microsoft/fabric-cicd/issues/206)) + +### โšก Additional Optimizations + +- User-agent now available in API headers ([#207](https://github.com/microsoft/fabric-cicd/issues/207)) +- Fixed error log typo in fabric_endpoint + +### ๐Ÿ”ง Bug Fix + +- Fix break with invalid optional parameters ([#192](https://github.com/microsoft/fabric-cicd/issues/192)) +- Fix bug where all workspace ids were not being replaced by parameterization ([#186](https://github.com/microsoft/fabric-cicd/issues/186)) diff --git a/.changes/v0.1.14.md b/.changes/v0.1.14.md new file mode 100644 index 00000000..a3695906 --- /dev/null +++ b/.changes/v0.1.14.md @@ -0,0 +1,15 @@ +## [v0.1.14](https://pypi.org/project/fabric-cicd/0.1.14) - April 09, 2025 + +### โœจ New Functionality + +- Optimized & beautified terminal output +- Added changelog to output of old version check + +### ๐Ÿ”ง Bug Fix + +- Fix workspace folder deployments in root folder ([#221](https://github.com/microsoft/fabric-cicd/issues/221)) +- Fix unpublish of workspace folders without publish ([#222](https://github.com/microsoft/fabric-cicd/issues/222)) + +### โšก Additional Optimizations + +- Removed Colorama and Colorlog Dependency diff --git a/.changes/v0.1.15.md b/.changes/v0.1.15.md new file mode 100644 index 00000000..fbb632a1 --- /dev/null +++ b/.changes/v0.1.15.md @@ -0,0 +1,14 @@ +## [v0.1.15](https://pypi.org/project/fabric-cicd/0.1.15) - April 21, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix folders moving with every publish ([#236](https://github.com/microsoft/fabric-cicd/issues/236)) + +### โšก Additional Optimizations + +- Introduce parallel deployments to reduce publish times ([#237](https://github.com/microsoft/fabric-cicd/issues/237)) +- Improvements to check version logic + +### ๐Ÿ“ Documentation Update + +- Updated Examples section in docs diff --git a/.changes/v0.1.16.md b/.changes/v0.1.16.md new file mode 100644 index 00000000..2b7c4435 --- /dev/null +++ b/.changes/v0.1.16.md @@ -0,0 +1,14 @@ +## [v0.1.16](https://pypi.org/project/fabric-cicd/0.1.16) - April 25, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix bug with folder deployment to root ([#255](https://github.com/microsoft/fabric-cicd/issues/255)) + +### โšก Additional Optimizations + +- Add Workspace Name in FabricWorkspaceObject ([#200](https://github.com/microsoft/fabric-cicd/issues/200)) +- New function to check SQL endpoint provision status ([#226](https://github.com/microsoft/fabric-cicd/issues/226)) + +### ๐Ÿ“ Documentation Update + +- Updated Authentication docs + menu sort order diff --git a/.changes/v0.1.17.md b/.changes/v0.1.17.md new file mode 100644 index 00000000..430405ce --- /dev/null +++ b/.changes/v0.1.17.md @@ -0,0 +1,25 @@ +## [v0.1.17](https://pypi.org/project/fabric-cicd/0.1.17) - May 13, 2025 + +### โš ๏ธ Breaking Change + +- Deprecate old parameter file structure ([#283](https://github.com/microsoft/fabric-cicd/issues/283)) + +### ๐Ÿ†• New Items Support + +- Onboard CopyJob item type ([#122](https://github.com/microsoft/fabric-cicd/issues/122)) +- Onboard Eventstream item type ([#170](https://github.com/microsoft/fabric-cicd/issues/170)) +- Onboard Eventhouse/KQL Database item type ([#169](https://github.com/microsoft/fabric-cicd/issues/169)) +- Onboard Data Activator item type ([#291](https://github.com/microsoft/fabric-cicd/issues/291)) +- Onboard KQL Queryset item type ([#292](https://github.com/microsoft/fabric-cicd/issues/292)) + +### ๐Ÿ”ง Bug Fix + +- Fix post publish operations for skipped items ([#277](https://github.com/microsoft/fabric-cicd/issues/277)) + +### โšก Additional Optimizations + +- New function `key_value_replace` for key-based replacement operations in JSON and YAML + +### ๐Ÿ“ Documentation Update + +- Add publish regex example to demonstrate how to use the `publish_all_items` with regex for excluding item names diff --git a/.changes/v0.1.18.md b/.changes/v0.1.18.md new file mode 100644 index 00000000..1e8d8c74 --- /dev/null +++ b/.changes/v0.1.18.md @@ -0,0 +1,5 @@ +## [v0.1.18](https://pypi.org/project/fabric-cicd/0.1.18) - May 14, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix bug with check environment publish state ([#295](https://github.com/microsoft/fabric-cicd/issues/295)) diff --git a/.changes/v0.1.19.md b/.changes/v0.1.19.md new file mode 100644 index 00000000..95fb013c --- /dev/null +++ b/.changes/v0.1.19.md @@ -0,0 +1,10 @@ +## [v0.1.19](https://pypi.org/project/fabric-cicd/0.1.19) - May 21, 2025 + +### ๐Ÿ†• New Items Support + +- Onboard SQL Database item type (shell-only deployment) ([#301](https://github.com/microsoft/fabric-cicd/issues/301)) +- Onboard Warehouse item type (shell-only deployment) ([#204](https://github.com/microsoft/fabric-cicd/issues/204)) + +### ๐Ÿ”ง Bug Fix + +- Fix bug with unpublish workspace folders ([#273](https://github.com/microsoft/fabric-cicd/issues/273)) diff --git a/.changes/v0.1.2.md b/.changes/v0.1.2.md new file mode 100644 index 00000000..00da361f --- /dev/null +++ b/.changes/v0.1.2.md @@ -0,0 +1,11 @@ +## [v0.1.2](https://pypi.org/project/fabric-cicd/0.1.2) - January 27, 2025 + +### โœจ New Functionality + +- Introduces max retry and backoff for long running / throttled calls ([#27](https://github.com/microsoft/fabric-cicd/issues/27)) + +### ๐Ÿ”ง Bug Fix + +- Fix Environment publish uses arbitrary wait time ([#50](https://github.com/microsoft/fabric-cicd/issues/50)) +- Fix Environment publish doesn't wait for success ([#56](https://github.com/microsoft/fabric-cicd/issues/56)) +- Fix Long running operation steps out early for notebook publish ([#58](https://github.com/microsoft/fabric-cicd/issues/58)) diff --git a/.changes/v0.1.20.md b/.changes/v0.1.20.md new file mode 100644 index 00000000..da8ac0a6 --- /dev/null +++ b/.changes/v0.1.20.md @@ -0,0 +1,18 @@ +## [v0.1.20](https://pypi.org/project/fabric-cicd/0.1.20) - June 12, 2025 + +### โœจ New Functionality + +- Parameterization support for find_value regex and replace_value variables ([#326](https://github.com/microsoft/fabric-cicd/issues/326)) + +### ๐Ÿ†• New Items Support + +- Onboard KQL Dashboard item type ([#329](https://github.com/microsoft/fabric-cicd/issues/329)) +- Onboard Dataflow Gen2 item type ([#111](https://github.com/microsoft/fabric-cicd/issues/111)) + +### ๐Ÿ”ง Bug Fix + +- Fix bug with deploying environment libraries with special chars ([#336](https://github.com/microsoft/fabric-cicd/issues/336)) + +### โšก Additional Optimizations + +- Improved test coverage for subfolder creation/modification ([#211](https://github.com/microsoft/fabric-cicd/issues/211)) diff --git a/.changes/v0.1.21.md b/.changes/v0.1.21.md new file mode 100644 index 00000000..2aa58b06 --- /dev/null +++ b/.changes/v0.1.21.md @@ -0,0 +1,9 @@ +## [v0.1.21](https://pypi.org/project/fabric-cicd/0.1.21) - June 18, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix bug with workspace ID replacement in JSON files for pipeline deployments ([#345](https://github.com/microsoft/fabric-cicd/issues/345)) + +### โšก Additional Optimizations + +- Increased max retry for Warehouses and Dataflows diff --git a/.changes/v0.1.22.md b/.changes/v0.1.22.md new file mode 100644 index 00000000..99b24168 --- /dev/null +++ b/.changes/v0.1.22.md @@ -0,0 +1,13 @@ +## [v0.1.22](https://pypi.org/project/fabric-cicd/0.1.22) - June 25, 2025 + +### ๐Ÿ†• New Items Support + +- Onboard API for GraphQL item type ([#287](https://github.com/microsoft/fabric-cicd/issues/287)) + +### ๐Ÿ”ง Bug Fix + +- Fix Fabric API call error during dataflow publish ([#352](https://github.com/microsoft/fabric-cicd/issues/352)) + +### โšก Additional Optimizations + +- Expanded test coverage to handle folder edge cases ([#358](https://github.com/microsoft/fabric-cicd/issues/358)) diff --git a/.changes/v0.1.23.md b/.changes/v0.1.23.md new file mode 100644 index 00000000..ad91a5c5 --- /dev/null +++ b/.changes/v0.1.23.md @@ -0,0 +1,18 @@ +## [v0.1.23](https://pypi.org/project/fabric-cicd/0.1.23) - July 08, 2025 + +### โœจ New Functionality + +- New functionalities for GitHub Copilot Agent and PR-to-Issue linking + +### ๐Ÿ“ Documentation Update + +- Fix formatting and examples in the How to and Examples pages + +### ๐Ÿ”ง Bug Fix + +- Fix issue with lakehouse shortcuts publishing ([#379](https://github.com/microsoft/fabric-cicd/issues/379)) +- Add validation for empty logical IDs to prevent deployment corruption ([#86](https://github.com/microsoft/fabric-cicd/issues/86)) +- Fix SQL provision print statement ([#329](https://github.com/microsoft/fabric-cicd/issues/329)) +- Rename the error code for reserved item name per updated Microsoft Fabric API ([#388](https://github.com/microsoft/fabric-cicd/issues/388)) +- Fix lakehouse exclude_regex to exclude shortcut publishing ([#385](https://github.com/microsoft/fabric-cicd/issues/385)) +- Remove max retry limit to handle large deployments ([#299](https://github.com/microsoft/fabric-cicd/issues/299)) diff --git a/.changes/v0.1.24.md b/.changes/v0.1.24.md new file mode 100644 index 00000000..6279184a --- /dev/null +++ b/.changes/v0.1.24.md @@ -0,0 +1,21 @@ +## [v0.1.24](https://pypi.org/project/fabric-cicd/0.1.24) - August 04, 2025 + +### โš ๏ธ Breaking Change + +- Require parameterization for Dataflow and Semantic Model references in Data Pipeline activities +- Require specific parameterization for deploying a Dataflow that depends on another in the same workspace (see Parameterization docs) + +### ๐Ÿ“ Documentation Update + +- Improve Parameterization documentation ([#415](https://github.com/microsoft/fabric-cicd/issues/415)) + +### โšก Additional Optimizations + +- Support for Eventhouse query URI parameterization ([#414](https://github.com/microsoft/fabric-cicd/issues/414)) +- Support for Warehouse SQL endpoint parameterization ([#392](https://github.com/microsoft/fabric-cicd/issues/392)) + +### ๐Ÿ”ง Bug Fix + +- Fix Dataflow/Data Pipeline deployment failures caused by workspace permissions ([#419](https://github.com/microsoft/fabric-cicd/issues/419)) +- Prevent duplicate logical ID issue in Report and Semantic Model deployment ([#405](https://github.com/microsoft/fabric-cicd/issues/405)) +- Fix deployment of items without assigned capacity ([#402](https://github.com/microsoft/fabric-cicd/issues/402)) diff --git a/.changes/v0.1.25.md b/.changes/v0.1.25.md new file mode 100644 index 00000000..9ada86b4 --- /dev/null +++ b/.changes/v0.1.25.md @@ -0,0 +1,25 @@ +## [v0.1.25](https://pypi.org/project/fabric-cicd/0.1.25) - August 19, 2025 + +### โš ๏ธ Breaking Change + +- Modify the default for item_types_in_scope and add thorough validation ([#464](https://github.com/microsoft/fabric-cicd/issues/464)) + +### โœจ New Functionality + +- Add new experimental feature flag to enable selective deployment ([#384](https://github.com/microsoft/fabric-cicd/issues/384)) +- Support "ALL" environment concept in parameterization ([#320](https://github.com/microsoft/fabric-cicd/issues/320)) + +### ๐Ÿ“ Documentation Update + +- Enhance Overview section in Parameterization docs ([#495](https://github.com/microsoft/fabric-cicd/issues/495)) + +### โšก Additional Optimizations + +- Eliminate ACCEPTED_ITEM_TYPES_NON_UPN constant and unify with ACCEPTED_ITEM_TYPES ([#477](https://github.com/microsoft/fabric-cicd/issues/477)) +- Add comprehensive GitHub Copilot instructions for effective codebase development ([#468](https://github.com/microsoft/fabric-cicd/issues/468)) + +### ๐Ÿ”ง Bug Fix + +- Add feature flags and warnings for Warehouse, SQL Database, and Eventhouse unpublish operations ([#483](https://github.com/microsoft/fabric-cicd/issues/483)) +- Fix code formatting inconsistencies in fabric_workspace unit test ([#474](https://github.com/microsoft/fabric-cicd/issues/474)) +- Fix KeyError when deploying Reports with Semantic Model dependencies in Report-only scope case ([#278](https://github.com/microsoft/fabric-cicd/issues/278)) diff --git a/.changes/v0.1.26.md b/.changes/v0.1.26.md new file mode 100644 index 00000000..1e0ec009 --- /dev/null +++ b/.changes/v0.1.26.md @@ -0,0 +1,23 @@ +## [v0.1.26](https://pypi.org/project/fabric-cicd/0.1.26) - September 05, 2025 + +### โš ๏ธ Breaking Change + +- Deprecate Base API URL kwarg in Fabric Workspace ([#529](https://github.com/microsoft/fabric-cicd/issues/529)) + +### โœจ New Functionality + +- Support Schedules parameterization ([#508](https://github.com/microsoft/fabric-cicd/issues/508)) +- Support YAML configuration file-based deployment ([#470](https://github.com/microsoft/fabric-cicd/issues/470)) + +### ๐Ÿ“ Documentation Update + +- Add dynamically generated Python version requirements to documentation ([#520](https://github.com/microsoft/fabric-cicd/issues/520)) + +### โšก Additional Optimizations + +- Enhance pytest output to limit console verbosity ([#514](https://github.com/microsoft/fabric-cicd/issues/514)) + +### ๐Ÿ”ง Bug Fix + +- Fix Report item schema handling ([#518](https://github.com/microsoft/fabric-cicd/issues/518)) +- Fix deployment order to publish Mirrored Database before Lakehouse ([#482](https://github.com/microsoft/fabric-cicd/issues/482)) diff --git a/.changes/v0.1.27.md b/.changes/v0.1.27.md new file mode 100644 index 00000000..4247fd0a --- /dev/null +++ b/.changes/v0.1.27.md @@ -0,0 +1,5 @@ +## [v0.1.27](https://pypi.org/project/fabric-cicd/0.1.27) - September 05, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix trailing comma in report schema ([#534](https://github.com/microsoft/fabric-cicd/issues/534)) diff --git a/.changes/v0.1.28.md b/.changes/v0.1.28.md new file mode 100644 index 00000000..81d95fb2 --- /dev/null +++ b/.changes/v0.1.28.md @@ -0,0 +1,11 @@ +## [v0.1.28](https://pypi.org/project/fabric-cicd/0.1.28) - September 15, 2025 + +### โœจ New Functionality + +- Add folder exclusion feature for publish operations ([#427](https://github.com/microsoft/fabric-cicd/issues/427)) +- Expand workspace ID dynamic replacement capabilities in parameterization ([#408](https://github.com/microsoft/fabric-cicd/issues/408)) + +### ๐Ÿ”ง Bug Fix + +- Fix unexpected behavior with file_path parameter filter ([#545](https://github.com/microsoft/fabric-cicd/issues/545)) +- Fix unpublish exclude_regex bug in configuration file-based deployment ([#544](https://github.com/microsoft/fabric-cicd/issues/544)) diff --git a/.changes/v0.1.29.md b/.changes/v0.1.29.md new file mode 100644 index 00000000..203c25ab --- /dev/null +++ b/.changes/v0.1.29.md @@ -0,0 +1,15 @@ +## [v0.1.29](https://pypi.org/project/fabric-cicd/0.1.29) - October 01, 2025 + +### โœจ New Functionality + +- Support dynamic replacement for cross-workspace item IDs ([#558](https://github.com/microsoft/fabric-cicd/issues/558)) +- Add option to return API response for publish operations in publish_all_items ([#497](https://github.com/microsoft/fabric-cicd/issues/497)) + +### ๐Ÿ†• New Items Support + +- Onboard Apache Airflow Job item type ([#565](https://github.com/microsoft/fabric-cicd/issues/565)) +- Onboard Mounted Data Factory item type ([#406](https://github.com/microsoft/fabric-cicd/issues/406)) + +### ๐Ÿ”ง Bug Fix + +- Fix publish order of Eventhouses and Semantic Models ([#566](https://github.com/microsoft/fabric-cicd/issues/566)) diff --git a/.changes/v0.1.3.md b/.changes/v0.1.3.md new file mode 100644 index 00000000..226899d0 --- /dev/null +++ b/.changes/v0.1.3.md @@ -0,0 +1,16 @@ +## [v0.1.3](https://pypi.org/project/fabric-cicd/0.1.3) - January 29, 2025 + +### โœจ New Functionality + +- Add PyPI check version to encourage version bumps ([#75](https://github.com/microsoft/fabric-cicd/issues/75)) + +### ๐Ÿ”ง Bug Fix + +- Fix Semantic model initial publish results in None Url error ([#61](https://github.com/microsoft/fabric-cicd/issues/61)) +- Fix Integer parsed as float failing in handle_retry for <3.12 python ([#63](https://github.com/microsoft/fabric-cicd/issues/63)) +- Fix Default item types fail to unpublish ([#76](https://github.com/microsoft/fabric-cicd/issues/76)) +- Fix Items in subfolders are skipped ([#77](https://github.com/microsoft/fabric-cicd/issues/77)) + +### ๐Ÿ“ Documentation Update + +- Update documentation & examples diff --git a/.changes/v0.1.30.md b/.changes/v0.1.30.md new file mode 100644 index 00000000..f6cccea8 --- /dev/null +++ b/.changes/v0.1.30.md @@ -0,0 +1,19 @@ +## [v0.1.30](https://pypi.org/project/fabric-cicd/0.1.30) - October 20, 2025 + +### โœจ New Functionality + +- Add support for binding semantic models to on-premise gateways in Fabric workspaces ([#569](https://github.com/microsoft/fabric-cicd/issues/569)) + +### ๐Ÿ†• New Items Support + +- Add support for publishing and managing Data Agent items ([#556](https://github.com/microsoft/fabric-cicd/issues/556)) +- Add OrgApp item type support ([#586](https://github.com/microsoft/fabric-cicd/issues/586)) + +### โšก Additional Optimizations + +- Enhance cross-workspace variable support to allow referencing other attributes ([#583](https://github.com/microsoft/fabric-cicd/issues/583)) + +### ๐Ÿ”ง Bug Fix + +- Fix workspace name extraction bug for non-ID attrs using ITEM_ATTR_LOOKUP ([#583](https://github.com/microsoft/fabric-cicd/issues/583)) +- Fix capacity requirement check ([#593](https://github.com/microsoft/fabric-cicd/issues/593)) diff --git a/.changes/v0.1.31.md b/.changes/v0.1.31.md new file mode 100644 index 00000000..f4fe0f7d --- /dev/null +++ b/.changes/v0.1.31.md @@ -0,0 +1,29 @@ +## [v0.1.31](https://pypi.org/project/fabric-cicd/0.1.31) - December 01, 2025 + +### โš ๏ธ Breaking Change + +- Migrate to the latest Fabric Environment item APIs to simplify deployment and improve compatibility ([#173](https://github.com/microsoft/fabric-cicd/issues/173)) + +### โœจ New Functionality + +- Enable dynamic replacement of Lakehouse SQL Endpoint IDs ([#616](https://github.com/microsoft/fabric-cicd/issues/616)) +- Enable linking of Semantic Models to both cloud and gateway connections ([#602](https://github.com/microsoft/fabric-cicd/issues/602)) +- Allow use of the dynamic replacement variables within the key_value_replace parameter ([#567](https://github.com/microsoft/fabric-cicd/issues/567)) +- Add support for parameter file templates ([#499](https://github.com/microsoft/fabric-cicd/issues/499)) + +### ๐Ÿ†• New Items Support + +- Add support for the ML Experiment item type ([#600](https://github.com/microsoft/fabric-cicd/issues/600)) +- Add support for the User Data Function item type ([#588](https://github.com/microsoft/fabric-cicd/issues/588)) + +### ๐Ÿ“ Documentation Update + +- Update the advanced Dataflow parameterization example with the correct file_path value ([#633](https://github.com/microsoft/fabric-cicd/issues/633)) + +### ๐Ÿ”ง Bug Fix + +- Fix publishing issues for KQL Database items in folders ([#657](https://github.com/microsoft/fabric-cicd/issues/657)) +- Separate logic for 'items to include' feature between publish and unpublish operations ([#650](https://github.com/microsoft/fabric-cicd/issues/650)) +- Fix parameterization logic to properly handle find_value regex patterns and replacements ([#639](https://github.com/microsoft/fabric-cicd/issues/639)) +- Correct the publish order of Data Agent and Semantic Model items ([#628](https://github.com/microsoft/fabric-cicd/issues/628)) +- Fix Lakehouse item publishing errors when shortcuts refer to the default Lakehouse ID ([#610](https://github.com/microsoft/fabric-cicd/issues/610)) diff --git a/.changes/v0.1.32.md b/.changes/v0.1.32.md new file mode 100644 index 00000000..24e7a9c7 --- /dev/null +++ b/.changes/v0.1.32.md @@ -0,0 +1,5 @@ +## [v0.1.32](https://pypi.org/project/fabric-cicd/0.1.32) - December 03, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix publish bug for Environment items that contain only spark settings ([#664](https://github.com/microsoft/fabric-cicd/issues/664)) diff --git a/.changes/v0.1.33.md b/.changes/v0.1.33.md new file mode 100644 index 00000000..bb60b27d --- /dev/null +++ b/.changes/v0.1.33.md @@ -0,0 +1,18 @@ +## [v0.1.33](https://pypi.org/project/fabric-cicd/0.1.33) - December 16, 2025 + +### โœจ New Functionality + +- Add key_value_replace parameter support for YAML files ([#649](https://github.com/microsoft/fabric-cicd/issues/649)) +- Support selective shortcut publishing with regex exclusion ([#624](https://github.com/microsoft/fabric-cicd/issues/624)) + +### โšก Additional Optimizations + +- Add Linux development environment bootstrapping script ([#680](https://github.com/microsoft/fabric-cicd/issues/680)) +- Update item types in scope to be an optional parameter in validate parameter file function ([#669](https://github.com/microsoft/fabric-cicd/issues/669)) + +### ๐Ÿ”ง Bug Fix + +- Fix publish order for Notebook and Eventhouse dependent items ([#685](https://github.com/microsoft/fabric-cicd/issues/685)) +- Enable parameterizing multiple connections in the same Semantic Model item ([#674](https://github.com/microsoft/fabric-cicd/issues/674)) +- Fix missing description metadata in item payload for shell-only item deployments ([#672](https://github.com/microsoft/fabric-cicd/issues/672)) +- Resolve API long running operation handling when publishing Environment items ([#668](https://github.com/microsoft/fabric-cicd/issues/668)) diff --git a/.changes/v0.1.34.md b/.changes/v0.1.34.md new file mode 100644 index 00000000..0d0026f8 --- /dev/null +++ b/.changes/v0.1.34.md @@ -0,0 +1,20 @@ +## [v0.1.34](https://pypi.org/project/fabric-cicd/0.1.34) - January 20, 2026 + +### โœจ New Functionality + +- Enable dynamic replacement of SQL endpoint values from SQL Database items ([#720](https://github.com/microsoft/fabric-cicd/issues/720)) +- Support Fabric Notebook Authentication ([#707](https://github.com/microsoft/fabric-cicd/issues/707)) + +### ๐Ÿ†• New Items Support + +- Onboard Spark Job Definition item type ([#115](https://github.com/microsoft/fabric-cicd/issues/115)) + +### ๐Ÿ“ Documentation Update + +- Add `CONTRIBUTING.md` file to repository ([#723](https://github.com/microsoft/fabric-cicd/issues/723)) +- Add comprehensive troubleshooting guide to documentation ([#705](https://github.com/microsoft/fabric-cicd/issues/705)) +- Add parameterization documentation for Report items using ByConnection binding to Semantic Models ([#637](https://github.com/microsoft/fabric-cicd/issues/637)) + +### โšก Additional Optimizations + +- Add debug file for local Fabric REST API testing ([#714](https://github.com/microsoft/fabric-cicd/issues/714)) diff --git a/.changes/v0.1.4.md b/.changes/v0.1.4.md new file mode 100644 index 00000000..6132b0d2 --- /dev/null +++ b/.changes/v0.1.4.md @@ -0,0 +1,18 @@ +## [v0.1.4](https://pypi.org/project/fabric-cicd/0.1.4) - February 12, 2025 + +### โœจ New Functionality + +- Support Feature Flagging ([#96](https://github.com/microsoft/fabric-cicd/issues/96)) + +### ๐Ÿ”ง Bug Fix + +- Fix Image support in report deployment ([#88](https://github.com/microsoft/fabric-cicd/issues/88)) +- Fix Broken README link ([#92](https://github.com/microsoft/fabric-cicd/issues/92)) + +### โšก Additional Optimizations + +- Workspace ID replacement improved +- Increased error handling in activate script +- Onboard pytest and coverage +- Improvements to nested dictionaries ([#37](https://github.com/microsoft/fabric-cicd/issues/37)) +- Support Python Installed From Windows Store ([#87](https://github.com/microsoft/fabric-cicd/issues/87)) diff --git a/.changes/v0.1.5.md b/.changes/v0.1.5.md new file mode 100644 index 00000000..a004689c --- /dev/null +++ b/.changes/v0.1.5.md @@ -0,0 +1,9 @@ +## [v0.1.5](https://pypi.org/project/fabric-cicd/0.1.5) - February 18, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix Environment Failure without Public Library ([#103](https://github.com/microsoft/fabric-cicd/issues/103)) + +### โšก Additional Optimizations + +- Introduces pytest check for PRs ([#100](https://github.com/microsoft/fabric-cicd/issues/100)) diff --git a/.changes/v0.1.6.md b/.changes/v0.1.6.md new file mode 100644 index 00000000..2c98ef31 --- /dev/null +++ b/.changes/v0.1.6.md @@ -0,0 +1,17 @@ +## [v0.1.6](https://pypi.org/project/fabric-cicd/0.1.6) - February 24, 2025 + +### ๐Ÿ†• New Items Support + +- Onboard Lakehouse item type ([#116](https://github.com/microsoft/fabric-cicd/issues/116)) + +### ๐Ÿ“ Documentation Update + +- Update example docs ([#25](https://github.com/microsoft/fabric-cicd/issues/25)) +- Update find_replace docs ([#110](https://github.com/microsoft/fabric-cicd/issues/110)) + +### โšก Additional Optimizations + +- Standardized docstrings to Google format +- Onboard file objects ([#46](https://github.com/microsoft/fabric-cicd/issues/46)) +- Leverage UpdateDefinition Flag ([#28](https://github.com/microsoft/fabric-cicd/issues/28)) +- Convert repo and workspace dictionaries ([#45](https://github.com/microsoft/fabric-cicd/issues/45)) diff --git a/.changes/v0.1.7.md b/.changes/v0.1.7.md new file mode 100644 index 00000000..240942b3 --- /dev/null +++ b/.changes/v0.1.7.md @@ -0,0 +1,5 @@ +## [v0.1.7](https://pypi.org/project/fabric-cicd/0.1.7) - February 26, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix special character support in files ([#129](https://github.com/microsoft/fabric-cicd/issues/129)) diff --git a/.changes/v0.1.8.md b/.changes/v0.1.8.md new file mode 100644 index 00000000..2a6d506e --- /dev/null +++ b/.changes/v0.1.8.md @@ -0,0 +1,11 @@ +## [v0.1.8](https://pypi.org/project/fabric-cicd/0.1.8) - March 04, 2025 + +### ๐Ÿ”ง Bug Fix + +- Handle null byPath object in report definition file ([#143](https://github.com/microsoft/fabric-cicd/issues/143)) +- Support relative directories ([#136](https://github.com/microsoft/fabric-cicd/issues/136)) ([#132](https://github.com/microsoft/fabric-cicd/issues/132)) +- Increase special character support ([#134](https://github.com/microsoft/fabric-cicd/issues/134)) + +### โšก Additional Optimizations + +- Changelog now available with version check ([#127](https://github.com/microsoft/fabric-cicd/issues/127)) diff --git a/.changes/v0.1.9.md b/.changes/v0.1.9.md new file mode 100644 index 00000000..b26d035c --- /dev/null +++ b/.changes/v0.1.9.md @@ -0,0 +1,9 @@ +## [v0.1.9](https://pypi.org/project/fabric-cicd/0.1.9) - March 11, 2025 + +### ๐Ÿ†• New Items Support + +- Support for Mirrored Database item type ([#145](https://github.com/microsoft/fabric-cicd/issues/145)) + +### โšก Additional Optimizations + +- Increase reserved name wait time ([#135](https://github.com/microsoft/fabric-cicd/issues/135)) diff --git a/.changie.yaml b/.changie.yaml new file mode 100644 index 00000000..bbc61efc --- /dev/null +++ b/.changie.yaml @@ -0,0 +1,54 @@ +# docs: https://changie.dev/config/ +--- +changesDir: .changes +unreleasedDir: unreleased +headerPath: header.tpl.md +versionExt: md +changelogPath: docs/changelog.md +versionFormat: '## [{{.Version}}](https://pypi.org/project/fabric-cicd/{{.Version}}) - {{.Time.Format "January 02, 2006"}}' +kindFormat: "### {{.Kind}}" +changeFormat: "* {{.Body}} by [{{.Custom.Author}}]({{.Custom.AuthorLink}}) ([#{{.Custom.Issue}}]({{.Custom.IssueLink}}))" +kinds: + - label: "โš ๏ธ Breaking Change" + key: breaking + auto: major + - label: "๐Ÿ†• New Items Support" + key: new-items + auto: minor + - label: "โœจ New Functionality" + key: added + auto: minor + - label: "๐Ÿ”ง Bug Fix" + key: fixed + auto: patch + - label: "โšก Additional Optimizations" + key: optimization + auto: patch + - label: "๐Ÿ“ Documentation Update" + key: docs + auto: patch +newlines: + afterChangelogHeader: 0 + beforeChangelogVersion: 1 + endOfVersion: 1 + afterKind: 1 + beforeKind: 1 +envPrefix: CHANGIE_ +custom: + - key: Issue + label: Issue Number + type: int + minLength: 1 + - key: Author + label: Author's GitHub Username + type: string + minLength: 3 +post: + - key: AuthorLink + value: "https://github.com/{{.Custom.Author}}" + - key: IssueLink + value: "https://github.com/microsoft/fabric-cicd/issues/{{.Custom.Issue}}" +replacements: + - path: src/fabric_cicd/constants.py + find: 'VERSION = ".*"' + replace: 'VERSION = "{{.VersionNoPrefix}}"' diff --git a/.github/workflows/bump.yml b/.github/workflows/bump.yml index a6d2f44a..317317ef 100644 --- a/.github/workflows/bump.yml +++ b/.github/workflows/bump.yml @@ -178,7 +178,7 @@ jobs: // Extract changelog content for new release const versionNumber = '${{ steps.version-check.outputs.version_number }}'; const fs = require('fs'); - const path = 'src/fabric_cicd/changelog.md'; + const path = 'docs/changelog.md'; let changelogContent = `Release ${tagName}`; @@ -190,9 +190,9 @@ jobs: let content = []; for (const line of lines) { - if (line.startsWith('## Version ')) { + if (line.startsWith('## [v')) { if (found) break; // Hit next version, stop - if (line.includes(`Version ${versionNumber}`)) { + if (line.includes(`[v${versionNumber}]`)) { found = true; continue; // Skip the version header line } diff --git a/.github/workflows/changelog.yml b/.github/workflows/changelog.yml new file mode 100644 index 00000000..9a9b0fc5 --- /dev/null +++ b/.github/workflows/changelog.yml @@ -0,0 +1,97 @@ +# yaml-language-server: $schema=https://json.schemastore.org/github-workflow.json +--- +name: ๐Ÿ”„ Changelog + +on: + pull_request: + branches: + - main + types: + - opened + - reopened + - labeled + - unlabeled + - synchronize + workflow_dispatch: + +concurrency: + group: ${{ format('{0}-{1}-{2}-{3}-{4}', github.workflow, github.event_name, github.ref, github.base_ref || null, github.head_ref || null) }} + cancel-in-progress: true + +permissions: + contents: read + +jobs: + changelog-existence: + name: ๐Ÿ”„ Check Changelog + if: ${{ !contains(github.event.pull_request.labels.*.name, 'skip changelog') && github.actor != 'dependabot[bot]' }} + runs-on: ubuntu-latest + env: + BASE_SHA: ${{ github.event.pull_request.base.sha }} + HEAD_SHA: ${{ github.event.pull_request.head.sha }} + steps: + - name: โคต๏ธ Checkout + uses: actions/checkout@v4 + with: + fetch-depth: 0 # Required to access full commit history + + - name: โœ”๏ธ Check for changelog changes + id: changelog_check + uses: actions/github-script@v6 + with: + script: | + const { execSync } = require('child_process'); + const base = process.env.BASE_SHA; + const head = process.env.HEAD_SHA; + console.log(`Comparing changes from ${base} to ${head}`) + const output = execSync(`git diff --name-only --no-renames --diff-filter=AM ${base} ${head}`).toString(); + const files = output.split('\n').filter(Boolean); + const changelogExists = files.some(file => file.startsWith('.changes/unreleased/') && file.endsWith('.yaml')); + core.setOutput('exists', changelogExists); + + - name: ๐Ÿšง Setup Node + if: steps.changelog_check.outputs.exists == 'true' + uses: actions/setup-node@v6 + with: + node-version: "20" + + - name: ๐Ÿšง Install Changie + if: steps.changelog_check.outputs.exists == 'true' + run: npm i -g changie + + - name: ๐Ÿ”„ Prepare comment (changelog) + if: steps.changelog_check.outputs.exists == 'true' + run: | + echo -e "# Changelog Preview\n" > changie.md + changie batch patch --dry-run --prerelease 'dev' >> changie.md + cat changie.md >> $GITHUB_STEP_SUMMARY + + - name: ๐Ÿ”„ Prepare comment (missing) + if: steps.changelog_check.outputs.exists == 'false' + run: | + echo -e "# ๐Ÿ›‘ Changelog entry required to merge\n" > changie.md + echo "Run \`changie new\` to add a new changelog entry" >> changie.md + cat changie.md >> $GITHUB_STEP_SUMMARY + + - name: โœ… Pass if changelog entry exists + if: steps.changelog_check.outputs.exists == 'true' + run: | + echo "โœ… Changelog entry exists." + exit 0 + + - name: ๐Ÿ›‘ Fail if changelog entry is missing and required + if: steps.changelog_check.outputs.exists == 'false' + run: | + echo "๐Ÿ›‘ Changelog entry required to merge." + echo "๐Ÿ›‘ Please run 'changie new' to add a new changelog entry." + exit 1 + + changelog-skip: + name: โญ๏ธ Skip Changelog + if: ${{ contains(github.event.pull_request.labels.*.name, 'skip changelog') || github.actor == 'dependabot[bot]' }} + runs-on: ubuntu-latest + steps: + - name: โœ… Pass (skip) + run: | + echo "โญ๏ธ Changelog check skipped." >> $GITHUB_STEP_SUMMARY + exit 0 diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml index 59bb0024..1addfb5c 100644 --- a/.github/workflows/validate.yml +++ b/.github/workflows/validate.yml @@ -120,7 +120,7 @@ jobs: # 2. If title is version format, only constants.py and changelog.md should be changed if [[ "$PR_TITLE" =~ $VERSION_REGEX ]]; then - EXPECTED_FILES="src/fabric_cicd/constants.py src/fabric_cicd/changelog.md" + EXPECTED_FILES="src/fabric_cicd/constants.py docs/changelog.md" SORTED_CHANGED=$(echo $CHANGED_FILES | tr ' ' '\n' | sort | tr '\n' ' ') SORTED_EXPECTED=$(echo $EXPECTED_FILES | tr ' ' '\n' | sort | tr '\n' ' ') if [ "$SORTED_CHANGED" != "$SORTED_EXPECTED" ]; then diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c3212ce4..b7cd74d9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -30,6 +30,7 @@ We welcome several types of contributions: Before you begin, ensure you have the following installed: - [Python](https://www.python.org/downloads/) (see [Installation](https://microsoft.github.io/fabric-cicd/#installation) for version requirements) +- [Node.js and npm](https://nodejs.org/en/download/) - [PowerShell](https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell) - [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-windows) or [Az.Accounts PowerShell module](https://www.powershellgallery.com/packages/Az.Accounts/2.2.3) - [Visual Studio Code (VS Code)](https://code.visualstudio.com/) @@ -169,6 +170,51 @@ Verify that: - PRs will be labeled as "need author feedback" when there are comments to resolve - Approved PRs will be merged by the fabric-cicd team +### Documenting Changes with Changie + +All pull requests must include proper change documentation using [changie](https://changie.dev), which is pre-installed in the development container. This ensures that release notes are automatically generated and changes are properly tracked. + +#### Requirements + +**Every PR must include at least one change entry** created using `changie new`. You may add multiple entries if your PR introduces multiple distinct changes. + +#### How to Add Change Entries + +1. **From the Terminal, run `changie new` command**: + + ```bash + changie new + ``` + +2. **Select the appropriate change type** from the available options: + - **โš ๏ธ Breaking Change** - For changes that break backward compatibility + - **๐Ÿ†• New Items Support** - For adding support for new Fabric item types + - **โœจ New Functionality** - For new features, commands, or capabilities + - **๐Ÿ”ง Bug Fix** - For fixing existing issues or incorrect behavior + - **โšก Additional Optimizations** - For performance improvements or optimizations + - **๐Ÿ“ Documentation Update** - For documentation improvements or updates + +3. **Provide a clear description** of your change: + - Write in present tense (e.g., "Add support for..." not "Added support for...") + - Be specific and user-focused + - Include the affected command or feature if applicable + - Keep it concise but informative + +#### Examples of Good Change Descriptions + +- `Fix timeout issue in LRO polling` +- `Update workspace examples with new folder hierarchy patterns` +- `Optimize API response caching to reduce network calls` + +#### Guidelines + +- **One logical change per entry**: If your PR fixes a bug and adds a feature, create two separate entries +- **User-facing perspective**: Describe what users will experience, not internal implementation details +- **Clear and actionable**: Users should understand what changed and how it affects them +- **Consistent formatting**: Follow the examples and existing patterns in the changelog + +The change entries will be automatically included in the release notes when a new version is published. This process ensures that all improvements, fixes, and new features are properly communicated to users. + ## Resources to help you get started Here are some resources to help you get started: diff --git a/activate.ps1 b/activate.ps1 index f26c317c..63833d75 100644 --- a/activate.ps1 +++ b/activate.ps1 @@ -3,19 +3,21 @@ <# .SYNOPSIS -Script to check and install required Python packages, add directories to PATH, and activate a virtual environment. +Script to check and install required Python packages, Node.js tools, add directories to PATH, and activate a virtual environment. .DESCRIPTION This script performs the following tasks: 1. Checks if Python is installed. 2. Checks if pip is installed. -3. Checks and installs specified Python packages if they are not already installed. -4. Adds a specified directory to the system PATH if it is not already included. -5. Ensures the 'uv' command is available in the PATH. -6. Activates a virtual environment using 'uv'. +3. Checks if Node.js and npm are installed. +4. Checks and installs specified Python packages if they are not already installed. +5. Installs changie globally via npm if not already installed. +6. Adds a specified directory to the system PATH if it is not already included. +7. Ensures the 'uv' command is available in the PATH. +8. Activates a virtual environment using 'uv'. .NOTES -Make sure that Python and pip are installed and available in the system PATH. +Make sure that Python, pip, Node.js, and npm are installed and available in the system PATH. #> # Function to check if a dependency is available @@ -37,7 +39,7 @@ function Test-Dependancy { } # Function to install required packages if not already installed -function Test-And-Install-Package { +function Test-And-Install-Python-Package { param ( [string]$packageName ) @@ -59,6 +61,34 @@ function Test-And-Install-Package { } } +# Function to install changie globally via npm if not already installed +function Test-And-Install-Changie { + if (-not (Get-Command changie -ErrorAction SilentlyContinue)) { + Write-Host "changie not found, installing globally via npm..." + try { + npm install -g changie --registry https://registry.npmjs.org/ + + # Add npm global bin to PATH if needed + $npmGlobalPath = npm config get prefix + if ($npmGlobalPath) { + Add-DirectoryToPath -directory $npmGlobalPath + } + + # Refresh PATH for the current session + $env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User") + + Test-Dependancy -commandName "changie" + } + catch { + Write-Host "Failed to install changie via npm. Please check your npm installation and connection." + exit 1 + } + } + else { + Write-Host "changie is already installed." + } +} + # Function to add a directory to PATH function Add-DirectoryToPath { param ( @@ -74,10 +104,13 @@ function Add-DirectoryToPath { # Check if dependencies are installed and add directory to PATH Test-Dependancy -commandName "python" Test-Dependancy -commandName "pip" - +Test-Dependancy -commandName "node" +Test-Dependancy -commandName "npm" # Check and install required packages -Test-And-Install-Package -packageName "uv" -Test-And-Install-Package -packageName "ruff" +Test-And-Install-Python-Package -packageName "uv" +Test-And-Install-Python-Package -packageName "ruff" +Test-And-Install-Changie + # uv fallback to default path if unavailable in python directory if (-not (Get-Command uv -ErrorAction SilentlyContinue)) { diff --git a/activate.sh b/activate.sh index a4276ae4..738e45a5 100755 --- a/activate.sh +++ b/activate.sh @@ -1,7 +1,7 @@ #!/bin/bash # # -# Script to check and install required Python packages, +# Script to check and install required Python packages, Node.js tools, # add directories to PATH, and activate a virtual environment. # # --------------------------------------------------------------------------------------- @@ -9,17 +9,102 @@ set -e PACKAGES="" -if ! command -v python &> /dev/null; then PACKAGES="python3"; fi +if ! command -v python3.11 &> /dev/null; then PACKAGES="python3.11"; fi if ! command -v pip &> /dev/null; then PACKAGES="${PACKAGES:+$PACKAGES }python3-pip"; fi -if [ -n "$PACKAGES" ]; then - sudo apt-get update > /dev/null 2>&1 - sudo DEBIAN_FRONTEND=noninteractive apt-get install -y $PACKAGES > /dev/null 2>&1 +if ! command -v node &> /dev/null; then PACKAGES="${PACKAGES:+$PACKAGES }nodejs"; fi +if ! command -v npm &> /dev/null; then PACKAGES="${PACKAGES:+$PACKAGES }npm"; fi +if [[ "$OSTYPE" == "linux-gnu"* ]]; then + if [ -n "$PACKAGES" ]; then + echo "Installing required packages for Linux: $PACKAGES" + sudo apt-get update > /dev/null 2>&1 + if sudo DEBIAN_FRONTEND=noninteractive apt-get install -y $PACKAGES > /dev/null 2>&1; then + echo "Packages installed successfully." + else + echo "Failed to install packages." + exit 1 + fi + fi +elif [[ "$OSTYPE" == "darwin"* ]]; then + if ! command -v brew &> /dev/null; then + echo "Homebrew not found. Please install Homebrew first." + exit 1 + fi + + BREW_PACKAGES="" + if ! command -v python3.11 &> /dev/null; then BREW_PACKAGES="python@3.11"; fi + if ! command -v node &> /dev/null; then BREW_PACKAGES="${BREW_PACKAGES:+$BREW_PACKAGES }node"; fi + + if [ -n "$BREW_PACKAGES" ]; then + echo "Installing required packages for macOS: $BREW_PACKAGES" + if brew install $BREW_PACKAGES; then + echo "Packages installed successfully." + else + echo "Failed to install packages." + exit 1 + fi + fi +fi + +# Install uv if not present +if ! command -v uv &> /dev/null; then + echo "Installing uv..." + if curl -LsSf https://astral.sh/uv/install.sh | sh; then + echo "uv installed successfully." + else + echo "Failed to install uv." + exit 1 + fi +else + echo "uv is already installed." +fi + +# Install changie globally via npm if not present +if ! command -v changie &> /dev/null; then + echo "Installing changie globally via npm..." + if npm install -g changie; then + echo "changie installed successfully." + else + echo "Failed to install changie." + exit 1 + fi +else + echo "changie is already installed." +fi + +# Install VS Code Python extension if VS Code is available +if command -v code &> /dev/null; then + echo "Installing VS Code Python extension..." + if code --install-extension ms-python.python --force > /dev/null 2>&1; then + echo "VS Code Python extension installed successfully." + else + echo "Failed to install VS Code Python extension." + fi +else + echo "VS Code not found, skipping extension installation." +fi + +# Add required directories to PATH +if [[ ":$PATH:" != *":$HOME/.local/bin:"* ]]; then + export PATH="$PATH:$HOME/.local/bin" + echo "Added $HOME/.local/bin to PATH." +fi +if [[ ":$PATH:" != *":$HOME/.cargo/bin:"* ]]; then + export PATH="$PATH:$HOME/.cargo/bin" + echo "Added $HOME/.cargo/bin to PATH." fi -command -v uv &> /dev/null || curl -LsSf https://astral.sh/uv/install.sh | sh -command -v code &> /dev/null && code --install-extension ms-python.python --force > /dev/null 2>&1 || true -[[ ":$PATH:" != *":$HOME/.local/bin:"* ]] && export PATH="$PATH:$HOME/.local/bin" -[[ ":$PATH:" != *":$HOME/.cargo/bin:"* ]] && export PATH="$PATH:$HOME/.cargo/bin" +# Sync Python environment and activate +echo "Syncing Python environment with uv..." +if uv sync --python 3.11; then + echo "Python environment synced successfully." +else + echo "Failed to sync Python environment." + exit 1 +fi -uv sync --python 3.11 -[ -f .venv/bin/activate ] && source .venv/bin/activate +if [ -f .venv/bin/activate ]; then + source .venv/bin/activate + echo "Virtual environment activated." +else + echo "Virtual environment not found." +fi diff --git a/docs/changelog.md b/docs/changelog.md index 9a287f4b..bd388efb 100644 --- a/docs/changelog.md +++ b/docs/changelog.md @@ -1,5 +1,534 @@ # Changelog -{% -include "../src/fabric_cicd/changelog.md" -%} +## [v0.1.34](https://pypi.org/project/fabric-cicd/0.1.34) - January 20, 2026 + +### โœจ New Functionality + +- Enable dynamic replacement of SQL endpoint values from SQL Database items ([#720](https://github.com/microsoft/fabric-cicd/issues/720)) +- Support Fabric Notebook Authentication ([#707](https://github.com/microsoft/fabric-cicd/issues/707)) + +### ๐Ÿ†• New Items Support + +- Onboard Spark Job Definition item type ([#115](https://github.com/microsoft/fabric-cicd/issues/115)) + +### ๐Ÿ“ Documentation Update + +- Add `CONTRIBUTING.md` file to repository ([#723](https://github.com/microsoft/fabric-cicd/issues/723)) +- Add comprehensive troubleshooting guide to documentation ([#705](https://github.com/microsoft/fabric-cicd/issues/705)) +- Add parameterization documentation for Report items using ByConnection binding to Semantic Models ([#637](https://github.com/microsoft/fabric-cicd/issues/637)) + +### โšก Additional Optimizations + +- Add debug file for local Fabric REST API testing ([#714](https://github.com/microsoft/fabric-cicd/issues/714)) + +## [v0.1.33](https://pypi.org/project/fabric-cicd/0.1.33) - December 16, 2025 + +### โœจ New Functionality + +- Add key_value_replace parameter support for YAML files ([#649](https://github.com/microsoft/fabric-cicd/issues/649)) +- Support selective shortcut publishing with regex exclusion ([#624](https://github.com/microsoft/fabric-cicd/issues/624)) + +### โšก Additional Optimizations + +- Add Linux development environment bootstrapping script ([#680](https://github.com/microsoft/fabric-cicd/issues/680)) +- Update item types in scope to be an optional parameter in validate parameter file function ([#669](https://github.com/microsoft/fabric-cicd/issues/669)) + +### ๐Ÿ”ง Bug Fix + +- Fix publish order for Notebook and Eventhouse dependent items ([#685](https://github.com/microsoft/fabric-cicd/issues/685)) +- Enable parameterizing multiple connections in the same Semantic Model item ([#674](https://github.com/microsoft/fabric-cicd/issues/674)) +- Fix missing description metadata in item payload for shell-only item deployments ([#672](https://github.com/microsoft/fabric-cicd/issues/672)) +- Resolve API long running operation handling when publishing Environment items ([#668](https://github.com/microsoft/fabric-cicd/issues/668)) + +## [v0.1.32](https://pypi.org/project/fabric-cicd/0.1.32) - December 03, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix publish bug for Environment items that contain only spark settings ([#664](https://github.com/microsoft/fabric-cicd/issues/664)) + +## [v0.1.31](https://pypi.org/project/fabric-cicd/0.1.31) - December 01, 2025 + +### โš ๏ธ Breaking Change + +- Migrate to the latest Fabric Environment item APIs to simplify deployment and improve compatibility ([#173](https://github.com/microsoft/fabric-cicd/issues/173)) + +### โœจ New Functionality + +- Enable dynamic replacement of Lakehouse SQL Endpoint IDs ([#616](https://github.com/microsoft/fabric-cicd/issues/616)) +- Enable linking of Semantic Models to both cloud and gateway connections ([#602](https://github.com/microsoft/fabric-cicd/issues/602)) +- Allow use of the dynamic replacement variables within the key_value_replace parameter ([#567](https://github.com/microsoft/fabric-cicd/issues/567)) +- Add support for parameter file templates ([#499](https://github.com/microsoft/fabric-cicd/issues/499)) + +### ๐Ÿ†• New Items Support + +- Add support for the ML Experiment item type ([#600](https://github.com/microsoft/fabric-cicd/issues/600)) +- Add support for the User Data Function item type ([#588](https://github.com/microsoft/fabric-cicd/issues/588)) + +### ๐Ÿ“ Documentation Update + +- Update the advanced Dataflow parameterization example with the correct file_path value ([#633](https://github.com/microsoft/fabric-cicd/issues/633)) + +### ๐Ÿ”ง Bug Fix + +- Fix publishing issues for KQL Database items in folders ([#657](https://github.com/microsoft/fabric-cicd/issues/657)) +- Separate logic for 'items to include' feature between publish and unpublish operations ([#650](https://github.com/microsoft/fabric-cicd/issues/650)) +- Fix parameterization logic to properly handle find_value regex patterns and replacements ([#639](https://github.com/microsoft/fabric-cicd/issues/639)) +- Correct the publish order of Data Agent and Semantic Model items ([#628](https://github.com/microsoft/fabric-cicd/issues/628)) +- Fix Lakehouse item publishing errors when shortcuts refer to the default Lakehouse ID ([#610](https://github.com/microsoft/fabric-cicd/issues/610)) + +## [v0.1.30](https://pypi.org/project/fabric-cicd/0.1.30) - October 20, 2025 + +### โœจ New Functionality + +- Add support for binding semantic models to on-premise gateways in Fabric workspaces ([#569](https://github.com/microsoft/fabric-cicd/issues/569)) + +### ๐Ÿ†• New Items Support + +- Add support for publishing and managing Data Agent items ([#556](https://github.com/microsoft/fabric-cicd/issues/556)) +- Add OrgApp item type support ([#586](https://github.com/microsoft/fabric-cicd/issues/586)) + +### โšก Additional Optimizations + +- Enhance cross-workspace variable support to allow referencing other attributes ([#583](https://github.com/microsoft/fabric-cicd/issues/583)) + +### ๐Ÿ”ง Bug Fix + +- Fix workspace name extraction bug for non-ID attrs using ITEM_ATTR_LOOKUP ([#583](https://github.com/microsoft/fabric-cicd/issues/583)) +- Fix capacity requirement check ([#593](https://github.com/microsoft/fabric-cicd/issues/593)) + +## [v0.1.29](https://pypi.org/project/fabric-cicd/0.1.29) - October 01, 2025 + +### โœจ New Functionality + +- Support dynamic replacement for cross-workspace item IDs ([#558](https://github.com/microsoft/fabric-cicd/issues/558)) +- Add option to return API response for publish operations in publish_all_items ([#497](https://github.com/microsoft/fabric-cicd/issues/497)) + +### ๐Ÿ†• New Items Support + +- Onboard Apache Airflow Job item type ([#565](https://github.com/microsoft/fabric-cicd/issues/565)) +- Onboard Mounted Data Factory item type ([#406](https://github.com/microsoft/fabric-cicd/issues/406)) + +### ๐Ÿ”ง Bug Fix + +- Fix publish order of Eventhouses and Semantic Models ([#566](https://github.com/microsoft/fabric-cicd/issues/566)) + +## [v0.1.28](https://pypi.org/project/fabric-cicd/0.1.28) - September 15, 2025 + +### โœจ New Functionality + +- Add folder exclusion feature for publish operations ([#427](https://github.com/microsoft/fabric-cicd/issues/427)) +- Expand workspace ID dynamic replacement capabilities in parameterization ([#408](https://github.com/microsoft/fabric-cicd/issues/408)) + +### ๐Ÿ”ง Bug Fix + +- Fix unexpected behavior with file_path parameter filter ([#545](https://github.com/microsoft/fabric-cicd/issues/545)) +- Fix unpublish exclude_regex bug in configuration file-based deployment ([#544](https://github.com/microsoft/fabric-cicd/issues/544)) + +## [v0.1.27](https://pypi.org/project/fabric-cicd/0.1.27) - September 05, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix traliing comma in report schema ([#534](https://github.com/microsoft/fabric-cicd/issues/534)) + +## [v0.1.26](https://pypi.org/project/fabric-cicd/0.1.26) - September 05, 2025 + +### โš ๏ธ Breaking Change + +- Deprecate Base API URL kwarg in Fabric Workspace ([#529](https://github.com/microsoft/fabric-cicd/issues/529)) + +### โœจ New Functionality + +- Support Schedules parameterization ([#508](https://github.com/microsoft/fabric-cicd/issues/508)) +- Support YAML configuration file-based deployment ([#470](https://github.com/microsoft/fabric-cicd/issues/470)) + +### ๐Ÿ“ Documentation Update + +- Add dynamically generated Python version requirements to documentation ([#520](https://github.com/microsoft/fabric-cicd/issues/520)) + +### โšก Additional Optimizations + +- Enhance pytest output to limit console verbosity ([#514](https://github.com/microsoft/fabric-cicd/issues/514)) + +### ๐Ÿ”ง Bug Fix + +- Fix Report item schema handling ([#518](https://github.com/microsoft/fabric-cicd/issues/518)) +- Fix deployment order to publish Mirrored Database before Lakehouse ([#482](https://github.com/microsoft/fabric-cicd/issues/482)) + +## [v0.1.25](https://pypi.org/project/fabric-cicd/0.1.25) - August 19, 2025 + +### โš ๏ธ Breaking Change + +- Modify the default for item_types_in_scope and add thorough validation ([#464](https://github.com/microsoft/fabric-cicd/issues/464)) + +### โœจ New Functionality + +- Add new experimental feature flag to enable selective deployment ([#384](https://github.com/microsoft/fabric-cicd/issues/384)) +- Support "ALL" environment concept in parameterization ([#320](https://github.com/microsoft/fabric-cicd/issues/320)) + +### ๐Ÿ“ Documentation Update + +- Enhance Overview section in Parameterization docs ([#495](https://github.com/microsoft/fabric-cicd/issues/495)) + +### โšก Additional Optimizations + +- Eliminate ACCEPTED_ITEM_TYPES_NON_UPN constant and unify with ACCEPTED_ITEM_TYPES ([#477](https://github.com/microsoft/fabric-cicd/issues/477)) +- Add comprehensive GitHub Copilot instructions for effective codebase development ([#468](https://github.com/microsoft/fabric-cicd/issues/468)) + +### ๐Ÿ”ง Bug Fix + +- Add feature flags and warnings for Warehouse, SQL Database, and Eventhouse unpublish operations ([#483](https://github.com/microsoft/fabric-cicd/issues/483)) +- Fix code formatting inconsistencies in fabric_workspace unit test ([#474](https://github.com/microsoft/fabric-cicd/issues/474)) +- Fix KeyError when deploying Reports with Semantic Model dependencies in Report-only scope case ([#278](https://github.com/microsoft/fabric-cicd/issues/278)) + +## [v0.1.24](https://pypi.org/project/fabric-cicd/0.1.24) - August 04, 2025 + +### โš ๏ธ Breaking Change + +- Require parameterization for Dataflow and Semantic Model references in Data Pipeline activities +- Require specific parameterization for deploying a Dataflow that depends on another in the same workspace (see Parameterization docs) + +### ๐Ÿ“ Documentation Update + +- Improve Parameterization documentation ([#415](https://github.com/microsoft/fabric-cicd/issues/415)) + +### โšก Additional Optimizations + +- Support for Eventhouse query URI parameterization ([#414](https://github.com/microsoft/fabric-cicd/issues/414)) +- Support for Warehouse SQL endpoint parameterization ([#392](https://github.com/microsoft/fabric-cicd/issues/392)) + +### ๐Ÿ”ง Bug Fix + +- Fix Dataflow/Data Pipeline deployment failures caused by workspace permissions ([#419](https://github.com/microsoft/fabric-cicd/issues/419)) +- Prevent duplicate logical ID issue in Report and Semantic Model deployment ([#405](https://github.com/microsoft/fabric-cicd/issues/405)) +- Fix deployment of items without assigned capacity ([#402](https://github.com/microsoft/fabric-cicd/issues/402)) + +## [v0.1.23](https://pypi.org/project/fabric-cicd/0.1.23) - July 08, 2025 + +### โœจ New Functionality + +- New functionalities for GitHub Copilot Agent and PR-to-Issue linking + +### ๐Ÿ“ Documentation Update + +- Fix formatting and examples in the How to and Examples pages + +### ๐Ÿ”ง Bug Fix + +- Fix issue with lakehouse shortcuts publishing ([#379] (https://github.com/microsoft/fabric-cicd/issues/379)) +- Add validation for empty logical IDs to prevent deployment corruption ([#86](https://github.com/microsoft/fabric-cicd/issues/86)) +- Fix SQL provision print statement ([#329](https://github.com/microsoft/fabric-cicd/issues/329)) +- Rename the error code for reserved item name per updated Microsoft Fabric API ([#388](https://github.com/microsoft/fabric-cicd/issues/388)) +- Fix lakehouse exclude_regex to exclude shortcut publishing ([#385](https://github.com/microsoft/fabric-cicd/issues/385)) +- Remove max retry limit to handle large deployments ([#299] (https://github.com/microsoft/fabric-cicd/issues/299)) + +## [v0.1.22](https://pypi.org/project/fabric-cicd/0.1.22) - June 25, 2025 + +### ๐Ÿ†• New Items Support + +- Onboard API for GraphQL item type ([#287](https://github.com/microsoft/fabric-cicd/issues/287)) + +### ๐Ÿ”ง Bug Fix + +- Fix Fabric API call error during dataflow publish ([#352](https://github.com/microsoft/fabric-cicd/issues/352)) + +### โšก Additional Optimizations + +- Expanded test coverage to handle folder edge cases ([#358](https://github.com/microsoft/fabric-cicd/issues/358)) + +## [v0.1.21](https://pypi.org/project/fabric-cicd/0.1.21) - June 18, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix bug with workspace ID replacement in JSON files for pipeline deployments ([#345](https://github.com/microsoft/fabric-cicd/issues/345)) + +### โšก Additional Optimizations + +- Increased max retry for Warehouses and Dataflows + +## [v0.1.20](https://pypi.org/project/fabric-cicd/0.1.20) - June 12, 2025 + +### โœจ New Functionality + +- Parameterization support for find_value regex and replace_value variables ([#326](https://github.com/microsoft/fabric-cicd/issues/326)) + +### ๐Ÿ†• New Items Support + +- Onboard KQL Dashboard item type ([#329](https://github.com/microsoft/fabric-cicd/issues/329)) +- Onboard Dataflow Gen2 item type ([#111](https://github.com/microsoft/fabric-cicd/issues/111)) + +### ๐Ÿ”ง Bug Fix + +- Fix bug with deploying environment libraries with special chars ([#336](https://github.com/microsoft/fabric-cicd/issues/336)) + +### โšก Additional Optimizations + +- Improved test coverage for subfolder creation/modification ([#211](https://github.com/microsoft/fabric-cicd/issues/211)) + +## [v0.1.19](https://pypi.org/project/fabric-cicd/0.1.19) - May 21, 2025 + +### ๐Ÿ†• New Items Support + +- Onboard SQL Database item type (shell-only deployment) ([#301](https://github.com/microsoft/fabric-cicd/issues/301)) +- Onboard Warehouse item type (shell-only deployment) ([#204](https://github.com/microsoft/fabric-cicd/issues/204)) + +### ๐Ÿ”ง Bug Fix + +- Fix bug with unpublish workspace folders ([#273](https://github.com/microsoft/fabric-cicd/issues/273)) + +## [v0.1.18](https://pypi.org/project/fabric-cicd/0.1.18) - May 14, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix bug with check environment publish state ([#295](https://github.com/microsoft/fabric-cicd/issues/295)) + +## [v0.1.17](https://pypi.org/project/fabric-cicd/0.1.17) - May 13, 2025 + +### โš ๏ธ Breaking Change + +- Deprecate old parameter file structure ([#283](https://github.com/microsoft/fabric-cicd/issues/283)) + +### ๐Ÿ†• New Items Support + +- Onboard CopyJob item type ([#122](https://github.com/microsoft/fabric-cicd/issues/122)) +- Onboard Eventstream item type ([#170](https://github.com/microsoft/fabric-cicd/issues/170)) +- Onboard Eventhouse/KQL Database item type ([#169](https://github.com/microsoft/fabric-cicd/issues/169)) +- Onboard Data Activator item type ([#291](https://github.com/microsoft/fabric-cicd/issues/291)) +- Onboard KQL Queryset item type ([#292](https://github.com/microsoft/fabric-cicd/issues/292)) + +### ๐Ÿ”ง Bug Fix + +- Fix post publish operations for skipped items ([#277](https://github.com/microsoft/fabric-cicd/issues/277)) + +### โšก Additional Optimizations + +- New function `key_value_replace` for key-based replacement operations in JSON and YAML + +### ๐Ÿ“ Documentation Update + +- Add publish regex example to demonstrate how to use the `publish_all_items` with regex for excluding item names + +## [v0.1.16](https://pypi.org/project/fabric-cicd/0.1.16) - April 25, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix bug with folder deployment to root ([#255](https://github.com/microsoft/fabric-cicd/issues/255)) + +### โšก Additional Optimizations + +- Add Workspace Name in FabricWorkspaceObject ([#200](https://github.com/microsoft/fabric-cicd/issues/200)) +- New function to check SQL endpoint provision status ([#226](https://github.com/microsoft/fabric-cicd/issues/226)) + +### ๐Ÿ“ Documentation Update + +- Updated Authentication docs + menu sort order + +## [v0.1.15](https://pypi.org/project/fabric-cicd/0.1.15) - April 21, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix folders moving with every publish ([#236](https://github.com/microsoft/fabric-cicd/issues/236)) + +### โšก Additional Optimizations + +- Introduce parallel deployments to reduce publish times ([#237](https://github.com/microsoft/fabric-cicd/issues/237)) +- Improvements to check version logic + +### ๐Ÿ“ Documentation Update + +- Updated Examples section in docs + +## [v0.1.14](https://pypi.org/project/fabric-cicd/0.1.14) - April 09, 2025 + +### โœจ New Functionality + +- Optimized & beautified terminal output +- Added changelog to output of old version check + +### ๐Ÿ”ง Bug Fix + +- Fix workspace folder deployments in root folder ([#221](https://github.com/microsoft/fabric-cicd/issues/221)) +- Fix unpublish of workspace folders without publish ([#222](https://github.com/microsoft/fabric-cicd/issues/222)) + +### โšก Additional Optimizations + +- Removed Colorama and Colorlog Dependency + +## [v0.1.13](https://pypi.org/project/fabric-cicd/0.1.13) - April 07, 2025 + +### โœจ New Functionality + +- Added support for Lakehouse Shortcuts +- New `enable_environment_variable_replacement` feature flag ([#160](https://github.com/microsoft/fabric-cicd/issues/160)) + +### ๐Ÿ†• New Items Support + +- Onboard Workspace Folders ([#81](https://github.com/microsoft/fabric-cicd/issues/81)) +- Onboard Variable Library item type ([#206](https://github.com/microsoft/fabric-cicd/issues/206)) + +### โšก Additional Optimizations + +- User-agent now available in API headers ([#207](https://github.com/microsoft/fabric-cicd/issues/207)) +- Fixed error log typo in fabric_endpoint + +### ๐Ÿ”ง Bug Fix + +- Fix break with invalid optional parameters ([#192](https://github.com/microsoft/fabric-cicd/issues/192)) +- Fix bug where all workspace ids were not being replaced by parameterization ([#186](https://github.com/microsoft/fabric-cicd/issues/186)) + +## [v0.1.12](https://pypi.org/project/fabric-cicd/0.1.12) - March 27, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix constant overwrite failures ([#190](https://github.com/microsoft/fabric-cicd/issues/190)) +- Fix bug where all workspace ids were not being replaced ([#186](https://github.com/microsoft/fabric-cicd/issues/186)) +- Fix type hints for older versions of Python ([#156](https://github.com/microsoft/fabric-cicd/issues/156)) +- Fix accepted item types constant in pre-build + +## [v0.1.11](https://pypi.org/project/fabric-cicd/0.1.11) - March 25, 2025 + +### โš ๏ธ Breaking Change + +- Parameterization refactor introducing a new parameter file structure and parameter file validation functionality ([#113](https://github.com/microsoft/fabric-cicd/issues/113)) + +### โœจ New Functionality + +- Support regex for publish exclusion ([#121](https://github.com/microsoft/fabric-cicd/issues/121)) +- Override max retries via constants ([#146](https://github.com/microsoft/fabric-cicd/issues/146)) + +### ๐Ÿ“ Documentation Update + +- Update to [parameterization](https://microsoft.github.io/fabric-cicd/latest/how_to/parameterization/) docs + +## [v0.1.10](https://pypi.org/project/fabric-cicd/0.1.10) - March 19, 2025 + +### โœจ New Functionality + +- DataPipeline SPN Support ([#133](https://github.com/microsoft/fabric-cicd/issues/133)) + +### ๐Ÿ”ง Bug Fix + +- Workspace ID replacement in data pipelines ([#164](https://github.com/microsoft/fabric-cicd/issues/164)) + +### ๐Ÿ“ Documentation Update + +- Sample for passing in arguments from Azure DevOps Pipelines + +## [v0.1.9](https://pypi.org/project/fabric-cicd/0.1.9) - March 11, 2025 + +### ๐Ÿ†• New Items Support + +- Support for Mirrored Database item type ([#145](https://github.com/microsoft/fabric-cicd/issues/145)) + +### โšก Additional Optimizations + +- Increase reserved name wait time ([#135](https://github.com/microsoft/fabric-cicd/issues/135)) + +## [v0.1.8](https://pypi.org/project/fabric-cicd/0.1.8) - March 04, 2025 + +### ๐Ÿ”ง Bug Fix + +- Handle null byPath object in report definition file ([#143](https://github.com/microsoft/fabric-cicd/issues/143)) +- Support relative directories ([#136](https://github.com/microsoft/fabric-cicd/issues/136)) ([#132](https://github.com/microsoft/fabric-cicd/issues/132)) +- Increase special character support ([#134](https://github.com/microsoft/fabric-cicd/issues/134)) + +### โšก Additional Optimizations + +- Changelog now available with version check ([#127](https://github.com/microsoft/fabric-cicd/issues/127)) + +## [v0.1.7](https://pypi.org/project/fabric-cicd/0.1.7) - February 26, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix special character support in files ([#129](https://github.com/microsoft/fabric-cicd/issues/129)) + +## [v0.1.6](https://pypi.org/project/fabric-cicd/0.1.6) - February 24, 2025 + +### ๐Ÿ†• New Items Support + +- Onboard Lakehouse item type ([#116](https://github.com/microsoft/fabric-cicd/issues/116)) + +### ๐Ÿ“ Documentation Update + +- Update example docs ([#25](https://github.com/microsoft/fabric-cicd/issues/25)) +- Update find_replace docs ([#110](https://github.com/microsoft/fabric-cicd/issues/110)) + +### โšก Additional Optimizations + +- Standardized docstrings to Google format +- Onboard file objects ([#46](https://github.com/microsoft/fabric-cicd/issues/46)) +- Leverage UpdateDefinition Flag ([#28](https://github.com/microsoft/fabric-cicd/issues/28)) +- Convert repo and workspace dictionaries ([#45](https://github.com/microsoft/fabric-cicd/issues/45)) + +## [v0.1.5](https://pypi.org/project/fabric-cicd/0.1.5) - February 18, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix Environment Failure without Public Library ([#103](https://github.com/microsoft/fabric-cicd/issues/103)) + +### โšก Additional Optimizations + +- Introduces pytest check for PRs ([#100](https://github.com/microsoft/fabric-cicd/issues/100)) + +## [v0.1.4](https://pypi.org/project/fabric-cicd/0.1.4) - February 12, 2025 + +### โœจ New Functionality + +- Support Feature Flagging ([#96](https://github.com/microsoft/fabric-cicd/issues/96)) + +### ๐Ÿ”ง Bug Fix + +- Fix Image support in report deployment ([#88](https://github.com/microsoft/fabric-cicd/issues/88)) +- Fix Broken README link ([#92](https://github.com/microsoft/fabric-cicd/issues/92)) + +### โšก Additional Optimizations + +- Workspace ID replacement improved +- Increased error handling in activate script +- Onboard pytest and coverage +- Improvements to nested dictionaries ([#37](https://github.com/microsoft/fabric-cicd/issues/37)) +- Support Python Installed From Windows Store ([#87](https://github.com/microsoft/fabric-cicd/issues/87)) + +## [v0.1.3](https://pypi.org/project/fabric-cicd/0.1.3) - January 29, 2025 + +### โœจ New Functionality + +- Add PyPI check version to encourage version bumps ([#75](https://github.com/microsoft/fabric-cicd/issues/75)) + +### ๐Ÿ”ง Bug Fix + +- Fix Semantic model initial publish results in None Url error ([#61](https://github.com/microsoft/fabric-cicd/issues/61)) +- Fix Integer parsed as float failing in handle_retry for <3.12 python ([#63](https://github.com/microsoft/fabric-cicd/issues/63)) +- Fix Default item types fail to unpublish ([#76](https://github.com/microsoft/fabric-cicd/issues/76)) +- Fix Items in subfolders are skipped ([#77](https://github.com/microsoft/fabric-cicd/issues/77)) + +### ๐Ÿ“ Documentation Update + +- Update documentation & examples + +## [v0.1.2](https://pypi.org/project/fabric-cicd/0.1.2) - January 27, 2025 + +### โœจ New Functionality + +- Introduces max retry and backoff for long running / throttled calls ([#27](https://github.com/microsoft/fabric-cicd/issues/27)) + +### ๐Ÿ”ง Bug Fix + +- Fix Environment publish uses arbitrary wait time ([#50](https://github.com/microsoft/fabric-cicd/issues/50)) +- Fix Environment publish doesn't wait for success ([#56](https://github.com/microsoft/fabric-cicd/issues/56)) +- Fix Long running operation steps out early for notebook publish ([#58](https://github.com/microsoft/fabric-cicd/issues/58)) + +## [v0.1.1](https://pypi.org/project/fabric-cicd/0.1.1) - January 23, 2025 + +### ๐Ÿ”ง Bug Fix + +- Fix Environment stuck in publish ([#51](https://github.com/microsoft/fabric-cicd/issues/51)) + +## [v0.1.0](https://pypi.org/project/fabric-cicd/0.1.0) - January 23, 2025 + +### โœจ New Functionality + +- Initial public preview release +- Supports Notebook, Pipeline, Semantic Model, Report, and Environment deployments +- Supports User and System Identity authentication +- Released to PyPi +- Onboarded to Github Pages diff --git a/docs/contribution.md b/docs/contribution.md deleted file mode 100644 index 2c12bcd8..00000000 --- a/docs/contribution.md +++ /dev/null @@ -1,98 +0,0 @@ -# Contribution - -This project welcomes contributions and suggestions. Most contributions require you to agree to a -Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us -the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com. - -When you submit a pull request, a CLA bot will automatically determine whether you need to provide -a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions -provided by the bot. You will only need to do this once across all repos using our CLA. - -This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). - -## Prerequisites - -Before you begin, ensure you have the following installed: - -- [Python](https://www.python.org/downloads/) (see [Installation](index.md#installation) for version requirements) -- [PowerShell](https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell) -- [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-windows) or [Az.Accounts PowerShell module](https://www.powershellgallery.com/packages/Az.Accounts/2.2.3) -- [Visual Studio Code (VS Code)](https://code.visualstudio.com/) - -## Initial Configuration - -1. **Fork the Repository on GitHub**: - - - Go to the repository [fabric-cicd](https://github.com/microsoft/fabric-cicd) on GitHub. - - In the top right corner, click on the **Fork** button. - - This will create a copy of the repository in your own GitHub account. - -1. **Clone Your Forked Repository**: - - - Once the fork is complete, go to your GitHub account and open the forked repository. - - Click on the **Code** button, and clone to VS Code. - -1. **Run activate.ps1**: - - - Open the Project in VS Code - - Open PowerShell terminal - - Run activate.ps1 which will install uv, and ruff if not already found. And set up the default environment leveraging uv sync. - ```powershell - .\activate.ps1 - ``` - _Note that this is technically optional and is designed to work with PowerShell. You can execute these steps manually as well, this is merely a helper_ - _For Linux, run `activate.sh` instead_ - -1. **Select Python Interpreter**: - - - Open the Command Palette (Ctrl+Shift+P) and select `Python: Select Interpreter`. - - Choose the interpreter from the `venv` directory. - -1. **Ensure All VS Code Extensions Are Installed**: - - - Open the Command Palette (Ctrl+Shift+P) and select `Extensions: Show Recommended Extensions`. - - Install all extensions recommended for the workspace. - -## Development - -### Managing Dependencies - -- All dependencies in this project are managed by uv which will resolve all dependencies and lock the versions to speed up virtual environment creation. -- For additions, run: - ```sh - uv add - ``` -- For removals, run: - ```sh - uv remove - ``` - -### Code Formatting & Linting - -- The python code within this project is maintained by ruff. -- If you install the recommended extensions, ruff will auto format on save of any file. -- Before being able to merge a PR, ruff is ran in a Github Action to ensure the files are properly formatted and maintained. -- To force linting, run the following. - ```sh - ruff format - ruff check - ``` - -### Pull Request Requirements - -- **All pull requests must be linked to an issue.** This ensures proper tracking and context for changes. -- **PR Title MUST follow this exact format**: "Fixes #123 - Short Description" where #123 is the issue number - - Use "Fixes" for bug fixes, "Closes" for features, "Resolves" for other changes - - Example: "Fixes #520 - Add Python version requirements to documentation" - - Version bump PRs are an exception: title must be "vX.X.X" format only -- Before creating a pull request: - 1. **Create or identify an existing issue** that describes the problem, feature request, or change you're addressing. - 2. **Use the proper PR title format** as specified above to link to the issue. - 3. You can also reference issues in commit messages using the same format. -- **GitHub Actions will automatically check** that your PR is linked to a valid issue and will fail if no valid reference is found. -- If you need to create an issue, use the appropriate template: - - [Bug Report](https://github.com/microsoft/fabric-cicd/issues/new?template=1-bug.yml) - - [Feature Request](https://github.com/microsoft/fabric-cicd/issues/new?template=2-feature.yml) - - [Documentation](https://github.com/microsoft/fabric-cicd/issues/new?template=3-documentation.yml) - - [Question](https://github.com/microsoft/fabric-cicd/issues/new?template=4-question.yml) - - [Technical Debt](https://github.com/microsoft/fabric-cicd/issues/new?template=5-tech-debt.yml) diff --git a/mkdocs.yml b/mkdocs.yml index 71a2d912..f2564af1 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -22,7 +22,6 @@ nav: - Deployment Variables: example/deployment_variable.md - Release Pipelines: example/release_pipeline.md - Code Reference: code_reference.md - - Contribution: contribution.md - Changelog: changelog.md - About: about.md @@ -119,6 +118,7 @@ markdown_extensions: - md_in_html - toc: permalink: true + toc_depth: 2 - admonition - pymdownx.highlight: anchor_linenums: true diff --git a/pyproject.toml b/pyproject.toml index d3f4ac55..f1b4a6ba 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -7,7 +7,6 @@ requires-python = ">=3.9,<3.13" license = "MIT" license-files = ["LICENSE"] dynamic = ["version"] -urls.Repository = "https://github.com/microsoft/fabric-cicd.git" classifiers = [ "Programming Language :: Python", "Programming Language :: Python :: 3.9", @@ -15,7 +14,6 @@ classifiers = [ "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", ] - dependencies = [ "azure-identity>=1.19.0", "dpath>=2.2.0", @@ -26,6 +24,10 @@ dependencies = [ "requests>=2.32.3", ] +[project.urls] +Repository = "https://github.com/microsoft/fabric-cicd.git" +Changelog = "https://github.com/microsoft/fabric-cicd/blob/main/docs/changelog.md" + [dependency-groups] dev = [ "coverage>=7.6.10", @@ -47,7 +49,6 @@ build-backend = "setuptools.build_meta" [tool.setuptools] dynamic.version = { attr = "fabric_cicd.constants.VERSION" } packages.find.where = ["src"] -package-data = { "fabric_cicd" = ["changelog.md"] } [tool.uv] package = true diff --git a/src/fabric_cicd/_common/_check_utils.py b/src/fabric_cicd/_common/_check_utils.py index 0a8c22cf..461656a8 100644 --- a/src/fabric_cicd/_common/_check_utils.py +++ b/src/fabric_cicd/_common/_check_utils.py @@ -26,7 +26,7 @@ def parse_changelog() -> dict[str, list[str]]: try: response = requests.get( - "https://raw.githubusercontent.com/microsoft/fabric-cicd/refs/heads/main/src/fabric_cicd/changelog.md" + "https://raw.githubusercontent.com/microsoft/fabric-cicd/refs/heads/main/docs/changelog.md" ) if response.status_code == 200: content = response.text @@ -37,7 +37,7 @@ def parse_changelog() -> dict[str, list[str]]: logger.debug(f"Error fetching online changelog: {e}") return {} - version_pattern = r"## Version (\d+\.\d+\.\d+).*?(?=## Version|\Z)" + version_pattern = r"## \[v(\d+\.\d+\.\d+)\].*?(?=## \[v|\Z)" changelog_dict = {} for match in re.finditer(version_pattern, content, re.DOTALL): diff --git a/src/fabric_cicd/changelog.md b/src/fabric_cicd/changelog.md deleted file mode 100644 index 78794165..00000000 --- a/src/fabric_cicd/changelog.md +++ /dev/null @@ -1,354 +0,0 @@ -The following contains all major, minor, and patch version release notes. - -- ๐Ÿ’ฅ Breaking change! -- โœจ New Functionality -- ๐Ÿ”ง Bug Fix -- ๐Ÿ“ Documentation Update -- โšก Internal Optimization - -## Version 0.1.34 - -Release Date: 2026-01-20 - -- โœจ Enable dynamic replacement of SQL endpoint values from SQL Database items ([#720](https://github.com/microsoft/fabric-cicd/issues/720)) -- โœจ Support Fabric Notebook Authentication ([#707](https://github.com/microsoft/fabric-cicd/issues/707)) -- โœจ Onboard Spark Job Definition item type ([#115](https://github.com/microsoft/fabric-cicd/issues/115)) -- ๐Ÿ“ Add `CONTRIBUTING.md` file to repository ([#723](https://github.com/microsoft/fabric-cicd/issues/723)) -- ๐Ÿ“ Add comprehensive troubleshooting guide to documentation ([#705](https://github.com/microsoft/fabric-cicd/issues/705)) -- ๐Ÿ“ Add parameterization documentation for Report items using ByConnection binding to Semantic Models ([#637](https://github.com/microsoft/fabric-cicd/issues/637)) -- โšก Add debug file for local Fabric REST API testing ([#714](https://github.com/microsoft/fabric-cicd/issues/714)) - -## Version 0.1.33 - -Release Date: 2025-12-16 - -- โœจ Add key_value_replace parameter support for YAML files ([#649](https://github.com/microsoft/fabric-cicd/issues/649)) -- โœจ Support selective shortcut publishing with regex exclusion ([#624](https://github.com/microsoft/fabric-cicd/issues/624)) -- โšก Add Linux development environment bootstrapping script ([#680](https://github.com/microsoft/fabric-cicd/issues/680)) -- โšก Update item types in scope to be an optional parameter in validate parameter file function ([#669](https://github.com/microsoft/fabric-cicd/issues/669)) -- ๐Ÿ”ง Fix publish order for Notebook and Eventhouse dependent items ([#685](https://github.com/microsoft/fabric-cicd/issues/685)) -- ๐Ÿ”ง Enable parameterizing multiple connections in the same Semantic Model item ([#674](https://github.com/microsoft/fabric-cicd/issues/674)) -- ๐Ÿ”ง Fix missing description metadata in item payload for shell-only item deployments ([#672](https://github.com/microsoft/fabric-cicd/issues/672)) -- ๐Ÿ”ง Resolve API long running operation handling when publishing Environment items ([#668](https://github.com/microsoft/fabric-cicd/issues/668)) - -## Version 0.1.32 - -Release Date: 2025-12-03 - -- ๐Ÿ”ง Fix publish bug for Environment items that contain only spark settings ([#664](https://github.com/microsoft/fabric-cicd/issues/664)) - -## Version 0.1.31 - -Release Date: 2025-12-01 - -- ๐Ÿ’ฅ Migrate to the latest Fabric Environment item APIs to simplify deployment and improve compatibility ([#173](https://github.com/microsoft/fabric-cicd/issues/173)) -- โœจ Add support for the ML Experiment item type ([#600](https://github.com/microsoft/fabric-cicd/issues/600)) -- โœจ Add support for the User Data Function item type ([#588](https://github.com/microsoft/fabric-cicd/issues/588)) -- โœจ Enable dynamic replacement of Lakehouse SQL Endpoint IDs ([#616](https://github.com/microsoft/fabric-cicd/issues/616)) -- โœจ Enable linking of Semantic Models to both cloud and gateway connections ([#602](https://github.com/microsoft/fabric-cicd/issues/602)) -- โœจ Allow use of the dynamic replacement variables within the key_value_replace parameter ([#567](https://github.com/microsoft/fabric-cicd/issues/567)) -- โœจ Add support for parameter file templates ([#499](https://github.com/microsoft/fabric-cicd/issues/499)) -- ๐Ÿ“ Update the advanced Dataflow parameterization example with the correct file_path value ([#633](https://github.com/microsoft/fabric-cicd/issues/633)) -- ๐Ÿ”ง Fix publishing issues for KQL Database items in folders ([#657](https://github.com/microsoft/fabric-cicd/issues/657)) -- ๐Ÿ”ง Separate logic for 'items to include' feature between publish and unpublish operations ([#650](https://github.com/microsoft/fabric-cicd/issues/650)) -- ๐Ÿ”ง Fix parameterization logic to properly handle find_value regex patterns and replacements ([#639](https://github.com/microsoft/fabric-cicd/issues/639)) -- ๐Ÿ”ง Correct the publish order of Data Agent and Semantic Model items ([#628](https://github.com/microsoft/fabric-cicd/issues/628)) -- ๐Ÿ”ง Fix Lakehouse item publishing errors when shortcuts refer to the default Lakehouse ID ([#610](https://github.com/microsoft/fabric-cicd/issues/610)) - -## Version 0.1.30 - -Release Date: 2025-10-20 - -- โœจ Add support for binding semantic models to on-premise gateways in Fabric workspaces ([#569](https://github.com/microsoft/fabric-cicd/issues/569)) -- โœจ Add support for publishing and managing Data Agent items ([#556](https://github.com/microsoft/fabric-cicd/issues/556)) -- โœจ Add OrgApp item type support ([#586](https://github.com/microsoft/fabric-cicd/issues/586)) -- โšก Enhance cross-workspace variable support to allow referencing other attributes ([#583](https://github.com/microsoft/fabric-cicd/issues/583)) -- ๐Ÿ”ง Fix workspace name extraction bug for non-ID attrs using ITEM_ATTR_LOOKUP ([#583](https://github.com/microsoft/fabric-cicd/issues/583)) -- ๐Ÿ”ง Fix capacity requirement check ([#593](https://github.com/microsoft/fabric-cicd/issues/593)) - -## Version 0.1.29 - -Release Date: 2025-10-01 - -- โœจ Onboard Apache Airflow Job item type ([#565](https://github.com/microsoft/fabric-cicd/issues/565)) -- โœจ Onboard Mounted Data Factory item type ([#406](https://github.com/microsoft/fabric-cicd/issues/406)) -- โœจ Support dynamic replacement for cross-workspace item IDs ([#558](https://github.com/microsoft/fabric-cicd/issues/558)) -- โœจ Add option to return API response for publish operations in publish_all_items ([#497](https://github.com/microsoft/fabric-cicd/issues/497)) -- ๐Ÿ”ง Fix publish order of Eventhouses and Semantic Models ([#566](https://github.com/microsoft/fabric-cicd/issues/566)) - -## Version 0.1.28 - -Release Date: 2025-09-15 - -- โœจ Add folder exclusion feature for publish operations ([#427](https://github.com/microsoft/fabric-cicd/issues/427)) -- โœจ Expand workspace ID dynamic replacement capabilities in parameterization ([#408](https://github.com/microsoft/fabric-cicd/issues/408)) -- ๐Ÿ”ง Fix unexpected behavior with file_path parameter filter ([#545](https://github.com/microsoft/fabric-cicd/issues/545)) -- ๐Ÿ”ง Fix unpublish exclude_regex bug in configuration file-based deployment ([#544](https://github.com/microsoft/fabric-cicd/issues/544)) - -## Version 0.1.27 - -Release Date: 2025-09-05 - -- ๐Ÿ”ง Fix traliing comma in report schema ([#534](https://github.com/microsoft/fabric-cicd/issues/534)) - -## Version 0.1.26 - -Release Date: 2025-09-05 - -- ๐Ÿ’ฅ Deprecate Base API URL kwarg in Fabric Workspace ([#529](https://github.com/microsoft/fabric-cicd/issues/529)) -- โœจ Support Schedules parameterization ([#508](https://github.com/microsoft/fabric-cicd/issues/508)) -- โœจ Support YAML configuration file-based deployment ([#470](https://github.com/microsoft/fabric-cicd/issues/470)) -- ๐Ÿ“ Add dynamically generated Python version requirements to documentation ([#520](https://github.com/microsoft/fabric-cicd/issues/520)) -- โšก Enhance pytest output to limit console verbosity ([#514](https://github.com/microsoft/fabric-cicd/issues/514)) -- ๐Ÿ”ง Fix Report item schema handling ([#518](https://github.com/microsoft/fabric-cicd/issues/518)) -- ๐Ÿ”ง Fix deployment order to publish Mirrored Database before Lakehouse ([#482](https://github.com/microsoft/fabric-cicd/issues/482)) - -## Version 0.1.25 - -Release Date: 2025-08-19 - -- ๐Ÿ’ฅ Modify the default for item_types_in_scope and add thorough validation ([#464](https://github.com/microsoft/fabric-cicd/issues/464)) -- โœจ Add new experimental feature flag to enable selective deployment ([#384](https://github.com/microsoft/fabric-cicd/issues/384)) -- โœจ Support "ALL" environment concept in parameterization ([#320](https://github.com/microsoft/fabric-cicd/issues/320)) -- ๐Ÿ“ Enhance Overview section in Parameterization docs ([#495](https://github.com/microsoft/fabric-cicd/issues/495)) -- โšก Eliminate ACCEPTED_ITEM_TYPES_NON_UPN constant and unify with ACCEPTED_ITEM_TYPES ([#477](https://github.com/microsoft/fabric-cicd/issues/477)) -- โšก Add comprehensive GitHub Copilot instructions for effective codebase development ([#468](https://github.com/microsoft/fabric-cicd/issues/468)) -- ๐Ÿ”ง Add feature flags and warnings for Warehouse, SQL Database, and Eventhouse unpublish operations ([#483](https://github.com/microsoft/fabric-cicd/issues/483)) -- ๐Ÿ”ง Fix code formatting inconsistencies in fabric_workspace unit test ([#474](https://github.com/microsoft/fabric-cicd/issues/474)) -- ๐Ÿ”ง Fix KeyError when deploying Reports with Semantic Model dependencies in Report-only scope case ([#278](https://github.com/microsoft/fabric-cicd/issues/278)) - -## Version 0.1.24 - -Release Date: 2025-08-04 - -- ๐Ÿ’ฅ Require parameterization for Dataflow and Semantic Model references in Data Pipeline activities -- ๐Ÿ’ฅ Require specific parameterization for deploying a Dataflow that depends on another in the same workspace (see Parameterization docs) -- ๐Ÿ”ง Fix Dataflow/Data Pipeline deployment failures caused by workspace permissions ([#419](https://github.com/microsoft/fabric-cicd/issues/419)) -- ๐Ÿ”ง Prevent duplicate logical ID issue in Report and Semantic Model deployment ([#405](https://github.com/microsoft/fabric-cicd/issues/405)) -- ๐Ÿ”ง Fix deployment of items without assigned capacity ([#402](https://github.com/microsoft/fabric-cicd/issues/402)) -- ๐Ÿ“ Improve Parameterization documentation ([#415](https://github.com/microsoft/fabric-cicd/issues/415)) -- โšก Support for Eventhouse query URI parameterization ([#414](https://github.com/microsoft/fabric-cicd/issues/414)) -- โšก Support for Warehouse SQL endpoint parameterization ([#392](https://github.com/microsoft/fabric-cicd/issues/392)) - -## Version 0.1.23 - -Release Date: 2025-07-08 - -- โœจ New functionalities for GitHub Copilot Agent and PR-to-Issue linking -- ๐Ÿ”ง Fix issue with lakehouse shortcuts publishing ([#379] (https://github.com/microsoft/fabric-cicd/issues/379)) -- ๐Ÿ”ง Add validation for empty logical IDs to prevent deployment corruption ([#86](https://github.com/microsoft/fabric-cicd/issues/86)) -- ๐Ÿ”ง Fix SQL provision print statement ([#329](https://github.com/microsoft/fabric-cicd/issues/329)) -- ๐Ÿ”ง Rename the error code for reserved item name per updated Microsoft Fabric API ([#388](https://github.com/microsoft/fabric-cicd/issues/388)) -- ๐Ÿ”ง Fix lakehouse exclude_regex to exclude shortcut publishing ([#385](https://github.com/microsoft/fabric-cicd/issues/385)) -- ๐Ÿ”ง Remove max retry limit to handle large deployments ([#299] (https://github.com/microsoft/fabric-cicd/issues/299)) -- ๐Ÿ“ Fix formatting and examples in the How to and Examples pages - -## Version 0.1.22 - -Release Date: 2025-06-25 - -- โœจ Onboard API for GraphQL item type ([#287](https://github.com/microsoft/fabric-cicd/issues/287)) -- ๐Ÿ”ง Fix Fabric API call error during dataflow publish ([#352](https://github.com/microsoft/fabric-cicd/issues/352)) -- โšก Expanded test coverage to handle folder edge cases ([#358](https://github.com/microsoft/fabric-cicd/issues/358)) - -## Version 0.1.21 - -Release Date: 2025-06-18 - -- ๐Ÿ”ง Fix bug with workspace ID replacement in JSON files for pipeline deployments ([#345](https://github.com/microsoft/fabric-cicd/issues/345)) -- โšก Increased max retry for Warehouses and Dataflows - -## Version 0.1.20 - -Release Date: 2025-06-12 - -- โœจ Onboard KQL Dashboard item type ([#329](https://github.com/microsoft/fabric-cicd/issues/329)) -- โœจ Onboard Dataflow Gen2 item type ([#111](https://github.com/microsoft/fabric-cicd/issues/111)) -- โœจ Parameterization support for find_value regex and replace_value variables ([#326](https://github.com/microsoft/fabric-cicd/issues/326)) -- ๐Ÿ”ง Fix bug with deploying environment libraries with special chars ([#336](https://github.com/microsoft/fabric-cicd/issues/336)) -- โšก Improved test coverage for subfolder creation/modification ([#211](https://github.com/microsoft/fabric-cicd/issues/211)) - -## Version 0.1.19 - -Release Date: 2025-05-21 - -- โœจ Onboard SQL Database item type (shell-only deployment) ([#301](https://github.com/microsoft/fabric-cicd/issues/301)) -- โœจ Onboard Warehouse item type (shell-only deployment) ([#204](https://github.com/microsoft/fabric-cicd/issues/204)) -- ๐Ÿ”ง Fix bug with unpublish workspace folders ([#273](https://github.com/microsoft/fabric-cicd/issues/273)) - -## Version 0.1.18 - -Release Date: 2025-05-14 - -- ๐Ÿ”ง Fix bug with check environment publish state ([#295](https://github.com/microsoft/fabric-cicd/issues/295)) - -## Version 0.1.17 - -Release Date: 2025-05-13 - -- ๐Ÿ’ฅ Deprecate old parameter file structure ([#283](https://github.com/microsoft/fabric-cicd/issues/283)) -- โœจ Onboard CopyJob item type ([#122](https://github.com/microsoft/fabric-cicd/issues/122)) -- โœจ Onboard Eventstream item type ([#170](https://github.com/microsoft/fabric-cicd/issues/170)) -- โœจ Onboard Eventhouse/KQL Database item type ([#169](https://github.com/microsoft/fabric-cicd/issues/169)) -- โœจ Onboard Data Activator item type ([#291](https://github.com/microsoft/fabric-cicd/issues/291)) -- โœจ Onboard KQL Queryset item type ([#292](https://github.com/microsoft/fabric-cicd/issues/292)) -- ๐Ÿ”ง Fix post publish operations for skipped items ([#277](https://github.com/microsoft/fabric-cicd/issues/277)) -- โšก New function `key_value_replace` for key-based replacement operations in JSON and YAML -- ๐Ÿ“ Add publish regex example to demonstrate how to use the `publish_all_items` with regex for excluding item names - -## Version 0.1.16 - -Release Date: 2025-04-25 - -- ๐Ÿ”ง Fix bug with folder deployment to root ([#255](https://github.com/microsoft/fabric-cicd/issues/255)) -- โšก Add Workspace Name in FabricWorkspaceObject ([#200](https://github.com/microsoft/fabric-cicd/issues/200)) -- โšก New function to check SQL endpoint provision status ([#226](https://github.com/microsoft/fabric-cicd/issues/226)) -- ๐Ÿ“ Updated Authentication docs + menu sort order - -## Version 0.1.15 - -Release Date: 2025-04-21 - -- ๐Ÿ”ง Fix folders moving with every publish ([#236](https://github.com/microsoft/fabric-cicd/issues/236)) -- โšก Introduce parallel deployments to reduce publish times ([#237](https://github.com/microsoft/fabric-cicd/issues/237)) -- โšก Improvements to check version logic -- ๐Ÿ“ Updated Examples section in docs - -## Version 0.1.14 - -Release Date: 2025-04-09 - -- โœจ Optimized & beautified terminal output -- โœจ Added changelog to output of old version check -- ๐Ÿ”ง Fix workspace folder deployments in root folder ([#221](https://github.com/microsoft/fabric-cicd/issues/221)) -- ๐Ÿ”ง Fix unpublish of workspace folders without publish ([#222](https://github.com/microsoft/fabric-cicd/issues/222)) -- โšก Removed Colorama and Colorlog Dependency - -## Version 0.1.13 - -Release Date: 2025-04-07 - -- โœจ Onboard Workspace Folders ([#81](https://github.com/microsoft/fabric-cicd/issues/81)) -- โœจ Onboard Variable Library item type ([#206](https://github.com/microsoft/fabric-cicd/issues/206)) -- โœจ Added support for Lakehouse Shortcuts -- โœจ New `enable_environment_variable_replacement` feature flag ([#160](https://github.com/microsoft/fabric-cicd/issues/160)) -- โšก User-agent now available in API headers ([#207](https://github.com/microsoft/fabric-cicd/issues/207)) -- โšก Fixed error log typo in fabric_endpoint -- ๐Ÿ”ง Fix break with invalid optional parameters ([#192](https://github.com/microsoft/fabric-cicd/issues/192)) -- ๐Ÿ”ง Fix bug where all workspace ids were not being replaced by parameterization ([#186](https://github.com/microsoft/fabric-cicd/issues/186)) - -## Version 0.1.12 - -Release Date: 2025-03-27 - -- ๐Ÿ”ง Fix constant overwrite failures ([#190](https://github.com/microsoft/fabric-cicd/issues/190)) -- ๐Ÿ”ง Fix bug where all workspace ids were not being replaced ([#186](https://github.com/microsoft/fabric-cicd/issues/186)) -- ๐Ÿ”ง Fix type hints for older versions of Python ([#156](https://github.com/microsoft/fabric-cicd/issues/156)) -- ๐Ÿ”ง Fix accepted item types constant in pre-build - -## Version 0.1.11 - -Release Date: 2025-03-25 - -- ๐Ÿ’ฅ Parameterization refactor introducing a new parameter file structure and parameter file validation functionality ([#113](https://github.com/microsoft/fabric-cicd/issues/113)) -- ๐Ÿ“ Update to [parameterization](https://microsoft.github.io/fabric-cicd/latest/how_to/parameterization/) docs -- โœจ Support regex for publish exclusion ([#121](https://github.com/microsoft/fabric-cicd/issues/121)) -- โœจ Override max retries via constants ([#146](https://github.com/microsoft/fabric-cicd/issues/146)) - -## Version 0.1.10 - -Release Date: 2025-03-19 - -- โœจ DataPipeline SPN Support ([#133](https://github.com/microsoft/fabric-cicd/issues/133)) -- ๐Ÿ”ง Workspace ID replacement in data pipelines ([#164](https://github.com/microsoft/fabric-cicd/issues/164)) -- ๐Ÿ“ Sample for passing in arguments from Azure DevOps Pipelines - -## Version 0.1.9 - -Release Date: 2025-03-11 - -- โœจ Support for Mirrored Database item type ([#145](https://github.com/microsoft/fabric-cicd/issues/145)) -- โšก Increase reserved name wait time ([#135](https://github.com/microsoft/fabric-cicd/issues/135)) - -## Version 0.1.8 - -Release Date: 2025-03-04 - -- ๐Ÿ”ง Handle null byPath object in report definition file ([#143](https://github.com/microsoft/fabric-cicd/issues/143)) -- ๐Ÿ”ง Support relative directories ([#136](https://github.com/microsoft/fabric-cicd/issues/136)) ([#132](https://github.com/microsoft/fabric-cicd/issues/132)) -- ๐Ÿ”ง Increase special character support ([#134](https://github.com/microsoft/fabric-cicd/issues/134)) -- โšก Changelog now available with version check ([#127](https://github.com/microsoft/fabric-cicd/issues/127)) - -## Version 0.1.7 - -Release Date: 2025-02-26 - -- ๐Ÿ”ง Fix special character support in files ([#129](https://github.com/microsoft/fabric-cicd/issues/129)) - -## Version 0.1.6 - -Release Date: 2025-02-24 - -- โœจ Onboard Lakehouse item type ([#116](https://github.com/microsoft/fabric-cicd/issues/116)) -- ๐Ÿ“ Update example docs ([#25](https://github.com/microsoft/fabric-cicd/issues/25)) -- ๐Ÿ“ Update find_replace docs ([#110](https://github.com/microsoft/fabric-cicd/issues/110)) -- โšก Standardized docstrings to Google format -- โšก Onboard file objects ([#46](https://github.com/microsoft/fabric-cicd/issues/46)) -- โšก Leverage UpdateDefinition Flag ([#28](https://github.com/microsoft/fabric-cicd/issues/28)) -- โšก Convert repo and workspace dictionaries ([#45](https://github.com/microsoft/fabric-cicd/issues/45)) - -## Version 0.1.5 - -Release Date: 2025-02-18 - -- ๐Ÿ”ง Fix Environment Failure without Public Library ([#103](https://github.com/microsoft/fabric-cicd/issues/103)) -- โšก Introduces pytest check for PRs ([#100](https://github.com/microsoft/fabric-cicd/issues/100)) - -## Version 0.1.4 - -Release Date: 2025-02-12 - -- โœจ Support Feature Flagging ([#96](https://github.com/microsoft/fabric-cicd/issues/96)) -- ๐Ÿ”ง Fix Image support in report deployment ([#88](https://github.com/microsoft/fabric-cicd/issues/88)) -- ๐Ÿ”ง Fix Broken README link ([#92](https://github.com/microsoft/fabric-cicd/issues/92)) -- โšก Workspace ID replacement improved -- โšก Increased error handling in activate script -- โšก Onboard pytest and coverage -- โšก Improvements to nested dictionaries ([#37](https://github.com/microsoft/fabric-cicd/issues/37)) -- โšก Support Python Installed From Windows Store ([#87](https://github.com/microsoft/fabric-cicd/issues/87)) - -## Version 0.1.3 - -Release Date: 2025-01-29 - -- โœจ Add PyPI check version to encourage version bumps ([#75](https://github.com/microsoft/fabric-cicd/issues/75)) -- ๐Ÿ”ง Fix Semantic model initial publish results in None Url error ([#61](https://github.com/microsoft/fabric-cicd/issues/61)) -- ๐Ÿ”ง Fix Integer parsed as float failing in handle_retry for <3.12 python ([#63](https://github.com/microsoft/fabric-cicd/issues/63)) -- ๐Ÿ”ง Fix Default item types fail to unpublish ([#76](https://github.com/microsoft/fabric-cicd/issues/76)) -- ๐Ÿ”ง Fix Items in subfolders are skipped ([#77](https://github.com/microsoft/fabric-cicd/issues/77)) -- ๐Ÿ“ Update documentation & examples - -## Version 0.1.2 - -Release Date: 2025-01-27 - -- โœจ Introduces max retry and backoff for long running / throttled calls ([#27](https://github.com/microsoft/fabric-cicd/issues/27)) -- ๐Ÿ”ง Fix Environment publish uses arbitrary wait time ([#50](https://github.com/microsoft/fabric-cicd/issues/50)) -- ๐Ÿ”ง Fix Environment publish doesn't wait for success ([#56](https://github.com/microsoft/fabric-cicd/issues/56)) -- ๐Ÿ”ง Fix Long running operation steps out early for notebook publish ([#58](https://github.com/microsoft/fabric-cicd/issues/58)) - -## Version 0.1.1 - -Release Date: 2025-01-23 - -- ๐Ÿ”ง Fix Environment stuck in publish ([#51](https://github.com/microsoft/fabric-cicd/issues/51)) - -## Version 0.1.0 - -Release Date: 2025-01-23 - -- โœจ Initial public preview release -- โœจ Supports Notebook, Pipeline, Semantic Model, Report, and Environment deployments -- โœจ Supports User and System Identity authentication -- โœจ Released to PyPi -- โœจ Onboarded to Github Pages