Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AC-1682] Flexible collections: data migrations for deprecated permissions #3437

Merged
merged 48 commits into from
Jan 25, 2024
Merged

[AC-1682] Flexible collections: data migrations for deprecated permissions #3437

Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
e14ba4d
[AC-1682] Data migrations for OrgUsers or Groups with AccessAll enabled
r-tome Nov 10, 2023
1527af7
[AC-1682] Added script to update [dbo].[CollectionUser] with [Manage]…
r-tome Nov 10, 2023
577e591
[AC-1682] Updated sql data migration procedures with performance reco…
r-tome Nov 13, 2023
23a3e5c
[AC-1682] Moved data migration scripts to DbScripts_transition folder
r-tome Nov 13, 2023
d693511
Apply suggestions from code review: Remove Manage permission from Col…
r-tome Dec 6, 2023
0475b93
[AC-1682] Removed unnecessary Collection table join on ManagersEditAs…
r-tome Dec 6, 2023
3e7e3a0
[AC-1682] Change JOIN to INNER JOIN in SQL scripts
r-tome Dec 6, 2023
be35a2b
[AC-1682] Renamed sql script to recent date and added correct order t…
r-tome Dec 6, 2023
b7773b6
Merge branch 'master' into ac/ac-1682/data-migrations-for-deprecated-…
r-tome Dec 6, 2023
7d718b6
[AC-1682] Add new rows to CollectionUser for Managers and users with …
r-tome Dec 18, 2023
69c0997
Merge branch 'main' into ac/ac-1682/data-migrations-for-deprecated-pe…
r-tome Dec 18, 2023
e7ce15f
Merge branch 'main' into ac/ac-1682/data-migrations-for-deprecated-pe…
r-tome Jan 9, 2024
fcd2dd3
[AC-1682] Update FC data migration scripts to clear AccessAll flags a…
r-tome Jan 10, 2024
8bf17eb
[AC-1682] Updated data migration scripts to bump the account revision…
r-tome Jan 10, 2024
dfc4032
[AC-1682] Created Organization_EnableCollectionEnhancements to migrat…
r-tome Jan 10, 2024
0d8609a
[AC-1682] Added script to migrate all organization data for flexible …
r-tome Jan 10, 2024
54cc6fa
[AC-1682] Deleted old data migration scripts
r-tome Jan 10, 2024
a214c60
Revert "[AC-1682] Deleted old data migration scripts"
r-tome Jan 11, 2024
ee43767
[AC-1682] Modified AccessAllCollectionUsers script to bump revision d…
r-tome Jan 11, 2024
71978fe
Merge branch 'main' into ac/ac-1682/data-migrations-for-deprecated-pe…
r-tome Jan 11, 2024
f4450c0
[AC-1682] Update data migration script to only enable collection enha…
r-tome Jan 11, 2024
8cc8895
[AC-1682] Updated AccessAllCollectionGroups migration script to use U…
r-tome Jan 11, 2024
8a5ea76
[AC-1682] Bumped up the date on data migration scripts
r-tome Jan 11, 2024
68c586f
[AC-1682] Added back batching system to AccessAllCollectionUsers data…
r-tome Jan 11, 2024
ba06076
[AC-1682] Added data migration script to set FlexibleCollections = 1 …
r-tome Jan 11, 2024
f78e28f
[AC-1682] Modified data migration script to contain multiple transact…
r-tome Jan 11, 2024
3bf1b53
[AC-1682] Deleted old data migration scripts
r-tome Jan 11, 2024
c4ad7d7
[AC-1682] Placed temp tables outside transactions
r-tome Jan 11, 2024
96b5278
[AC-1682] Removed batching from AllOrgsEnableCollectionEnhancements s…
r-tome Jan 11, 2024
667f3cd
Merge branch 'main' into ac/ac-1682/data-migrations-for-deprecated-pe…
r-tome Jan 12, 2024
7a8cbd6
Merge branch 'main' into ac/ac-1682/data-migrations-for-deprecated-pe…
r-tome Jan 17, 2024
d1a8720
[AC-1682] Removed bulk data migration script
r-tome Jan 17, 2024
e651d1f
[AC-1682] Refactor stored procedure to enable collection enhancements
r-tome Jan 17, 2024
95977b1
[AC-1682] Added missing where clause
r-tome Jan 17, 2024
160fc3e
[AC-1682] Modified data migration script to have just one big transac…
r-tome Jan 18, 2024
6a519b9
[AC-1682] Combining all updated OrganizationUserIds to bump all revis…
r-tome Jan 18, 2024
6579ede
Update src/Sql/dbo/Stored Procedures/Organization_EnableCollectionEnh…
r-tome Jan 22, 2024
ea244df
[AC-1682] Renamed aliases
r-tome Jan 22, 2024
7dbb9e6
Merge branch 'ac/ac-1682/data-migrations-for-deprecated-permissions' …
r-tome Jan 22, 2024
0144c25
[AC-1682] Simplified inner queries
r-tome Jan 23, 2024
1e2a15d
[AC-1682] Bumping each modified groups RevisionDate
r-tome Jan 23, 2024
086c88f
[AC-1682] Removed updating CollectionUser existing records with [Read…
r-tome Jan 23, 2024
6fa4687
[AC-1682] Updating OrganizationUser RevisionDate
r-tome Jan 23, 2024
f1fb994
[AC-1682] Updated the stored procedure file
r-tome Jan 23, 2024
ebecc69
[AC-1682] Selecting distinct values to insert into CollectionUser table
r-tome Jan 24, 2024
737b81f
Revert "[AC-1682] Removed updating CollectionUser existing records wi…
r-tome Jan 25, 2024
5cc8912
[AC-1682] Bumped up the date on the migration script
r-tome Jan 25, 2024
f14e04c
[AC-1682] Updating OrganizationUser RevisionDate
r-tome Jan 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
160 changes: 160 additions & 0 deletions src/Sql/dbo/Stored Procedures/Organization_EnableCollectionEnhancements.sql
View file
Open in desktop
eliykat marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
CREATE PROCEDURE [dbo].[Organization_EnableCollectionEnhancements]
@OrganizationId UNIQUEIDENTIFIER
AS
BEGIN
SET NOCOUNT ON

-- Step 1: AccessAll migration for Groups
-- Create a temporary table to store the groups with AccessAll = 1
SELECT [Id] AS [GroupId], [OrganizationId]
INTO #TempStep1
r-tome marked this conversation as resolved.
Show resolved Hide resolved
FROM [dbo].[Group]
WHERE [OrganizationId] = @OrganizationId
AND [AccessAll] = 1;

-- Step 2: AccessAll migration for OrganizationUsers
-- Create a temporary table to store the OrganizationUsers with AccessAll = 1
SELECT [Id] AS [OrganizationUserId], [OrganizationId]
INTO #TempStep2
FROM [dbo].[OrganizationUser]
WHERE [OrganizationId] = @OrganizationId
AND [AccessAll] = 1;

-- Step 3: For all OrganizationUsers with Manager role or 'EditAssignedCollections' permission update their existing CollectionUser rows and insert new rows with [Manage] = 1
-- and finally update all OrganizationUsers with Manager role to User role
-- Create a temporary table to store the OrganizationUsers with Manager role or 'EditAssignedCollections' permission
SELECT ou.[Id] AS [OrganizationUserId],
CASE WHEN ou.[Type] = 3 THEN 1 ELSE 0 END AS [IsManager]
INTO #TempStep3
FROM [dbo].[OrganizationUser] ou
WHERE ou.[OrganizationId] = @OrganizationId
AND (ou.[Type] = 3 OR (ou.[Permissions] IS NOT NULL
AND ISJSON(ou.[Permissions]) > 0 AND JSON_VALUE(ou.[Permissions], '$.editAssignedCollections') = 'true'));

-- Step 4: Bump AccountRevisionDate for all OrganizationUsers updated in the previous steps
-- Combine and union the distinct OrganizationUserIds from all steps into a single variable
DECLARE @OrgUsersToBump [dbo].[GuidIdArray]
INSERT INTO @OrgUsersToBump
SELECT DISTINCT [OrganizationUserId] AS Id
FROM (
-- Step 1
SELECT GU.[OrganizationUserId]
FROM [dbo].[GroupUser] GU
INNER JOIN #TempStep1 TG ON GU.[GroupId] = TG.[GroupId]

UNION

-- Step 2
SELECT [OrganizationUserId]
FROM #TempStep2

UNION

-- Step 3
SELECT [OrganizationUserId]
FROM #TempStep3
) AS CombinedOrgUsers;
eliykat marked this conversation as resolved.
Show resolved Hide resolved

BEGIN TRY
BEGIN TRANSACTION;
-- Step 1
-- Update existing rows in [dbo].[CollectionGroup]
UPDATE CG
SET
CG.[ReadOnly] = 0,
CG.[HidePasswords] = 0,
CG.[Manage] = 0
FROM [dbo].[CollectionGroup] CG
INNER JOIN [dbo].[Collection] C ON CG.[CollectionId] = C.[Id]
INNER JOIN #TempStep1 TG ON CG.[GroupId] = TG.[GroupId]
WHERE C.[OrganizationId] = TG.[OrganizationId];

-- Insert new rows into [dbo].[CollectionGroup]
INSERT INTO [dbo].[CollectionGroup] ([CollectionId], [GroupId], [ReadOnly], [HidePasswords], [Manage])
SELECT C.[Id], TG.[GroupId], 0, 0, 0
FROM [dbo].[Collection] C
INNER JOIN #TempStep1 TG ON C.[OrganizationId] = TG.[OrganizationId]
LEFT JOIN [dbo].[CollectionGroup] CG ON CG.[CollectionId] = C.[Id] AND CG.[GroupId] = TG.[GroupId]
WHERE CG.[CollectionId] IS NULL;

-- Update Group to clear AccessAll flag
UPDATE G
SET [AccessAll] = 0
FROM [dbo].[Group] G
INNER JOIN #TempStep1 TG ON G.[Id] = TG.[GroupId];

-- Step 2
-- Update existing rows in [dbo].[CollectionUser]
UPDATE target
SET
target.[ReadOnly] = 0,
target.[HidePasswords] = 0,
target.[Manage] = 0
FROM [dbo].[CollectionUser] AS target
INNER JOIN (
SELECT C.[Id] AS [CollectionId], T.[OrganizationUserId]
FROM [dbo].[Collection] C
INNER JOIN #TempStep2 T ON C.[OrganizationId] = T.[OrganizationId]
) AS source
ON target.[CollectionId] = source.[CollectionId] AND target.[OrganizationUserId] = source.[OrganizationUserId];

-- Insert new rows into [dbo].[CollectionUser]
INSERT INTO [dbo].[CollectionUser] ([CollectionId], [OrganizationUserId], [ReadOnly], [HidePasswords], [Manage])
SELECT source.[CollectionId], source.[OrganizationUserId], 0, 0, 0
FROM (
SELECT C.[Id] AS [CollectionId], T.[OrganizationUserId]
FROM [dbo].[Collection] C
INNER JOIN #TempStep2 T ON C.[OrganizationId] = T.[OrganizationId]
) AS source
LEFT JOIN [dbo].[CollectionUser] AS target
ON target.[CollectionId] = source.[CollectionId] AND target.[OrganizationUserId] = source.[OrganizationUserId]
WHERE target.[CollectionId] IS NULL;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💭 Why doesn't this follow the same pattern/structure as Step 1? I thought it would be pretty similar because it's the same thing but for orgUsers instead of Groups. However, this uses target/source naming and has nested SELECT queries. Is there some difference here that I'm missing or is this just a result of removing the batching logic?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense and its easier to read. I have rewritten those parts here, what do you think?

UPDATE target
SET
    target.[ReadOnly] = 0,
    target.[HidePasswords] = 0,
    target.[Manage] = 0
FROM [dbo].[CollectionUser] AS target
INNER JOIN [dbo].[Collection] AS C ON target.[CollectionId] = C.[Id]
INNER JOIN #TempUsersAccessAll AS TU ON C.[OrganizationId] = TU.[OrganizationId] AND target.[OrganizationUserId] = TU.[OrganizationUserId];

INSERT INTO [dbo].[CollectionUser] ([CollectionId], [OrganizationUserId], [ReadOnly], [HidePasswords], [Manage])
SELECT C.[Id] AS [CollectionId], TU.[OrganizationUserId], 0, 0, 0
FROM [dbo].[Collection] C
INNER JOIN #TempUsersAccessAll TU ON C.[OrganizationId] = TU.[OrganizationId]
LEFT JOIN [dbo].[CollectionUser] target
    ON target.[CollectionId] = C.[Id] AND target.[OrganizationUserId] = TU.[OrganizationUserId]
WHERE target.[CollectionId] IS NULL;

@rkac-bw will perform some testing to see if there isn't any performance loss.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


-- Update OrganizationUser to clear AccessAll flag
UPDATE OU
SET [AccessAll] = 0
FROM [dbo].[OrganizationUser] OU
r-tome marked this conversation as resolved.
Show resolved Hide resolved
INNER JOIN #TempStep2 T ON OU.[Id] = T.[OrganizationUserId];

-- Step 3
-- Update [dbo].[CollectionUser] with [Manage] = 1 using the temporary table
UPDATE cu
SET cu.[ReadOnly] = 0,
cu.[HidePasswords] = 0,
cu.[Manage] = 1
FROM [dbo].[CollectionUser] cu
INNER JOIN #TempStep3 temp ON cu.[OrganizationUserId] = temp.[OrganizationUserId];

-- Insert rows to [dbo].[CollectionUser] with [Manage] = 1 using the temporary table
r-tome marked this conversation as resolved.
Show resolved Hide resolved
INSERT INTO [dbo].[CollectionUser] ([CollectionId], [OrganizationUserId], [ReadOnly], [HidePasswords], [Manage])
SELECT cg.[CollectionId], ou.[OrganizationUserId], 0, 0, 1
FROM [dbo].[CollectionGroup] cg
INNER JOIN [dbo].[GroupUser] gu ON cg.[GroupId] = gu.[GroupId]
INNER JOIN #TempStep3 ou ON gu.[OrganizationUserId] = ou.[OrganizationUserId]
WHERE NOT EXISTS (
SELECT 1 FROM [dbo].[CollectionUser] cu
WHERE cu.[CollectionId] = cg.[CollectionId] AND cu.[OrganizationUserId] = ou.[OrganizationUserId]
);

-- Update [dbo].[OrganizationUser] to migrate all OrganizationUsers with Manager role to User role
UPDATE ou
SET ou.[Type] = 2 -- User
FROM [dbo].[OrganizationUser] ou
INNER JOIN #TempStep3 temp ON ou.[Id] = temp.[OrganizationUserId]
WHERE temp.[IsManager] = 1; -- Filter for Managers

-- Step 4
-- Execute User_BumpAccountRevisionDateByOrganizationUserIds for the distinct OrganizationUserIds
EXEC [dbo].[User_BumpAccountRevisionDateByOrganizationUserIds] @OrgUsersToBump;
COMMIT TRANSACTION;
END TRY
BEGIN CATCH
ROLLBACK TRANSACTION;
THROW;
END CATCH;

-- Drop the temporary table
DROP TABLE #TempStep1;
DROP TABLE #TempStep2;
DROP TABLE #TempStep3;
END
161 changes: 161 additions & 0 deletions util/Migrator/DbScripts/2024-01-11_00_Organization_EnableCollectionEnhancements.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
CREATE OR ALTER PROCEDURE [dbo].[Organization_EnableCollectionEnhancements]
@OrganizationId UNIQUEIDENTIFIER
AS
BEGIN
SET NOCOUNT ON

-- Step 1: AccessAll migration for Groups
-- Create a temporary table to store the groups with AccessAll = 1
SELECT [Id] AS [GroupId], [OrganizationId]
INTO #TempStep1
FROM [dbo].[Group]
WHERE [OrganizationId] = @OrganizationId
AND [AccessAll] = 1;

-- Step 2: AccessAll migration for OrganizationUsers
-- Create a temporary table to store the OrganizationUsers with AccessAll = 1
SELECT [Id] AS [OrganizationUserId], [OrganizationId]
INTO #TempStep2
FROM [dbo].[OrganizationUser]
WHERE [OrganizationId] = @OrganizationId
AND [AccessAll] = 1;

-- Step 3: For all OrganizationUsers with Manager role or 'EditAssignedCollections' permission update their existing CollectionUser rows and insert new rows with [Manage] = 1
-- and finally update all OrganizationUsers with Manager role to User role
-- Create a temporary table to store the OrganizationUsers with Manager role or 'EditAssignedCollections' permission
SELECT ou.[Id] AS [OrganizationUserId],
CASE WHEN ou.[Type] = 3 THEN 1 ELSE 0 END AS [IsManager]
INTO #TempStep3
FROM [dbo].[OrganizationUser] ou
WHERE ou.[OrganizationId] = @OrganizationId
AND (ou.[Type] = 3 OR (ou.[Permissions] IS NOT NULL
AND ISJSON(ou.[Permissions]) > 0 AND JSON_VALUE(ou.[Permissions], '$.editAssignedCollections') = 'true'));

-- Step 4: Bump AccountRevisionDate for all OrganizationUsers updated in the previous steps
-- Combine and union the distinct OrganizationUserIds from all steps into a single variable
DECLARE @OrgUsersToBump [dbo].[GuidIdArray]
INSERT INTO @OrgUsersToBump
SELECT DISTINCT [OrganizationUserId] AS Id
FROM (
-- Step 1
SELECT GU.[OrganizationUserId]
FROM [dbo].[GroupUser] GU
INNER JOIN #TempStep1 TG ON GU.[GroupId] = TG.[GroupId]

UNION

-- Step 2
SELECT [OrganizationUserId]
FROM #TempStep2

UNION

-- Step 3
SELECT [OrganizationUserId]
FROM #TempStep3
) AS CombinedOrgUsers;

BEGIN TRY
BEGIN TRANSACTION;
-- Step 1
-- Update existing rows in [dbo].[CollectionGroup]
UPDATE CG
SET
CG.[ReadOnly] = 0,
CG.[HidePasswords] = 0,
CG.[Manage] = 0
FROM [dbo].[CollectionGroup] CG
INNER JOIN [dbo].[Collection] C ON CG.[CollectionId] = C.[Id]
INNER JOIN #TempStep1 TG ON CG.[GroupId] = TG.[GroupId]
WHERE C.[OrganizationId] = TG.[OrganizationId];

-- Insert new rows into [dbo].[CollectionGroup]
INSERT INTO [dbo].[CollectionGroup] ([CollectionId], [GroupId], [ReadOnly], [HidePasswords], [Manage])
SELECT C.[Id], TG.[GroupId], 0, 0, 0
FROM [dbo].[Collection] C
INNER JOIN #TempStep1 TG ON C.[OrganizationId] = TG.[OrganizationId]
LEFT JOIN [dbo].[CollectionGroup] CG ON CG.[CollectionId] = C.[Id] AND CG.[GroupId] = TG.[GroupId]
WHERE CG.[CollectionId] IS NULL;

-- Update Group to clear AccessAll flag
UPDATE G
SET [AccessAll] = 0
FROM [dbo].[Group] G
INNER JOIN #TempStep1 TG ON G.[Id] = TG.[GroupId];

-- Step 2
-- Update existing rows in [dbo].[CollectionUser]
UPDATE target
SET
target.[ReadOnly] = 0,
target.[HidePasswords] = 0,
target.[Manage] = 0
FROM [dbo].[CollectionUser] AS target
INNER JOIN (
SELECT C.[Id] AS [CollectionId], T.[OrganizationUserId]
FROM [dbo].[Collection] C
INNER JOIN #TempStep2 T ON C.[OrganizationId] = T.[OrganizationId]
) AS source
ON target.[CollectionId] = source.[CollectionId] AND target.[OrganizationUserId] = source.[OrganizationUserId];

-- Insert new rows into [dbo].[CollectionUser]
INSERT INTO [dbo].[CollectionUser] ([CollectionId], [OrganizationUserId], [ReadOnly], [HidePasswords], [Manage])
SELECT source.[CollectionId], source.[OrganizationUserId], 0, 0, 0
FROM (
SELECT C.[Id] AS [CollectionId], T.[OrganizationUserId]
FROM [dbo].[Collection] C
INNER JOIN #TempStep2 T ON C.[OrganizationId] = T.[OrganizationId]
) AS source
LEFT JOIN [dbo].[CollectionUser] AS target
ON target.[CollectionId] = source.[CollectionId] AND target.[OrganizationUserId] = source.[OrganizationUserId]
WHERE target.[CollectionId] IS NULL;

-- Update OrganizationUser to clear AccessAll flag
UPDATE OU
SET [AccessAll] = 0
FROM [dbo].[OrganizationUser] OU
INNER JOIN #TempStep2 T ON OU.[Id] = T.[OrganizationUserId];

-- Step 3
-- Update [dbo].[CollectionUser] with [Manage] = 1 using the temporary table
UPDATE cu
SET cu.[ReadOnly] = 0,
cu.[HidePasswords] = 0,
cu.[Manage] = 1
FROM [dbo].[CollectionUser] cu
INNER JOIN #TempStep3 temp ON cu.[OrganizationUserId] = temp.[OrganizationUserId];

-- Insert rows to [dbo].[CollectionUser] with [Manage] = 1 using the temporary table
INSERT INTO [dbo].[CollectionUser] ([CollectionId], [OrganizationUserId], [ReadOnly], [HidePasswords], [Manage])
SELECT cg.[CollectionId], ou.[OrganizationUserId], 0, 0, 1
FROM [dbo].[CollectionGroup] cg
INNER JOIN [dbo].[GroupUser] gu ON cg.[GroupId] = gu.[GroupId]
INNER JOIN #TempStep3 ou ON gu.[OrganizationUserId] = ou.[OrganizationUserId]
WHERE NOT EXISTS (
SELECT 1 FROM [dbo].[CollectionUser] cu
WHERE cu.[CollectionId] = cg.[CollectionId] AND cu.[OrganizationUserId] = ou.[OrganizationUserId]
);

-- Update [dbo].[OrganizationUser] to migrate all OrganizationUsers with Manager role to User role
UPDATE ou
SET ou.[Type] = 2 -- User
FROM [dbo].[OrganizationUser] ou
INNER JOIN #TempStep3 temp ON ou.[Id] = temp.[OrganizationUserId]
WHERE temp.[IsManager] = 1; -- Filter for Managers

-- Step 4
-- Execute User_BumpAccountRevisionDateByOrganizationUserIds for the distinct OrganizationUserIds
EXEC [dbo].[User_BumpAccountRevisionDateByOrganizationUserIds] @OrgUsersToBump;
COMMIT TRANSACTION;
END TRY
BEGIN CATCH
ROLLBACK TRANSACTION;
THROW;
END CATCH;

-- Drop the temporary table
DROP TABLE #TempStep1;
DROP TABLE #TempStep2;
DROP TABLE #TempStep3;
END
GO
Loading