Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AC-1682] Flexible collections: data migrations for deprecated permissions #3437

Merged
merged 48 commits into from
Jan 25, 2024

Conversation

r-tome
Copy link
Contributor

@r-tome r-tome commented Nov 10, 2023

Type of change

- [ ] Bug fix
- [X] New feature development
- [ ] Tech debt (refactoring, code cleanup, dependency upgrades, etc)
- [ ] Build/deploy pipeline (DevOps)
- [ ] Other

Objective

With Flexible Collections, the Manager role and 'EditAssignedCollections' permission are deprecated. Therefore, we must explicitly grant access to those users for the collections they previously had access to, but now with the new Manage permission. Additionally, users and groups with the AccessAll permission will now have access to all collections within their organizations.

Code changes

  • util/Migrator/DbScripts/2023-11-10_00_AccessAllCollectionGroups.sql: Add entries to [dbo].[CollectionGroup] with [Manage] = 1 for all groups with 'AccessAll' permission
  • util/Migrator/DbScripts/2023-11-10_00_AccessAllCollectionUsers.sql: Add entries to [dbo].[CollectionUser] with [Manage] = 1 for all organization users with 'AccessAll' permission
  • util/Migrator/DbScripts/2023-11-10_00_ManagersEditAssignedCollectionUsers.sql: Update [dbo].[CollectionUser] with [Manage] = 1 for all users with Manager role or 'EditAssignedCollections' permission

Before you submit

  • Please check for formatting errors (dotnet format --verify-no-changes) (required)
  • If making database changes - make sure you also update Entity Framework queries and/or migrations
  • Please add unit tests where it makes sense to do so (encouraged but not required)
  • If this change requires a documentation update - notify the documentation team
  • If this change has particular deployment requirements - notify the DevOps team

@r-tome r-tome requested a review from eliykat November 10, 2023 16:26
@r-tome r-tome marked this pull request as ready for review November 10, 2023 16:26
@r-tome r-tome requested a review from a team as a code owner November 10, 2023 16:26
@bitwarden-bot
Copy link

bitwarden-bot commented Nov 10, 2023

Logo
Checkmarx One – Scan Summary & Detailsab18185d-3294-4015-9196-0047dd984927

Fixed Issues

Severity Issue Source File / Package
HIGH Missing User Instruction /Dockerfile: 1
HIGH Missing User Instruction /Dockerfile: 1
HIGH Missing User Instruction /Dockerfile: 1
HIGH Missing User Instruction /Dockerfile: 1
HIGH Missing User Instruction /Dockerfile: 1
HIGH Missing User Instruction /Dockerfile: 1
HIGH Missing User Instruction /Dockerfile: 1
HIGH Missing User Instruction /Dockerfile: 1
HIGH Missing User Instruction /Dockerfile: 1
HIGH Missing User Instruction /Dockerfile: 1
HIGH Missing User Instruction /Dockerfile: 1
HIGH Missing User Instruction /Dockerfile: 1
HIGH Missing User Instruction /Dockerfile: 1
HIGH Missing User Instruction /Dockerfile: 1
HIGH Missing User Instruction /Dockerfile: 1
HIGH Passwords And Secrets - Generic Password /test-database.yml: 93
HIGH Passwords And Secrets - Generic Password /test-database.yml: 74
HIGH Passwords And Secrets - Generic Password /test-database.yml: 159
HIGH Passwords And Secrets - Generic Password /test-database.yml: 87
HIGH Passwords And Secrets - Generic Password /test-database.yml: 155
HIGH Reflected_XSS_All_Clients /src/Admin/Views/Shared/_OrganizationFormScripts.cshtml: 166
HIGH Reflected_XSS_All_Clients /bitwarden_license/src/Sso/Views/Shared/Error.cshtml: 10
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 7
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile-k8s: 8
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 7
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile-k8s: 8
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM Apt Get Install Pin Version Not Defined /Dockerfile: 5
MEDIUM CSRF /src/Identity/Controllers/AccountsController.cs: 73
MEDIUM CSRF /src/Identity/Controllers/AccountsController.cs: 51
MEDIUM CSRF /src/Api/Vault/Controllers/FoldersController.cs: 45
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 587
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 114
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 100
MEDIUM CSRF /src/Api/Tools/Controllers/ImportCiphersController.cs: 56
MEDIUM CSRF /src/Api/Tools/Controllers/ImportCiphersController.cs: 40
MEDIUM CSRF /src/Api/Controllers/SelfHosted/SelfHostedOrganizationLicensesController.cs: 51
MEDIUM CSRF /src/Api/Controllers/UsersController.cs: 22
MEDIUM CSRF /src/Api/Controllers/DevicesController.cs: 70
MEDIUM CSRF /src/Api/Controllers/DevicesController.cs: 57
MEDIUM CSRF /src/Api/Controllers/CollectionsController.cs: 198
MEDIUM CSRF /src/Api/Controllers/CollectionsController.cs: 189
MEDIUM CSRF /src/Api/Controllers/CollectionsController.cs: 158
MEDIUM CSRF /src/Api/Auth/Controllers/AccountsController.cs: 542
MEDIUM CSRF /src/Api/AdminConsole/Public/Controllers/PoliciesController.cs: 69
MEDIUM CSRF /src/Api/AdminConsole/Public/Controllers/PoliciesController.cs: 49
MEDIUM CSRF /src/Api/AdminConsole/Public/Controllers/OrganizationController.cs: 42
MEDIUM CSRF /src/Api/AdminConsole/Public/Controllers/GroupsController.cs: 92
MEDIUM CSRF /src/Api/AdminConsole/Public/Controllers/GroupsController.cs: 49
MEDIUM CSRF /src/Api/AdminConsole/Controllers/ProviderUsersController.cs: 142
MEDIUM CSRF /src/Api/AdminConsole/Controllers/ProviderOrganizationsController.cs: 52
MEDIUM CSRF /src/Api/AdminConsole/Controllers/PoliciesController.cs: 148
MEDIUM CSRF /src/Api/AdminConsole/Controllers/PoliciesController.cs: 78
MEDIUM CSRF /src/Api/AdminConsole/Controllers/PoliciesController.cs: 61
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 280
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 262
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 180
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 77
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 526
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 302
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 201
MEDIUM CSRF /src/Api/AdminConsole/Controllers/GroupsController.cs: 104
MEDIUM CSRF /src/Api/AdminConsole/Controllers/GroupsController.cs: 67
MEDIUM CSRF /src/Admin/Controllers/ProvidersController.cs: 175
MEDIUM CSRF /src/Admin/Controllers/OrganizationsController.cs: 284
MEDIUM CSRF /bitwarden_license/src/Sso/Controllers/AccountController.cs: 163
MEDIUM CSRF /bitwarden_license/src/Sso/Controllers/AccountController.cs: 96
MEDIUM CSRF /bitwarden_license/src/Scim/Controllers/v2/UsersController.cs: 50
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 209
MEDIUM CSRF /src/Api/Auth/Controllers/AccountsController.cs: 295
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 355
MEDIUM CSRF /src/Api/Auth/Controllers/AccountsController.cs: 553
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 442
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 248
MEDIUM CSRF /src/Api/AdminConsole/Controllers/GroupsController.cs: 193
MEDIUM CSRF /src/Api/AdminConsole/Controllers/GroupsController.cs: 118
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 208
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 316
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 424
MEDIUM CSRF /src/Api/Auth/Controllers/AccountsController.cs: 415
MEDIUM CSRF /src/Api/AdminConsole/Public/Controllers/GroupsController.cs: 161
MEDIUM CSRF /src/Api/Controllers/SettingsController.cs: 36
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 209
MEDIUM CSRF /src/Api/Auth/Controllers/EmergencyAccessController.cs: 159
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 556
MEDIUM CSRF /src/Api/AdminConsole/Controllers/GroupsController.cs: 135
MEDIUM CSRF /src/Api/AdminConsole/Controllers/GroupsController.cs: 135
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 263
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 526
MEDIUM CSRF /src/Api/Auth/Controllers/AccountsController.cs: 865
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 386
MEDIUM CSRF /src/Api/Auth/Controllers/AccountsController.cs: 390
MEDIUM CSRF /src/Api/Auth/Controllers/AccountsController.cs: 270
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 148
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 292
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 292
MEDIUM CSRF /src/Api/Auth/Controllers/AccountsController.cs: 924
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 308
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 308
MEDIUM CSRF /src/Api/AdminConsole/Controllers/ProviderUsersController.cs: 188
MEDIUM CSRF /bitwarden_license/src/Scim/Controllers/v2/GroupsController.cs: 98
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 255
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 456
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 691
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 691
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 642
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 642
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 725
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 180
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 668
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 668
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 163
MEDIUM CSRF /src/Api/Auth/Controllers/AccountsController.cs: 789
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 268
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 268
MEDIUM CSRF /src/Api/Auth/Controllers/TwoFactorController.cs: 403
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 526
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 526
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 526
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 745
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 255
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 292
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 292
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 133
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 523
MEDIUM CSRF /src/Api/AdminConsole/Controllers/ProviderUsersController.cs: 175
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 180
MEDIUM CSRF /src/Api/Auth/Controllers/AccountsController.cs: 568
MEDIUM CSRF /src/Api/AdminConsole/Controllers/GroupsController.cs: 151
MEDIUM CSRF /src/Api/Auth/Controllers/AccountsController.cs: 344
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 248
MEDIUM CSRF /src/Api/Auth/Controllers/AccountsController.cs: 595
MEDIUM CSRF /src/Api/AdminConsole/Public/Controllers/OrganizationController.cs: 42
MEDIUM CSRF /src/Api/AdminConsole/Public/Controllers/OrganizationController.cs: 42
MEDIUM CSRF /src/Api/AdminConsole/Public/Controllers/OrganizationController.cs: 42
MEDIUM CSRF /bitwarden_license/src/Scim/Controllers/v2/GroupsController.cs: 88
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 208
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 295
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 295
MEDIUM CSRF /src/Api/AdminConsole/Controllers/GroupsController.cs: 176
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 367
MEDIUM CSRF /src/Api/Auth/Controllers/AccountsController.cs: 204
MEDIUM

More results are available on AST platform

@r-tome
Copy link
Contributor Author

r-tome commented Nov 10, 2023

EF migrations are missing at this moment, I will add them soon.

@eliykat
Copy link
Member

eliykat commented Nov 13, 2023

Data migrations should be in the dbScripts_transition please, we want a bit of control over when they're run rather than being run automatically on deployment.

rkac-bw
rkac-bw previously approved these changes Nov 16, 2023
Copy link
Contributor

@rkac-bw rkac-bw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@eliykat eliykat added the hold Hold this PR or item until later; DO NOT MERGE label Nov 23, 2023
@eliykat
Copy link
Member

eliykat commented Nov 23, 2023

For visibility - I'm working through your other PRs first which I think are higher priority, so I probably won't get to this until next week.

@eliykat
Copy link
Member

eliykat commented Dec 5, 2023

Started looking at this today, but would like a second review from another Flexible Collections expert.

@eliykat eliykat removed the hold Hold this PR or item until later; DO NOT MERGE label Dec 5, 2023
FROM [dbo].[CollectionGroup] CG
INNER JOIN [dbo].[Collection] C ON CG.[CollectionId] = C.[Id]
INNER JOIN #TempGroup TG ON CG.[GroupId] = TG.[GroupId]
WHERE C.[OrganizationId] = TG.[OrganizationId];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% sure, but is this WHERE clause doing anything?

The INNER JOIN ensures that records exist in both tables, so if we INNER JOIN CollectionGroup, Collection and TempGroup, we should only end up with CollectionGroup records for Groups where AccessAll = 1. Then we can update the values without reference to the orgId. I think.

That said, I'm happy to leave it if that's safer or if there's another reason.

@rkac-bw

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, it also seems redundant to me, but perhaps its for performance reasons.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No performance gain with where clause, if does not add anything then it can be removed

@r-tome r-tome requested review from rkac-bw and eliykat December 6, 2023 16:20
Copy link
Member

@eliykat eliykat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work on this.

@rkac-bw could you please check the latest changes in response to my feedback. Thanks

…EditAssignedCollections permission assigned to groups with collection access
@r-tome
Copy link
Contributor Author

r-tome commented Jan 18, 2024

@eliykat / @rkac-bw I have removed the cursors from the script and also combined all distinct OrganizationUserId into one variable and then use that to bump all revision dates in one go. Does it make sense or is it safer to update them once per step like before?

Copy link
Member

@eliykat eliykat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this has been a lot of work and revisions, thanks for putting the time in to be the expert on these changes. I have a few questions below.

Comment on lines 86 to 111
-- Step 2
-- Update existing rows in [dbo].[CollectionUser]
UPDATE target
SET
target.[ReadOnly] = 0,
target.[HidePasswords] = 0,
target.[Manage] = 0
FROM [dbo].[CollectionUser] AS target
INNER JOIN (
SELECT C.[Id] AS [CollectionId], T.[OrganizationUserId]
FROM [dbo].[Collection] C
INNER JOIN #TempStep2 T ON C.[OrganizationId] = T.[OrganizationId]
) AS source
ON target.[CollectionId] = source.[CollectionId] AND target.[OrganizationUserId] = source.[OrganizationUserId];

-- Insert new rows into [dbo].[CollectionUser]
INSERT INTO [dbo].[CollectionUser] ([CollectionId], [OrganizationUserId], [ReadOnly], [HidePasswords], [Manage])
SELECT source.[CollectionId], source.[OrganizationUserId], 0, 0, 0
FROM (
SELECT C.[Id] AS [CollectionId], T.[OrganizationUserId]
FROM [dbo].[Collection] C
INNER JOIN #TempStep2 T ON C.[OrganizationId] = T.[OrganizationId]
) AS source
LEFT JOIN [dbo].[CollectionUser] AS target
ON target.[CollectionId] = source.[CollectionId] AND target.[OrganizationUserId] = source.[OrganizationUserId]
WHERE target.[CollectionId] IS NULL;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💭 Why doesn't this follow the same pattern/structure as Step 1? I thought it would be pretty similar because it's the same thing but for orgUsers instead of Groups. However, this uses target/source naming and has nested SELECT queries. Is there some difference here that I'm missing or is this just a result of removing the batching logic?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense and its easier to read. I have rewritten those parts here, what do you think?

UPDATE target
SET
    target.[ReadOnly] = 0,
    target.[HidePasswords] = 0,
    target.[Manage] = 0
FROM [dbo].[CollectionUser] AS target
INNER JOIN [dbo].[Collection] AS C ON target.[CollectionId] = C.[Id]
INNER JOIN #TempUsersAccessAll AS TU ON C.[OrganizationId] = TU.[OrganizationId] AND target.[OrganizationUserId] = TU.[OrganizationUserId];
INSERT INTO [dbo].[CollectionUser] ([CollectionId], [OrganizationUserId], [ReadOnly], [HidePasswords], [Manage])
SELECT C.[Id] AS [CollectionId], TU.[OrganizationUserId], 0, 0, 0
FROM [dbo].[Collection] C
INNER JOIN #TempUsersAccessAll TU ON C.[OrganizationId] = TU.[OrganizationId]
LEFT JOIN [dbo].[CollectionUser] target
    ON target.[CollectionId] = C.[Id] AND target.[OrganizationUserId] = TU.[OrganizationUserId]
WHERE target.[CollectionId] IS NULL;

@rkac-bw will perform some testing to see if there isn't any performance loss.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

eliykat
eliykat previously approved these changes Jan 25, 2024
Copy link
Contributor

@rkac-bw rkac-bw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@r-tome r-tome removed the hold Hold this PR or item until later; DO NOT MERGE label Jan 25, 2024
@r-tome r-tome merged commit bac0676 into main Jan 25, 2024
87 of 89 checks passed
@r-tome r-tome deleted the ac/ac-1682/data-migrations-for-deprecated-permissions branch January 25, 2024 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants