Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(users/services): Local groups cleanup #150

Merged
merged 5 commits into from
Jan 9, 2024

Conversation

denisonbarbosa
Copy link
Member

@denisonbarbosa denisonbarbosa commented Dec 13, 2023

Due to limitations with our cache, we can't know exactly which users were deleted when the database was corrupted and it got cleared up and rebuilt, so we need to have a routine that periodically cleans the local groups from any unexistent users.

This is a complex task that could potentially spawn a lot of subprocesses (in the form of gpasswd --delete calls) to clean the local groups. However, the dominant factor of complexity in the code is the number of groups (and the number of users that belong to those groups) rather than the number of existent users. Since we query directly /etc/groups to look for the groups, we are only dealing with local groups here and, as such, we should be fine with performance.

UDENG-1864

@denisonbarbosa denisonbarbosa marked this pull request as ready for review December 13, 2023 13:11
@denisonbarbosa denisonbarbosa requested a review from a team as a code owner December 13, 2023 13:11
internal/users/users.go Show resolved Hide resolved
internal/users/users.go Outdated Show resolved Hide resolved
internal/users/users.go Outdated Show resolved Hide resolved
internal/users/users.go Outdated Show resolved Hide resolved
internal/services/manager.go Outdated Show resolved Hide resolved
internal/services/manager_test.go Outdated Show resolved Hide resolved
Copy link
Member

@didrocks didrocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, apart from the lack of getent mocking which actually explains why there is only one positive use case and less granularity, the rest of the actual internal/users/ implementation looks good.

However, I didn’t review internal/services/: thinking about it, I changed a little bit my vision and I think we can do better in term of handling and when triggering all of this so that this routine is exceptional:

  • we should still try, when removing an user from a consistent database (because of expired cache), to clean the local groups this user is in. This is the normal procedure and we know exactly which user is currently under purge, and remove it.
  • I really see that as a fallback when the database is seen as corrupted and we shouldn’t attempt anything until getting a "database is corrupted" signal. That way, cleaning the user from system group will become again just a part of the corrupted database cleanup and will not be called more than this. It’s then decoupled from the manager.

Mind reworking this PR to follow this?

internal/users/tests/tests.go Outdated Show resolved Hide resolved
internal/users/users.go Outdated Show resolved Hide resolved
internal/users/tests/tests.go Outdated Show resolved Hide resolved
internal/users/tests/tests.go Outdated Show resolved Hide resolved
internal/users/users.go Outdated Show resolved Hide resolved
internal/users/users.go Outdated Show resolved Hide resolved
internal/users/users.go Outdated Show resolved Hide resolved
internal/users/users_test.go Outdated Show resolved Hide resolved
internal/users/users_test.go Outdated Show resolved Hide resolved
internal/users/users_test.go Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented Jan 4, 2024

Codecov Report

Attention: 5 lines in your changes are missing coverage. Please review.

Comparison is base (b92fb6a) 88.83% compared to head (3c81c49) 88.98%.

Files Patch % Lines
internal/users/users.go 92.85% 3 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #150      +/-   ##
==========================================
+ Coverage   88.83%   88.98%   +0.15%     
==========================================
  Files          35       36       +1     
  Lines        2650     2733      +83     
==========================================
+ Hits         2354     2432      +78     
- Misses        228      231       +3     
- Partials       68       70       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@didrocks
Copy link
Member

didrocks commented Jan 4, 2024

Is that really ready for review? I still see the manager/ directory calling the users one when we agreed to remove all this for this PR?

Copy link
Member

@didrocks didrocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, really take the time to look at what is needed or not, and then only, request for review. Look at the diff on github if the local diff is too hard to read, but take the time to do ensure it to give confidence on the review.

internal/services/tests/tests.go Outdated Show resolved Hide resolved
Copy link
Member

@didrocks didrocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is your review! This time, no undeed leftor, but still some work to do and open questions! Feel free to tell me if anything is not understandable.

internal/users/export_test.go Outdated Show resolved Hide resolved
internal/users/tests/tests.go Outdated Show resolved Hide resolved
internal/users/tests/tests.go Show resolved Hide resolved
internal/users/tests/tests.go Outdated Show resolved Hide resolved
internal/users/users.go Outdated Show resolved Hide resolved
internal/users/users.go Outdated Show resolved Hide resolved
internal/users/users_test.go Show resolved Hide resolved
internal/users/users_test.go Show resolved Hide resolved
internal/users/users_test.go Show resolved Hide resolved
@denisonbarbosa denisonbarbosa changed the title feat(users/services): System groups cleanup feat(users/services): Local groups cleanup Jan 9, 2024
Copy link
Member

@didrocks didrocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, only one real comment with 2 parts remaining! If you look at the coverage (you do, right?), there should be the Go-C function that is lacking coverage, right? I mentioned yesterday that we should unit test it (even if it’s going to be basic, still better than nothing).

Otherwise, it’s funny to see how this cgo part simplify the code in the end (less parsing, easier to test…)

Copy link
Member

@didrocks didrocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still not an unit test :p So not the last round, but next one should be!

internal/users/users_test.go Outdated Show resolved Hide resolved
Copy link
Member

@didrocks didrocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to me once the test pass and is commits are rebased!

We already had a function to update the groups, but none to clean the
groups from inactive users or to just remove the user from all the
groups it belongs to.

Now we have one to clean a specific user and one to clean all inactive
users from the groups, based on the list of users returned by the
getpwent call.
@denisonbarbosa denisonbarbosa merged commit 67805db into main Jan 9, 2024
6 checks passed
@denisonbarbosa denisonbarbosa deleted the system-groups-deletion branch January 9, 2024 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants