Add gateway log rotatation #2988
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
s-fairchild
force-pushed
the
gateway-log-rotate
branch
from
961f39b
to
4d54040
Compare
facchettos
previously requested changes
Jun 28, 2023
s-fairchild
force-pushed
the
gateway-log-rotate
branch
4 times, most recently
from
e2ea971
to
bb81733
Compare
s-amann
previously requested changes
Jun 28, 2023
There was a problem hiding this comment.
@s-fairchild I believe you need to re-run make generate and commit/push the results in order for CI to pass.
kimorris27
reviewed
Jun 28, 2023
s-fairchild
force-pushed
the
gateway-log-rotate
branch
from
bb81733
to
a594af9
Compare
s-fairchild
requested review from
jewzaam,
bennerv,
hawkowl,
rogbas,
petrkotas,
jharrington22,
cblecker,
UlrichSchlueter,
SudoBrendan,
ellis-johnson,
yjst2012 and
anshulvermapatel
as code owners
s-amann
reviewed
Jun 30, 2023
| @@ -67,6 +72,18 @@ include /etc/logrotate.d | |||
| create 0600 root utmp | |||
| rotate 1 | |||
| } | |||
|
|
|||
| # Maximum log directory size is 100G with this configuration | |||
| # Setting limit to 100G to allow space for other logging services | |||
There was a problem hiding this comment.
please update this comment to match the size on line 80
There was a problem hiding this comment.
The comment is correct, it's a multiple of size and rotate options. 5 logs of size 20G will be kept.
|
Please rebase pull request. |
s-fairchild
force-pushed
the
gateway-log-rotate
branch
from
a594af9
to
c03e29d
Compare
s-fairchild
force-pushed
the
gateway-log-rotate
branch
from
c03e29d
to
d809427
Compare
|
Please rebase pull request. |
The gateway container log is mounted in /var/log/aro-gateway to allow logrotate access to the file. Logs are rotated based on size, not time. Logs are rotated when they reach 20G in size, and a maximum of 100G total. This is to allow enough space for other logs and system services in /var. copytruncate option prevents the log file from being recreated after being deleted, which would create a condition where the log watcher/shipper thinks they are new. copytruncate truncates the log file, leaving it's modification timestamp the same.
s-fairchild
force-pushed
the
gateway-log-rotate
branch
from
d809427
to
e67f647
Compare
cadenmarchese
approved these changes
Jul 17, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue this PR addresses:
Fixes
Improved fix for https://portal.microsofticm.com/imp/v3/incidents/details/399550776/home
What this PR does / why we need it:
The gateway container log is mounted in
/var/log/aro-gatewayto allowlogrotateaccess to the file. Logs are rotated based on size, not time. Logs are rotated when they reach 20G in size, and a maximum of 100G total. This is to allow enough space for other logs and system services in /var.copytruncateoption prevents the log file from being recreated after being deleted, which would create a condition where the log watcher/shipper thinks they are new.copytruncatetruncates the log file, leaving it's modification timestamp the same.Test plan for issue:
E2E tests, and I believe a full service RP will be required to actually test.
sshing into the gateway VMSS and testing the logrotate configuration manually.
Is there any documentation that needs to be updated for this PR?
No