- 
                Notifications
    You must be signed in to change notification settings 
- Fork 9.4k
          Add extra debug data to cache_invalidate log
          #40196
        
          New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: 2.4-develop
Are you sure you want to change the base?
Conversation
| Hi @convenient. Thank you for your contribution! 
 
 Allowed build names are:
 You can find more information about the builds here For more details, review the Code Contributions documentation. | 
| The security team has been informed about this pull request due to the presence of risky security keywords. For security vulnerability reports, please visit Adobe's vulnerability disclosure program on HackerOne or email [email protected]. | 
    
      
        2 similar comments
      
    
  
    | The security team has been informed about this pull request due to the presence of risky security keywords. For security vulnerability reports, please visit Adobe's vulnerability disclosure program on HackerOne or email [email protected]. | 
| The security team has been informed about this pull request due to the presence of risky security keywords. For security vulnerability reports, please visit Adobe's vulnerability disclosure program on HackerOne or email [email protected]. | 
| @magento run all tests | 
| $context['trace'] = (new \Exception)->getTrace(); | ||
| } | ||
| } | ||
| $this->logger->debug('cache_invalidate: ', $context); | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
btw, if. you pass exception to psr log based on monolog in format like $this->logger->debug('cache_invalidate: ', ['exception'=> new \Exception()]); it will add trace to message
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ooh i didn't know that thanks. I've just given it a quick test like this to see
$this->logger->debug('cache_invalidate: ', ['exception'=> new \Exception()]);and i get this log, it does the exception but without the full trace
main.DEBUG: cache_invalidate:  {"exception":"[object] (Exception(code: 0):  at /var/www/html/vendor/magento/framework/Cache/InvalidateLogger.php:54)"}
Am I missing a trick?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't speak to @kandy's message, but another option is casting the exception to string. That should give the message, code, and full formatted trace. (string)(new \Exception()) (see https://www.php.net/manual/en/exception.tostring.php)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rhoerr that's decent, looks like this
public function execute($invalidateInfo)
{
    $context = $this->makeParams($invalidateInfo);
    if (isset($invalidateInfo['tags'], $invalidateInfo['mode'])) {
        if ($invalidateInfo['mode'] === 'all' && is_array($invalidateInfo['tags']) && empty($invalidateInfo['tags'])) {
            // If we are sending a purge request to all cache storage capture the trace
            // This is not a usual flow, and likely a bug is causing a performance issue
            $context['exception'] =  (string)(new \Exception('full purge of cache storage triggered'));
        }
    }
    $this->logger->debug('cache_invalidate: ', $context);
}[2025-09-18T15:50:52.353321+00:00] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"tags":[],"mode":"all"},"exception":"Exception: full purge of cache storage triggered in /var/www/html/vendor/magento/framework/Cache/InvalidateLogger.php:51
Stack trace:
#0 /var/www/html/vendor/magento/framework/Cache/Frontend/Decorator/Logger.php(60): Magento\\Framework\\Cache\\InvalidateLogger->execute(Array)
#1 /var/www/html/vendor/magento/framework/Cache/Frontend/Decorator/Logger.php(48): Magento\\Framework\\Cache\\Frontend\\Decorator\\Logger->log(Array)
#2 /var/www/html/vendor/magento/framework/App/Cache.php(102): Magento\\Framework\\Cache\\Frontend\\Decorator\\Logger->clean()
#3 /var/www/html/vendor/magento/framework/App/Cache/Proxy.php(103): Magento\\Framework\\App\\Cache->clean(Array)
#4 /var/www/html/test.php(22): Magento\\Framework\\App\\Cache\\Proxy->clean(Array)
#5 {main}"} []
I'm happy with any approach really, so long as we get the full stack trace to work with. Can the adobe team please advise what they prefer :)
https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/40196/0b62a7b8683ef732e82264c00c926c60/Statics/report-sanity-ee.html ``` /** * Copyright {{CREATION_YEAR}} Adobe * All Rights Reserved. */ ```
| @magento create issue | 
| @magento run all tests | 
| @magento run all tests | 
| @magento run all tests | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @convenient for this PR! This is a valuable improvement for debugging cache performance issues.
Regarding @kandy's comment about exception handling here - they're absolutely right, and this would make the implementation even better.
Suggested change:
Instead of:
$context['exception'] = (string)(new \Exception('full purge of cache storage triggered'));
Use:
$context['exception'] = new \Exception('full purge of cache storage triggered');
This leverages Monolog's native exception handling per PSR-3 standard.
| Hey @engcom-Hotel can you please see my comment #40196 (comment) When I used that mechanism it did not contain the full trace, and therefore wouldnt be able to be used to help fix such issues | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @convenient for your reply!
I am approving this PR for further processing.
| @magento run all tests | 
| @magento run all tests | 
| Hi @convenient ✔️ QA Passed Preconditions: Manual testing scenarios: 
   Debug log with extra information for cache_invalidate Builds are failing, hence moving this PR for extended testing | 
| @magento run Functional Tests B2B, Functional Tests CE, Unit Tests | 
| The consistent failures in Functional B2B are known issues and JIRA is open for the same. The other failures are inconsistent and flaky. They neither part of the PR nor failing because of the PR changes. B2B Build 1 Build 2 Hence, Moving this PR Merge In Progress | 




Description (*)
Attach extra debug data to the
cache_invalidate:log. I recently came across another occurrence of this issue, but within a custom indexer magento/inventory@b6d7223. I believe having extra information on hand would make peoples lives easier.The indexer was erroneously purging all of redis several times, causing massive performance degradation. We can see frequent drops in redis performance on the web frontend.

And these are nicely correlated with the groupings of

cache_invalidatewith{"tags":[],"mode":"all"}purges.In this case I knew what I was looking for because I was aware of magento/inventory@b6d7223, but in cases like this I believe it would be neat if the Magento application just provided you with all the information to debug these scenarios up front. This change should make the system behave a bit nicer for both the core application in future (if any similar issues may occur) and for any custom development out there.
With this little bit of extra information, the stack trace will pinpoint the problem for the developers to take action without having to deploy similar logging or make guesswork. Given the impact, I think this is a sensible change. As far as I can see this flow is not really triggered normally in the application in any case.
Manual testing scenarios (*)
Cache flush
cache_invalidateis not called at all, because a flush purges the whole storage separately.Cache clean
As per #33122
cache:cleandoes not actually clean the whole Magento cache, it only purges the defined tags. So this log is not triggeredAdditional test cases
Here's where we see the logs being triggered, in the scenario where
cleanis being called without parameters.Contribution checklist (*)
Resolved issues:
cache_invalidatelog #40204: Add extra debug data tocache_invalidatelog