lando pull Corrupts Data in Local Database during Pull from Pantheon

[GuyPaddock]

Over the last few hours, I've been debugging the following error, which only appears when running a copy of a Drupal site locally but does not appear with the same site on Pantheon:

PHP Fatal error:  Uncaught TypeError: Symfony\Component\Routing\RouteCollection::add(): Argument #2 ($route) must be of type Symfony\Component\Routing\Route, bool given, called in /app/web/core/lib/Drupal/Core/Routing/RouteProvider.php on line 383 and defined in /app/vendor/symfony/routing/RouteCollection.php:76
Stack trace:
#0 /app/web/core/lib/Drupal/Core/Routing/RouteProvider.php(383): Symfony\Component\Routing\RouteCollection->add('entity.comment....', false)
#1 /app/web/core/lib/Drupal/Core/Routing/RouteProvider.php(332): Drupal\Core\Routing\RouteProvider->getRoutesByPath('/comment/%/edit')
#2 /app/web/modules/contrib/devel/src/Routing/RouteSubscriber.php(176): Drupal\Core\Routing\RouteProvider->getRoutesByPattern('/comment/{comme...')
#3 /app/web/modules/contrib/devel/src/Routing/RouteSubscriber.php(87): Drupal\devel\Routing\RouteSubscriber->getRouteParameters(Object(Drupal\Core\Entity\ContentEntityType), 'edit-form')
#4 /app/web/modules/contrib/devel/src/Routing/RouteSubscriber.php(53): Drupal\devel\Routing\RouteSubscriber->getEntityLoadRoute(Object(Drupal\Core\Entity\ContentEntityType))
#5 /app/web/core/lib/Drupal/Core/Routing/RouteSubscriberBase.php(37): Drupal\devel\Routing\RouteSubscriber->alterRoutes(Object(Symfony\Component\Routing\RouteCollection))
#6 [internal function]: Drupal\Core\Routing\RouteSubscriberBase->onAlterRoutes(Object(Drupal\Core\Routing\RouteBuildEvent), 'routing.route_a...', Object(Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher))
#7 /app/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(142): call_user_func(Array, Object(Drupal\Core\Routing\RouteBuildEvent), 'routing.route_a...', Object(Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher))

At first, I thought maybe this was an issue with PHP version or something strange with Symfony or Drupal, but the version of PHP I am running locally in Lando is identical to the one on Pantheon. Eventually, I discovered the root cause of the issue -- it appears that some records containing serialized data are getting corrupted/modified in transit.

Here's what the serialized data looks like on Pantheon for the record that's causing the error above:

O:31:"Symfony\Component\Routing\Route":9:{s:4:"path";s:23:"/comment/{comment}/edit";s:4:"host";s:0:"";s:8:"defaults";a:2:{s:6:"_title";s:4:"Edit";s:12:"_entity_form";s:15:"comment.default";}s:12:"requirements";a:2:{s:14:"_entity_access";s:14:"comment.update";s:7:"comment";s:3:"\d+";}s:7:"options";a:4:{s:14:"compiler_class";s:33:"Drupal\Core\Routing\RouteCompiler";s:4:"utf8";b:1;s:10:"parameters";a:1:{s:7:"comment";a:2:{s:4:"type";s:14:"entity:comment";s:9:"converter";s:21:"paramconverter.entity";}}s:14:"_access_checks";a:1:{i:0;s:19:"access_check.entity";}}s:7:"schemes";a:0:{}s:7:"methods";a:2:{i:0;s:3:"GET";i:1;s:4:"POST";}s:9:"condition";s:0:"";s:8:"compiled";O:33:"Drupal\Core\Routing\CompiledRoute":11:{s:4:"vars";a:1:{i:0;s:7:"comment";}s:11:"path_prefix";s:0:"";s:10:"path_regex";s:37:"#^/comment/(?P<comment>\d+)/edit$#sDu";s:11:"path_tokens";a:3:{i:0;a:2:{i:0;s:4:"text";i:1;s:5:"/edit";}i:1;a:5:{i:0;s:8:"variable";i:1;s:1:"/";i:2;s:3:"\d+";i:3;s:7:"comment";i:4;b:1;}i:2;a:2:{i:0;s:4:"text";i:1;s:8:"/comment";}}s:9:"path_vars";a:1:{i:0;s:7:"comment";}s:10:"host_regex";N;s:11:"host_tokens";a:0:{}s:9:"host_vars";a:0:{}s:3:"fit";i:5;s:14:"patternOutline";s:15:"/comment/%/edit";s:8:"numParts";i:3;}}

Here's what the same data looks like locally in Lando:

O:31:"Symfony\Component\Routing\Route":9:{s:4:"path";s:23:"/comment/{comment}/edit";s:4:"host";s:0:"";s:8:"defaults";a:2:{s:6:"_title";s:4:"Edit";s:12:"_entity_form";s:15:"comment.default";}s:12:"requirements";a:2:{s:14:"_entity_access";s:14:"comment.update";s:7:"comment";s:3:"\d+";}s:7:"options";a:4:{s:14:"compiler_class";s:33:"Drupal\Core\Routing\RouteCompiler";s:4:"utf8";b:1;s:10:"parameters";a:1:{s:7:"comment";a:2:{s:4:"type";s:14:"entity:comment";s:9:"converter";s:21:"paramconverter.entity";}}s:14:"_access_checks";a:1:{i:0;s:19:"access_check.entity";}}s:7:"schemes";a:0:{}s:7:"methods";a:2:{i:0;s:3:"GET";i:1;s:4:"POST";}s:9:"condition";s:0:"";s:8:"compiled";O:33:"Drupal\Core\Routing\CompiledRoute":11:{s:4:"vars";a:1:{i:0;s:7:"comment";}s:11:"path_prefix";s:0:"";s:10:"path_regex";s:37:"#^/comment/(?P\d+)/edit$#sDu";s:11:"path_tokens";a:3:{i:0;a:2:{i:0;s:4:"text";i:1;s:5:"/edit";}i:1;a:5:{i:0;s:8:"variable";i:1;s:1:"/";i:2;s:3:"\d+";i:3;s:7:"comment";i:4;b:1;}i:2;a:2:{i:0;s:4:"text";i:1;s:8:"/comment";}}s:9:"path_vars";a:1:{i:0;s:7:"comment";}s:10:"host_regex";N;s:11:"host_tokens";a:0:{}s:9:"host_vars";a:0:{}s:3:"fit";i:5;s:14:"patternOutline";s:15:"/comment/%/edit";s:8:"numParts";i:3;}}

There's a lot to compare, so it might not be obvious what the issue is. Here's a graphical diff (look at the part highlighted in gray):

It seems that <comment> is getting stripped out of the regular expression in the serialized string, leading to the string being 28 characters instead of 37 characters. This is what is causing the deserialization failure:

PHP Notice:  unserialize(): Error at offset 837 of 1208 bytes

This only seems to happen if I use the lando pull command to get the database. If I create a backup on Pantheon, download it, and then use lando db-import, the database imports correctly without the corruption.

[GuyPaddock]

May be related to #153 and #154.

[bob-hinrichs]

This is happening to me and my colleagues on another project are reporting the same issue.

[Adrianm6254]

I want to let you know this is happening to me as well on only 1 site on Pantheon. All the other sites we have work fine. Not sure why it is only the once site. If I download the database and import it manually it imports fine.

I am currently using lando v3.18.0, php 8.1.18 on POP_OS LINUX 22.04 and Docker 24.0.2 because I am also running DDEV on some projects (but never at the same time).

[alvarobgodoy]

This also happened to me, on a Pantheon Drupal 10.1.1 site. As @GuyPaddock commented, manually creating a backup on Pantheon and using lando db-import works fine.

[seanr]

Any updates to this? I'm getting it right now. Will try the workaround.

[reynoldsalec]

@seanr and another user on the Lando Slack reported issues that sounds similar/same. In their case, the problem was limited to a single site; other sites were able to be pulled down fine. I personally haven't been able to replicate, but do believe there's an issue somewhere based on these reports and the excellent sleuthing done on this ticket (thanks all!).

Lando uses this command to pull the DB:

pantheon/lib/utils.js

Lines 359 to 360 in 7c30cfb

	command: 'terminus remote:drush',
	options: '-- sql-dump --structure-tables-list=cache,cache_*',

We also have a fallback command, which would run if the first command fails:

pantheon/scripts/pull.sh

Line 129 in 7c30cfb

FALLBACK_PULL_DB="$(echo $(terminus connection:info $SITE.$DATABASE --field=mysql_command) | sed 's,^mysql,mysqldump --no-autocommit --single-transaction --opt -Q,')"

If someone who has experienced this issue consistently could try running those two commands, it would be interesting to see if one of them fails/works. I can help you configure the commands with the correct variables if you need help there (DM me on the Lando Slack).

[burkmarr]

I've also been running into this issue recently. I only noticed it on projects after we migrated from D9 to D10. It does not occur on all projects. Thankfully I found this issue today and @GuyPaddock's suggestion of using lando db-import is now allowing me to work around it.

[donutdan4114]

Same error, same place.

Insufficient data for unserializing - 1188 required, 1180 present RouteProvider.php:382
PHP Fatal error:  Uncaught TypeError: Argument 2 passed to Symfony\Component\Routing\RouteCollection::add() must be an instance of Symfony\Component\Routing\Route, bool given, called in /app/web/core/lib/Drupal/Core/Routing/RouteProvider.php on line 382 and defined in /app/vendor/symfony/routing/RouteCollection.php:75

[kylebehse]

This has been going on for a while now. To reproduce (using Lando 3.x, php 8.1, Pantheon):

1. Push local to dev using Lando push;
2. Make changes to dev db, by altering some config;
3. Pull those changes back to Lando, using Lando pull (no code or files, just db);
4. Site is now throwing errors as seen above. *drush cr nor updb will fix. You must download a copy of the db from Pantheon, then $ lando db-import it, once you have moved the downloaded gzip to the root dir

This has been an ongoing issue for a while now (well over a year). And the work around really sucks considering easy it is to normally use Lando and Pantheon together. I hope we get to the bottom of this. Here is to fingers-crossed!!

[trackleft]

I see this when specific kinds of configuration appear in a sites db.
Example line 6 of this configuration https://github.com/az-digital/az_quickstart/blob/main/modules/custom/az_publication/config/install/az_publication.az_citation_style.az_bluebook.yml#L6

This is easy to reproduce on pantheon using our custom upstream, which I think may be public "3162cc4c-2f75-40cb-8487-0c69bda99f39".

terminus site:upstream:set <site_name> <upstream_id>

[jefflogan303]

I also have this effecting some sites I look after - but not every site, as others have noted.

This also effects manually running the terminus command and then running the db-import command:
terminus remote:drush <site>.<env> -- sql-dump --structure-tables-list="cache,cache_*" > test.sql

PHP versions both on pantheon and lando are 8.2.18.
Mariadb has slightly different versions: 10.4.25 on pantheon and 10.4.17 on lando. Not sure if slight difference could be the cause effecting some sites.

However, the workaround of importing a backup works fine.

FYI There is a related closed issue on Drupal.org, where some users mentioned it was effecting the pantheon plugin: https://www.drupal.org/project/drupal/issues/3152660

[gaddman]

Running in debug mode I see the following errors repeated a number of times:

lando ERROR RangeError: Invalid string length
at Socket.<anonymous> (/snapshot/cli/node_modules/@lando/core/lib/shell.js)
at Socket.emit (node:events:517:28)
at addChunk (node:internal/streams/readable:368:12)
at readableAddChunk (node:internal/streams/readable:341:9)
at Readable.push (node:internal/streams/readable:278:10)
at Pipe.onStreamRead (node:internal/stream_base_commons:190:23)  +109ms

From a search of that error message it suggests the output data is too big to parse.

@reynoldsalec - the fallback works fine but from my read of the code it'll never be executed - the initial pull is piped (into pv and onwards) so the exit status of that pipeline is zero even if the first step fails:

pantheon/scripts/pull.sh

Line 160 in eeaa28e

$PULL_DB | pv | $LOCAL_MYSQL_CONNECT_STRING || $FALLBACK_PULL_DB | pv | $LOCAL_MYSQL_CONNECT_STRING

[alextenczar]

I ran into this issue when I recently enabled the Devel module on my Pantheon site and pulled the db. Not sure why, but when I disabled Devel again, it worked. I could re-enable the module locally and it worked fine.

Just thought I'd leave this here in case someone finds this useful.

[AnaLaura-cellardoormedia]

Lando drush un devel work for me. We have a split configuration for devel but it should not be enabled in production. Need to figure why is this failing on cim

[elotze]

Dropping another note here to confirm that the devel module can cause this error. If devel is enabled on your remote environment when you do a lando pull, the lando (re)start command fails with all URLs reporting 500 errors. Running a lando drush cr (or lando drush anything) will also fail and throw the OP's route error.

There may be other things that cause this, but if you (maybe, like me, inadvertently) have devel enabled on your remote site and are seeing this issue, try:

1. lando drush pmu devel
2. lando start (or restart)
3. re-enable drush if necessary

After the app starts up successfully once, this error seems to not reoccur.