defaultExtensions are way too fat, resulting in long installation times

[ostrolucky]

Running eg. nix develop github:loophp/nix-shell#php74 in folder with composer.json that has no ext- sections takes still quite long time. I can see it compiling things like gd, mysql, iconv and so on which are not needed at all. Is it possible to either improve the speed of installing these (binaries instead of having to compile it? Installing multiple extensions at the same time, or at least each with -J flag?), or skip them? Otherwise amazing feature of this shell, which is auto-installation of extensions from composer.json is quite under-utilized.

[drupol]

Hey hi!

Have you added nix-shell in Cachix ? This will actually download binaries instead of compiling stuff.

Let me know if you need help to enable it!

Cheers

[ostrolucky]

Unfortunately Cachix doesn't seem to do anything on my machine, because I have m1 arm64 and there are no build artifacts for this machine published. What's worse, Github doesn't seem to have such runners at the moment actions/runner-images#5631 (although it's possible to use self-hosted ones, or buildjet ones but that's not free)

This could be important:

➜  ~ uname -a
Darwin PL00562-3.local 22.3.0 Darwin Kernel Version 22.3.0: Thu Jan  5 20:48:54 PST 2023; root:xnu-8792.81.2~2/RELEASE_ARM64_T6000 arm64

[drupol]

Dear @ostrolucky ,

I aksed on the Nix PHP Matrix channel.

The issue comes from the fact that we merged the latest update of PHP without making sure that it was successfully built on that architecture.

If cannot be built (due to test or otherwise) it cannot be cached

@LeSuisse partially fixed the issue in NixOS/nixpkgs#215004 but it seems that there is another issue here NixOS/nixpkgs#215220.

I believe we need to wait the next release of PHP to make sure it builds correctly, unless there is an alternative, maybe @LeSuisse might know more than me on this?

[LeSuisse]

Yes there is one remaining flaky test on darwin, I opened NixOS/nixpkgs#215539 so we can work around it and have everything in cache.nixos.org again.

Note that is note the last PHP upgrade that caused the issue, it seems it is the case for quite a while (I did not bother further than that in time).

[drupol]

Thanks @LeSuisse, let's bury that issue in your new PR :)

[drupol]

Thanks patch has been merged 🎉 !

We just need to wait for the next hydra iteration (https://hydra.nixos.org/job/nixos/trunk-combined/tested#tabs-constituents) and you'll be able to use the binaries from the cache!

Thanks @LeSuisse and @etu !

[drupol]

@ostrolucky I think it should be fixed by now. Can you try ?

[etu]

@drupol Another thing to consider is that PHP 7.4 isn't cached by hydra since it's not a supported version in nixpkgs. Do you have a separate cache somewhere?

[drupol]

Yes, we do have fossar :) but it's not building for M1 architecture since it's not available on Github actions.

[ostrolucky]

I had to do nix channel --update first, right?

Following extensions are still being built
php-amqp, php-dom php-mysqlnd php-xdebug php-mysqli php-pdo_mysql php-xmlreader

but rest seems not, so I would say that's a big improvement :)

[drupol]

No, we don't use channels, we just use the flake feature, I never liked the idea of using channels.

To try correctly, do:

1. rm -rf ~/.cache/nix
2. nix run nixpkgs#php82 -- -v
3. That said, it is possible that extensions that are not enabled by default in PHP are being built locally on your machine.

And that should be quite fast now :)

[ostrolucky]

Still same thing with above listed 7 extensions, but yeah it's good enough now I would say. But I think we can still remove
these php-dom php-mysqlnd php-mysqli php-pdo_mysql php-xmlreader from here

nix-shell/src/phps.nix

Lines 5 to 43 in 24b0670

	defaultExtensions = [
	"bcmath"
	"calendar"
	"ctype"
	"curl"
	"dom"
	"exif"
	"fileinfo"
	"filter"
	"gd"
	"gettext"
	"gmp"
	"iconv"
	"intl"
	"mbstring"
	"mysqli"
	"mysqlnd"
	"opcache"
	"openssl"
	"pdo"
	"pdo_mysql"
	"pdo_odbc"
	"pdo_pgsql"
	"pdo_sqlite"
	"pgsql"
	"posix"
	"readline"
	"session"
	"simplexml"
	"sockets"
	"soap"
	"sodium"
	"sqlite3"
	"sysvsem"
	"tokenizer"
	"xmlreader"
	"xmlwriter"
	"zip"
	"zlib"

right? Or how can we convince upstream php pkg to include these packages by default?

[drupol]

I will think about it but if we want to convince upstream to have those enabled by default, then we have to motivate the request in an issue on NixOS/nixpkgs.

This is also something that we should discuss with the PHP Maintainers team (@jtojnar, @etu, @aanderse,
etc etc... )

What you could also do to avoid compilation time is to not use loophp/nix-shell and use php from nixpkgs only in the meantime.

[etu]

It's not a very hard change to implement, however, it's a breaking change.

Here's the steps of what we need to do to do that breaking change:

1. Decide which the new defaults should be (and a good motivation for this, the current defaults haven't been decided recently, they where decided many years ago before PHP in Nix was modular so to enable a module you needed to compile the whole thing.
2. Then we need to alter the three files (one for each supported version) 8.0 8.1 8.2 with the new defaults, however, this shouldn't just remove a bunch of things. Instead we should look at the stateVersion and make sure to not change the defaults if one is on an older version.
3. Document it in the release notes.

[drupol]

@etu are you sure this would be a breaking change? Here, we are suggesting adding extensions, not removing any. Why would this be considered a BC ?

[etu]

Oh, yeah, I misread that then... I'm however questioning if we have sensible defaults to begin with 😃

[drupol]

I don't think we have indeed... We should really discuss that thing :)

[drupol]

Oh crap,... the page I used to take inspiration from is no more existing.

[ostrolucky]

At https://symfony.com/doc/current/setup.html#technical-requirements only these extensions are listed:

• ctype
• iconv
• PCRE
• session
• simplexml
• tokenizer