Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

www-apps/mattermost: 2023 monorepo conversion notes #3

Closed
lkraav opened this issue Nov 29, 2023 · 11 comments
Closed

www-apps/mattermost: 2023 monorepo conversion notes #3

lkraav opened this issue Nov 29, 2023 · 11 comments

Comments

@lkraav
Copy link
Contributor

lkraav commented Nov 29, 2023

Heya. Miraculously, I've just managed to successfully refactor your past ebuild to become compatible with their recent monorepo conversion structure, and thought I'd post something here for glorious benefit of community members possibly looking for this info somewhere. AFAICT there's currently no overlays maintaining www-apps/mattermost-server ebuilds, which is quite a shame 😿

Any feedback is welcome, and I guess this issue can be closed after review.

--- mattermost-server-7.8.2.ebuild      2023-04-09 23:23:21.358555479 +0300
+++ mattermost-server-8.1.6.ebuild      2023-11-29 21:47:39.348567817 +0200
@@ -4,10 +4,8 @@
 EAPI=8
 
 # Change this when you update the ebuild
-GIT_COMMIT="2e0b83e9e86eee88578f8e0a9bb8215aa13bb46a"
-WEBAPP_COMMIT="502a4ceb03b62242214720cfbcccfaca1dae6665"
+GIT_COMMIT="04849fd3c8705e64b6fe223716ff38f20a5b1e40"
 EGO_PN="github.com/mattermost/${PN}"
-WEBAPP_P="mattermost-webapp-${PV}"
 MY_PV="${PV/_/-}"
 
 if [[ "$ARCH" != "x86" && "$ARCH" != "amd64" ]]; then UNSUPPORTED_ARCH="1" ; fi
@@ -21,15 +19,15 @@
 HOMEPAGE="https://mattermost.com"
 SRC_URI="
        https://${EGO_PN}/archive/v${MY_PV}.tar.gz -> ${P}.tar.gz
-       https://${EGO_PN/server/webapp}/archive/v${MY_PV}.tar.gz -> ${WEBAPP_P}.tar.gz
        ${EGO_SUM_SRC_URI}
 "
 RESTRICT="mirror test"
 
 LICENSE="AGPL-3"
-SLOT="0/7.8"
+SLOT="0/8.1"
 KEYWORDS="~amd64 ~arm ~arm64 ~x86 ~ppc64" # Untested: arm64 x86
-IUSE="+npm-audit debug pie static"
+IUSE="npm-audit debug pie static"
+S="${WORKDIR}/mattermost-${PV}/server"
 
 RDEPEND="!www-apps/mattermost-server-ee
        acct-group/mattermost
@@ -62,7 +60,7 @@
 
 src_unpack() {
        go-module_src_unpack
-       mv "${S}/../${WEBAPP_P/_/-}" "${S}/client" || die
+       mv "${S}/../webapp" "${S}/client" || die
 }
 
 src_prepare() {
@@ -106,8 +104,8 @@
        # shellcheck disable=SC1117
        # Remove the git call, as the tarball isn't a proper git repository
        sed -i \
-               -E "s/^(\s*)COMMIT_HASH:(.*),$/\1COMMIT_HASH: JSON.stringify\(\"${WEBAPP_COMMIT}\)\"\),/" \
-               client/webpack.config.js || die
+               -E "s/^(\s*)COMMIT_HASH:(.*),$/\1COMMIT_HASH: JSON.stringify\(\"${GIT_COMMIT}\)\"\),/" \
+               client/channels/webpack.config.js || die
 
        default
 }
@@ -144,7 +142,7 @@
        pushd client > /dev/null || die
        ( use arm || use arm64 ) && append-cppflags "-DPNG_ARM_NEON_OPT=0"
        ( use ppc || use ppc64 ) && append-cppflags "-DPNG_POWERPC_VSX_OPT=0"
-       emake build
+       emake dist
        if use npm-audit && [[ $(npm --version | cut -d "." -f 1) -gt 5 ]]; then
                ebegin "Attempting to fix potential vulnerabilities"
                npm audit fix --package-lock-only || true
@@ -173,7 +171,7 @@
        doins -r {fonts,i18n,templates}
 
        insinto /usr/share/mattermost/client
-       doins -r client/dist/*
+       doins -r client/channels/dist/*
 
        diropts -o mattermost -g mattermost -m 0750
        keepdir /var/{lib,log}/mattermost
@matoro
Copy link
Owner

matoro commented Nov 29, 2023

Hey thanks, I don't mind readding this if it builds but the thing that caused me to drop it was openssl-3. Does this work with e.g. >=dev-libs/openssl-3.1?

@lkraav
Copy link
Contributor Author

lkraav commented Nov 30, 2023

Hey thanks, I don't mind readding this if it builds but the thing that caused me to drop it was openssl-3. Does this work with e.g. >=dev-libs/openssl-3.1?

I had zero problems building it on Node 18 and openssl-3.0.x. Fairly certain 3.1 would be fine, too. Many moons have passed since openssl-3 was introduced into the tree, I'd say majority of kinks have been worked out by now.

@matoro
Copy link
Owner

matoro commented Nov 30, 2023

Hey thanks, I don't mind readding this if it builds but the thing that caused me to drop it was openssl-3. Does this work with e.g. >=dev-libs/openssl-3.1?

I had zero problems building it on Node 18 and openssl-3.0.x. Fairly certain 3.1 would be fine, too. Many moons have passed since openssl-3 was introduced into the tree, I'd say majority of kinks have been worked out by now.

This doesn't seem sufficient to me, are you sure there aren't any other changes you made? I see they removed their vendored dependencies and that is causing go-module.eclass to choke?

>>> Emerging (1 of 1) www-apps/mattermost-server-8.1.6::matoro
 * mattermost-server-8.1.6.tar.gz BLAKE2B SHA512 size ;-) ...                                                                                                                            [ ok ]
>>> Unpacking source...
>>> Unpacking mattermost-server-8.1.6.tar.gz to /var/tmp/portage/www-apps/mattermost-server-8.1.6/work
go mod verify
go: code.sajari.com/[email protected]: Get "https://proxy.golang.org/code.sajari.com/docconv/@v/v1.3.5.mod": dial tcp: lookup proxy.golang.org on [::1]:53: read udp [::1]:48299->[::1]:53: read: connection refused
 * ERROR: www-apps/mattermost-server-8.1.6::matoro failed (unpack phase):
 *   go mod verify failed

@lkraav
Copy link
Contributor Author

lkraav commented Dec 2, 2023

go: code.sajari.com/[email protected]: Get "https://proxy.golang.org/code.sajari.com/docconv/@v/v1.3.5.mod": dial tcp: lookup proxy.golang.org on [::1]:53: read udp [::1]:48299->[::1]:53: read: connection refused

This looks like some (temporary?) network connectivity issue to me?

You are building with FEATURES="-network-sandbox" right?

@matoro
Copy link
Owner

matoro commented Jan 29, 2024

go: code.sajari.com/[email protected]: Get "https://proxy.golang.org/code.sajari.com/docconv/@v/v1.3.5.mod": dial tcp: lookup proxy.golang.org on [::1]:53: read udp [::1]:48299->[::1]:53: read: connection refused

This looks like some (temporary?) network connectivity issue to me?

You are building with FEATURES="-network-sandbox" right?

Okay, I figured this out. Firstly, it seems that for some reason variables set via /etc/portage/env/category/package don't apply to overlays, but variables set via /etc/portage/package.env do. Rather, the former sets it but it doesn't take effect for some reason (?), still not sure of the reason for this.

Regardless, I think the correct way to fix this is with RESTRICT="network-sandbox" (allowed in overlays but not the main tree) like I do in net-analyzer/arkime. I'll check to make sure this works with the latest versions, and push up an update soon.

@matoro
Copy link
Owner

matoro commented Jan 29, 2024

@lkraav I don't seem to be able to build the latest version (9.4.1), the server build is erroring out with the following:

$ go install ./cmd/mattermost
# github.com/mattermost/mattermost/server/v8/einterfaces
einterfaces/cloud.go:57:46: undefined: model.AllowedIPRanges
einterfaces/cloud.go:58:38: undefined: model.AllowedIPRanges
einterfaces/cloud.go:59:41: undefined: model.Installation
einterfaces/ip_filtering.go:9:40: undefined: model.AllowedIPRanges
einterfaces/ip_filtering.go:10:25: undefined: model.AllowedIPRanges
einterfaces/metrics.go:45:42: undefined: model.WebsocketEventType
einterfaces/metrics.go:46:46: undefined: model.WebsocketEventType
# github.com/mattermost/mattermost/server/v8/channels/store
channels/store/store.go:581:44: undefined: model.OutgoingOAuthConnection
channels/store/store.go:582:46: undefined: model.OutgoingOAuthConnection
channels/store/store.go:583:50: undefined: model.OutgoingOAuthConnection
channels/store/store.go:584:46: undefined: model.OutgoingOAuthConnectionGetConnectionsFilter
channels/store/store.go:584:101: undefined: model.OutgoingOAuthConnection

What's the latest version you've been able to build?

@lkraav
Copy link
Contributor Author

lkraav commented Feb 6, 2024

What's the latest version you've been able to build?

Pardon, I've fallen a bit behind here. Currently still running 8.1.6, haven't tried building v9.

@lkraav
Copy link
Contributor Author

lkraav commented Jul 1, 2024

It seems your effort at mattermost/mattermost#26076 was successful 💪 did you also manage to build the app on Gentoo now? I'm looking to upgrade to v9.

@lkraav
Copy link
Contributor Author

lkraav commented Jul 1, 2024

We are definitely running into this now mattermost/mattermost#26221

There's also a dependency on Node 18:

>>> Source configured.                                                                                                                                                                                                                        
[91.0.5@gusto ] Phase: compile                                                                                                                                                                                                                
>>> Compiling source in /run/zram/portage/www-apps/mattermost-server-9.5.5/work/mattermost-9.5.5/server ...
make -j12 dist                                                                                                         
Getting dependencies using npm                                                                                                                                                                                                                
CPPFLAGS= npm install                 
npm notice                               
npm notice New minor version of npm available! 10.5.0 -> 10.8.1
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.1                      
npm notice Run npm install -g [email protected] to update!                                                                    
npm notice                                                                                                             
npm ERR! code EBADENGINE
npm ERR! engine Unsupported engine
npm ERR! engine Not compatible with your version of node/npm: undefined                            
npm ERR! notsup Not compatible with your version of node/npm: undefined
npm ERR! notsup Required: {"node":"^18.10.0","npm":"^9.0.0"}                                                       
npm ERR! notsup Actual:   {"npm":"10.5.0","node":"v20.12.1"}
                                                                                                                       
npm ERR! A complete log of this run can be found in: /run/zram/portage/www-apps/mattermost-server-9.5.5/homedir/.npm/_logs/2024-07-01T20_21_53_926Z-debug-0.log

@lkraav
Copy link
Contributor Author

lkraav commented Jul 1, 2024

9.9.0 builds well with Node 20

diff --git a/mattermost-server-7.8.2.ebuild b/mattermost-server-9.9.0.ebuild
index 45b4dfe47f6d..fc2ee620d443 100644
--- a/mattermost-server-7.8.2.ebuild
+++ b/mattermost-server-9.9.0.ebuild
@@ -1,13 +1,11 @@
-# Copyright 1999-2020 Gentoo Authors
+# Copyright 1999-2024 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 EAPI=8
 
 # Change this when you update the ebuild
-GIT_COMMIT="2e0b83e9e86eee88578f8e0a9bb8215aa13bb46a"
-WEBAPP_COMMIT="502a4ceb03b62242214720cfbcccfaca1dae6665"
+GIT_COMMIT="4179e17b491ec5292ca6e157d7b216b464bce397"
 EGO_PN="github.com/mattermost/${PN}"
-WEBAPP_P="mattermost-webapp-${PV}"
 MY_PV="${PV/_/-}"
 
 if [[ "$ARCH" != "x86" && "$ARCH" != "amd64" ]]; then UNSUPPORTED_ARCH="1" ; fi
@@ -21,22 +19,22 @@ DESCRIPTION="Open source Slack-alternative in Golang and React (Team Edition)"
 HOMEPAGE="https://mattermost.com"
 SRC_URI="
        https://${EGO_PN}/archive/v${MY_PV}.tar.gz -> ${P}.tar.gz
-       https://${EGO_PN/server/webapp}/archive/v${MY_PV}.tar.gz -> ${WEBAPP_P}.tar.gz
        ${EGO_SUM_SRC_URI}
 "
 RESTRICT="mirror test"
 
 LICENSE="AGPL-3"
-SLOT="0/7.8"
+SLOT="0/9.9"
 KEYWORDS="~amd64 ~arm ~arm64 ~x86 ~ppc64" # Untested: arm64 x86
-IUSE="+npm-audit debug pie static"
+IUSE="npm-audit debug pie static"
+S="${WORKDIR}/mattermost-${PV}/server"
 
 RDEPEND="!www-apps/mattermost-server-ee
        acct-group/mattermost
        acct-user/mattermost"
 
 DEPEND="${RDEPEND}
-       >net-libs/nodejs-6[npm]
+       >=net-libs/nodejs-20.11:0/20[npm]
 "
 
 QA_PRESTRIPPED="usr/libexec/.*"
@@ -62,7 +60,7 @@ pkg_pretend() {
 
 src_unpack() {
        go-module_src_unpack
-       mv "${S}/../${WEBAPP_P/_/-}" "${S}/client" || die
+       mv "${S}/../webapp" "${S}/client" || die
 }
 
 src_prepare() {
@@ -77,7 +75,11 @@ src_prepare() {
        # provided in the server code.  Existing config.json files are not affected
        # by this change."
        sed -i 's/$(GOFLAGS) run/run $(GOFLAGS)/g' "Makefile" || die
-       go mod vendor || die
+       # Requires Go 1.22
+       # @see https://github.com/mattermost/mattermost/issues/26221#issuecomment-1945351597
+       # @since 2024.07.01
+       make setup-go-work || die
+       go work vendor -e || die
        emake config-reset
        mv config/config.json config/default.json || die
 
@@ -106,8 +108,8 @@ src_prepare() {
        # shellcheck disable=SC1117
        # Remove the git call, as the tarball isn't a proper git repository
        sed -i \
-               -E "s/^(\s*)COMMIT_HASH:(.*),$/\1COMMIT_HASH: JSON.stringify\(\"${WEBAPP_COMMIT}\)\"\),/" \
-               client/webpack.config.js || die
+               -E "s/^(\s*)COMMIT_HASH:(.*),$/\1COMMIT_HASH: JSON.stringify\(\"${GIT_COMMIT}\)\"\),/" \
+               client/channels/webpack.config.js || die
 
        default
 }
@@ -144,7 +146,7 @@ src_compile() {
        pushd client > /dev/null || die
        ( use arm || use arm64 ) && append-cppflags "-DPNG_ARM_NEON_OPT=0"
        ( use ppc || use ppc64 ) && append-cppflags "-DPNG_POWERPC_VSX_OPT=0"
-       emake build
+       emake dist
        if use npm-audit && [[ $(npm --version | cut -d "." -f 1) -gt 5 ]]; then
                ebegin "Attempting to fix potential vulnerabilities"
                npm audit fix --package-lock-only || true
@@ -173,7 +175,7 @@ src_install() {
        doins -r {fonts,i18n,templates}
 
        insinto /usr/share/mattermost/client
-       doins -r client/dist/*
+       doins -r client/channels/dist/*
 
        diropts -o mattermost -g mattermost -m 0750
        keepdir /var/{lib,log}/mattermost

@matoro
Copy link
Owner

matoro commented Jul 1, 2024

Thanks, can you just post the raw ebuild for 9.9.0 and let me give it a shot?

@matoro matoro closed this as completed Aug 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants