Packaging: expand DEBHELPER in deb maintainer scripts and macros in rpm scriptlets #6319
Conversation
Use /bin/bash instead of /bin/sh so shfmt knows that these scriptlets run with bash by default. See: https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/#_default_shell
I initially copied the logic from some of the rpm's generated with native deb tooling. I wanted to link to the actual sources, however, to facilitate comparing the logic for future updates.
cognifloyd
requested review from
mickmcgrath13,
winem,
nzlosh,
mamercad,
rush-skills,
guzzijones,
amanda11,
bishopbm1 and
a team
pull-request-size
bot
added
the
size/L
I initially copied the logic from some of the rpm's generated with native deb tooling. I wanted to link to the actual sources, however, to facilitate comparing the logic for future updates.
I initially copied the logic from some of the rpm's generated with native rpm tooling. I wanted to link to the actual sources, however, to facilitate comparing the logic for future updates. st2.spec used %service_* macros which were defined in helper.spec which used %systemd_* macros. Now, I've linked to the actual systemd macro sources and cleanly combined the logic for EL8+9.
cognifloyd
force-pushed
the
packaging-scriptlets_expand
branch
from
438c4d8 to
fae9319
Compare
| PACKS_GROUP=%{packs_group} | ||
| SYS_USER=%{stanley_user} | ||
| ST2_USER=%{svc_user} |
There was a problem hiding this comment.
These macros were defined in st2.spec: https://github.com/StackStorm/st2-packages/blob/9974d04084b01cde9d11994f02abf46290c37fe3/packages/st2/rpm/st2.spec#L3-L5
The equivalent hard-coded values were already embedded in the deb maintainer scripts, so, this does not change anything. It only inlines the values for rpm.
| # TODO: Maybe remove this as 'preset' (on install above) enables units by default | ||
| systemctl --no-reload enable ${_ST2_SERVICES} &>/dev/null || : |
There was a problem hiding this comment.
I debated removing this now. I left a TODO for now. I think we should wait to remove it until we can actually test the built rpms.
This does not come from upstream. We defined it in st2-packages.git: https://github.com/StackStorm/st2-packages/blob/6c81c2e8e1c1e0f0ec0132c74c1f67730f6b9bf8/rpmspec/helpers.spec#L84-L90
|
I just pushed another commit that enables shellcheck for these sources and fixes several issues identified by shellcheck (including one place where I misspelled systemd). Almost all of the changed lines were added in this PR, so it made sense to include that here. |
This PR is working towards doing packaging via pantsbuild. Eventually, I hope to archive and stop using st2-packages.git.
When building system packages with native tooling, both deb maintainer scripts and rpm scriptlets get expanded.
deb, the#DEBHELPER#marker in maintainer scripts gets replaced with "debhelper snippets" from things registered in deb rules and other control files.rpm, macros (which start with%) get expanded during rpm build so that all macros have been expanded/replaced in the scripts that actually get embedded in the built rpm.When using pants + nfpm to build the deb/rpm packages, the build process is simplified so no expansion occurs on the scripts/scriptlets that get embedded in the package. Therefore, we need to expand these scripts ourselves.
Almost all of the debhelper snippets and rpm macros we rely on have to do with managing the systemd services. This PR focuses on expanding and then refactoring to simplify those scripts. An example of that refactoring is that I use a var
_ST2_SERVICESand then run the systemd commands once--or in a loop--instead of repeating the command for every service (which is what the native expansion does).At first, I extracted the expanded scripts from st2 deb/rpm files generated with native tooling. Then I used that to find the sources for the debhelper snippets and rpm macros we actually use. I tried to include comments linking to those upstream sources so that we can consult them in the future as we upgrade to newer versions of each OS. Debian and EL provide unique systemd management utils so the logic used to run them is necessarily a little different between Debian and EL (and sometimes between releases of the same OS).
NOTE: Beyond the systemd debhelper snippets and rpm macros, we also rely on some debhelper snippets and rpm macros to manage building the
/opt/stackstorm/st2virtualenv. I am NOT expanding those in this PR. Instead--in a follow-up PR--I will make the post-install scripts run the pex (added in #6307) to extract the virtualenv. Using the pex avoids thedh_virtualenvdebhelper that is unsupported upstream, and skips the complexity of using the external tool we used in rpms. This way, we'll use the same process to generate the venv for both deb and rpm.The commits add and then refactor this expanded code. So, reviewing each commit would probably be more annoying than just reviewing the final result. I do, however, recommend reading the descriptions of the commits to get some more background on why I've done some of the changes.