Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Preperation of orig.tar.gz with --git-overlay enabled #230

Open
andrewharle opened this issue Mar 29, 2020 · 5 comments · May be fixed by #231
Open

Preperation of orig.tar.gz with --git-overlay enabled #230

andrewharle opened this issue Mar 29, 2020 · 5 comments · May be fixed by #231
Labels
pending

Comments

@andrewharle
Copy link

Hi,
I receive an error gbp:error: Overlay must be used with --git-export-dir.

I have a repository with debian/gbp.conf as follows:

[DEFAULT]
upstream-tag = v%(version)s

[buildpackage]
overlay = True
export-dir = ../build-area

In the generate-git-snapshot script, --git-export-dir is unset. I was able to continue successfully with the build by overriding GBP_OPTS and explicitly setting --git-export-dir=../. I'm concerned however that this might break something else?

jenkins-debian-glue/scripts/generate-git-snapshot

Line 63 in ab25781

GBP_OPTS=" -nc --git-force-create --git-ignore-new --git-ignore-branch -S -us -uc --git-verbose --git-builder=/bin/true --git-cleaner=/bin/true --git-export-dir= "

For an example of this behaviour, try to build the https://packages.debian.org/source/buster/skalibs package.

Thanks,
@mika
Copy link
Owner

mika commented Mar 31, 2020

The export-dir = ../build-area configuration would cause the files ending up in an unexpected directory (from jenkins-debian-glue's point of view), that's why we have an explicit --git-export-dir= call in our default $GBP_OPTS.

@andrewharle: AFAIK I don't use overlay = True anywhere though, are you managing only the debian/ directory within git, or what's the rational for this?
Your change regarding --git-export-dir=../ makes sense to me, maybe we should even use and set this as a default in jenkins-debian-glue then?

@agx might I ask you - with your gbp hat on :) - about your opinion on that matter? Is there any reason coming to your mind, why we shouldn't explicitly set and force --git-export-dir to an explicit directory?
(BTW: for me the gbp:error: Overlay must be used with --git-export-dir was confusing at first, as we set the option but to an empty value. Is the error message something we should maybe clarify in the wording?)

@agx
Copy link

agx commented Mar 31, 2020

@agx might I ask you - with your gbp hat on :) - about your opinion on that matter? Is there any reason coming to your mind, why we shouldn't explicitly set and force --git-export-dir to an explicit directory?

it could say gbp:error: Overlay must be used with non-empty --git-export-dir (since from gbp's view "" is the same as unset).

Regarding overlay builds. To support these export-dir needs to be set so gbp knows where to overlay to so a possible solution might be to check gbp config buildpackage.overlay and only set build-area in that case to non-empty value.

mika pushed a commit that referenced this issue Apr 14, 2020
scripts/generate-git-snapshot: explicitly set git-export-dir option w…
345f782 
…hen building with enabled overlay option

If debian/gbp.conf contains something like:

| [buildpackage]
| overlay = True
| export-dir = ../build-area

then invoking gbp with our default $GBP_OPTS then it will fail with:

| gbp:error: Overlay must be used with --git-export-dir.

To avoid running into this, let's check for the overlay option,
and if it's enabled then explicitly set --git-export-dir= to '--git-export-dir=../'
to generate the orig.tar.* files at the expected place.

JFTR, we are *not* setting --git-export-dir=../ by default, as this
involves some further operations going on, which might be unexpected
in existing installations where the overlay feature isn't present/enabled.
See e.g.:

| % gbp buildpackage -nc --git-force-create --git-ignore-new --git-ignore-branch -S -us -uc --git-verbose --git-builder=/bin/true --git-cleaner=/bin/true --git-export-dir=
| gbp:debug: ['git', 'rev-parse', '--show-cdup']
| gbp:debug: ['git', 'rev-parse', '--is-bare-repository']
| gbp:debug: /bin/true [] []
| gbp:debug: ['git', 'symbolic-ref', 'HEAD']
| gbp:debug: ['git', 'show-ref', 'refs/heads/debian']
| gbp:debug: ['git', 'show-ref', 'refs/heads/pristine-tar']
| gbp:debug: ['git', 'ls-tree', 'upstream/5.8']
| gbp:info: zsh_5.8.orig.tar.gz does not exist, creating from 'upstream/5.8'
| gbp:debug: Building upstream tarball with compression 'gzip -9'
| gbp:debug: /bin/true ['-nc', '-S', '-us', '-uc'] []

vs:

| % gbp buildpackage -nc --git-force-create --git-ignore-new --git-ignore-branch -S -us -uc --git-verbose --git-builder=/bin/true --git-cleaner=/bin/true --git-export-dir=../
| gbp:debug: ['git', 'rev-parse', '--show-cdup']
| gbp:debug: ['git', 'rev-parse', '--is-bare-repository']
| gbp:debug: /bin/true [] []
| gbp:debug: ['git', 'symbolic-ref', 'HEAD']
| gbp:debug: ['git', 'show-ref', 'refs/heads/debian']
| gbp:debug: ['git', 'ls-tree', 'HEAD']
| gbp:debug: ['git', 'show', '--pretty=medium', 'HEAD:debian/source/format']
| gbp:debug: ['git', 'show', '--pretty=medium', 'HEAD:debian/changelog']
| gbp:debug: ['git', 'show-ref', 'refs/heads/pristine-tar']
| gbp:debug: ['git', 'ls-tree', 'upstream/5.8']
| gbp:info: zsh_5.8.orig.tar.gz does not exist, creating from 'upstream/5.8'
| gbp:debug: Building upstream tarball with compression 'gzip -9'
| gbp:info: Exporting 'HEAD' to '/home/buildd/zsh-tmp'
| gbp:debug: ['git', 'show', '--pretty=medium', 'HEAD:debian/source/format']
| gbp:info: Moving '/home/buildd/zsh-tmp' to '/home/buildd/zsh-5.8'
| gbp:debug: ['git', 'show', '--pretty=medium', 'HEAD:debian/source/format']
| gbp:debug: /bin/true ['-nc', '-S', '-us', '-uc'] []
| gbp:debug: rm ['-rf', '/home/buildd/zsh-5.8'] []

Thanks: Guido Günther for feedback
Closes: #230
@mika mika linked a pull request Apr 14, 2020 that will close this issue
Open
@mika
Copy link
Owner

mika commented Apr 14, 2020

Thank you @agx for your valuable feedback! I implemented what you suggested at #231 - as noted in its commit message, it seems to make a different whether the option is set to --git-export-dir=/.. or if its unset (--git-export-dir=). Does this make sense for you?

@andrewharle can you confirm that with #231 it works for you as expected?

@mika mika added the pending label Apr 14, 2020
@agx
Copy link

agx commented Apr 17, 2020

@mika you mean --export-dir=../ ? That should make a difference since unset means don't export anything while expoting to ../ would export to a random dir in .. and then rename (which will likely fail since the dir already exists)

@andrewharle
Copy link
Author

@mika Thanks for the comments. I've tested the pull request #231 and posted my observations.

mika pushed a commit that referenced this issue Jun 25, 2020
scripts/generate-git-snapshot: explicitly set git-export-dir option w…
a0692d7 
…hen building with enabled overlay option

If debian/gbp.conf contains something like:

| [buildpackage]
| overlay = True
| export-dir = ../build-area

then invoking gbp with our default $GBP_OPTS will fail with:

| gbp:error: Overlay must be used with --git-export-dir.

To avoid running into this, let's check for the overlay option,
and if it's enabled then explicitly set --git-export-dir= to '--git-export-dir=../'
to generate the orig.tar.* files at the expected place.

On Debian/jessie `gbp config buildpackage.overlay` reports something like:

| buildpackage.overlay=False

while on newer systems, `gbp config buildpackage.overlay` reports:

| False

So the check only grep's for "True".

JFTR, we are *not* setting --git-export-dir=../ by default, as this
involves some further operations going on, which might be unexpected
in existing installations where the overlay feature isn't present/enabled.
See e.g.:

| % gbp buildpackage -nc --git-force-create --git-ignore-new --git-ignore-branch -S -us -uc --git-verbose --git-builder=/bin/true --git-cleaner=/bin/true --git-export-dir=
| gbp:debug: ['git', 'rev-parse', '--show-cdup']
| gbp:debug: ['git', 'rev-parse', '--is-bare-repository']
| gbp:debug: /bin/true [] []
| gbp:debug: ['git', 'symbolic-ref', 'HEAD']
| gbp:debug: ['git', 'show-ref', 'refs/heads/debian']
| gbp:debug: ['git', 'show-ref', 'refs/heads/pristine-tar']
| gbp:debug: ['git', 'ls-tree', 'upstream/5.8']
| gbp:info: zsh_5.8.orig.tar.gz does not exist, creating from 'upstream/5.8'
| gbp:debug: Building upstream tarball with compression 'gzip -9'
| gbp:debug: /bin/true ['-nc', '-S', '-us', '-uc'] []

vs:

| % gbp buildpackage -nc --git-force-create --git-ignore-new --git-ignore-branch -S -us -uc --git-verbose --git-builder=/bin/true --git-cleaner=/bin/true --git-export-dir=../
| gbp:debug: ['git', 'rev-parse', '--show-cdup']
| gbp:debug: ['git', 'rev-parse', '--is-bare-repository']
| gbp:debug: /bin/true [] []
| gbp:debug: ['git', 'symbolic-ref', 'HEAD']
| gbp:debug: ['git', 'show-ref', 'refs/heads/debian']
| gbp:debug: ['git', 'ls-tree', 'HEAD']
| gbp:debug: ['git', 'show', '--pretty=medium', 'HEAD:debian/source/format']
| gbp:debug: ['git', 'show', '--pretty=medium', 'HEAD:debian/changelog']
| gbp:debug: ['git', 'show-ref', 'refs/heads/pristine-tar']
| gbp:debug: ['git', 'ls-tree', 'upstream/5.8']
| gbp:info: zsh_5.8.orig.tar.gz does not exist, creating from 'upstream/5.8'
| gbp:debug: Building upstream tarball with compression 'gzip -9'
| gbp:info: Exporting 'HEAD' to '/home/buildd/zsh-tmp'
| gbp:debug: ['git', 'show', '--pretty=medium', 'HEAD:debian/source/format']
| gbp:info: Moving '/home/buildd/zsh-tmp' to '/home/buildd/zsh-5.8'
| gbp:debug: ['git', 'show', '--pretty=medium', 'HEAD:debian/source/format']
| gbp:debug: /bin/true ['-nc', '-S', '-us', '-uc'] []
| gbp:debug: rm ['-rf', '/home/buildd/zsh-5.8'] []

Thanks: Guido Günther and Andrew Harle for feedback
Closes: #230
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
pending
Projects
None yet
Milestone
No milestone
3 participants
@mika @agx @andrewharle