Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: git write back Helm value files #663

Closed
wants to merge 13 commits into from
Closed

Conversation

burnb
Copy link

@burnb burnb commented Jan 8, 2024

Related: #650
Fixes for #636
Based on PR #651

@burnb burnb marked this pull request as draft January 8, 2024 13:53
@devopsidiot
Copy link

Oh I'm cheering for this.

@codecov-commenter
Copy link

codecov-commenter commented Jan 10, 2024

Codecov Report

Attention: Patch coverage is 74.19355% with 8 lines in your changes missing coverage. Please review.

Project coverage is 57.10%. Comparing base (cdb4428) to head (17e9e27).

Files Patch % Lines
pkg/argocd/update.go 74.19% 7 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #663      +/-   ##
==========================================
+ Coverage   57.03%   57.10%   +0.07%     
==========================================
  Files          30       30              
  Lines        2979     3003      +24     
==========================================
+ Hits         1699     1715      +16     
- Misses       1145     1152       +7     
- Partials      135      136       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@burnb burnb marked this pull request as ready for review January 10, 2024 09:18
@devopsidiot
Copy link

Hi @burnb - I really appreciate your work on this. I tried this out this morning and unfortunately still ran into the error Could not update application spec: could not find an image-name annotation for image app-test" application=test

@burnb
Copy link
Author

burnb commented Jan 16, 2024

Hi @burnb - I really appreciate your work on this. I tried this out this morning and unfortunately still ran into the error Could not update application spec: could not find an image-name annotation for image app-test" application=test

Hi @devopsidiot could you show me what kind of annotation did you try to execute?

@devopsidiot
Copy link

Hi @burnb - Here you go (and thanks for responding):

metadata:
  name: daily-worship-test
  namespace: argo
  annotations:
    argocd-image-updater.argoproj.io/image-list: <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/helm.image-name: <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/helm.image-tag: develop-v0.20.0-33
    argocd-image-updater.argoproj.io/allow-tags: regexp:develop-v.*
    argocd-image-updater.argoproj.io/update-strategy: newest-build
    argocd-image-updater.argoproj.io/pull-secret: ext:/scripts/ecr-login.sh
    argocd-image-updater.argoproj.io/git-repository: [email protected]:<company>/gitops.git
    argocd-image-updater.argoproj.io/write-back-method: git:secret:argo-image-updater/argocd-image-updater-git-creds
    argocd-image-updater.argoproj.io/write-back-target: "helmvalues:values-rnd.yaml"
    argocd-image-updater.argoproj.io/git-branch: rnd
    ```

@burnb
Copy link
Author

burnb commented Jan 16, 2024

Hi @burnb - Here you go (and thanks for responding):

metadata:
  name: daily-worship-test
  namespace: argo
  annotations:
    argocd-image-updater.argoproj.io/image-list: <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/helm.image-name: <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/helm.image-tag: develop-v0.20.0-33
    argocd-image-updater.argoproj.io/allow-tags: regexp:develop-v.*
    argocd-image-updater.argoproj.io/update-strategy: newest-build
    argocd-image-updater.argoproj.io/pull-secret: ext:/scripts/ecr-login.sh
    argocd-image-updater.argoproj.io/git-repository: [email protected]:<company>/gitops.git
    argocd-image-updater.argoproj.io/write-back-method: git:secret:argo-image-updater/argocd-image-updater-git-creds
    argocd-image-updater.argoproj.io/write-back-target: "helmvalues:values-rnd.yaml"
    argocd-image-updater.argoproj.io/git-branch: rnd
    ```

helm.image-name and helm.image-tag are the paths to the values which will be generated by updater in the helmvalues file, in your case in the values-rnd.yaml
So it could be for example:

argocd-image-updater.argoproj.io/helm.image-name: image.name
argocd-image-updater.argoproj.io/helm.image-tag: image.tag

and it will be represented in values-rnd.yaml after updater refresh like

image:
  name: <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
  tag: develop-v0.20.0-33

@devopsidiot
Copy link

devopsidiot commented Jan 16, 2024

So I did the things correctly, but it's still not updating the application:

level=debug msg="found 32 from 32 tags eligible for consideration" image="<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.20.0-33"
level=info msg="Setting new image to <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.21.0-181" alias= application=daily-worship-test image_name=daily-worship image_tag=develop-v0.20.0-33 registry=<accountid>.dkr.ecr.us-east-1.amazonaws.com
level=debug msg="target parameters: image-spec= image-name=image.name, image-tag=image.tag" application=daily-worship-test image=<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
time="2024-01-16T18:22:45Z" level=info msg="Successfully updated image '<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.20.0-33' to '<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.21.0-181', but pending spec update (dry run=false)" alias= application=daily-worship-test image_name=daily-worship image_tag=develop-v0.20.0-33 registry=<accountid>.dkr.ecr.us-east-1.amazonaws.com
level=debug msg="Using commit message: build: automatic update of daily-worship-test\n\nupdates image daily-worship tag 'develop-v0.20.0-33' to 'develop-v0.21.0-181'\n"
level=info msg="Committing 1 parameter update(s) for application daily-worship-test" application=daily-worship-test
level=info msg="Initializing [email protected]:<company>/gitops.git to /tmp/git-daily-worship-test32075355"
level=info msg="rm -rf /tmp/git-daily-worship-test32075355" dir= execID=6aead
level=info msg=Trace args="[rm -rf /tmp/git-daily-worship-test32075355]" dir= operation_name="exec rm" time_ms=0.7052700000000001
level=warning msg="temporarily disabling strict host key checking (i.e. '-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null'), please don't use in production"
level=info msg="git fetch origin --tags --force" dir=/tmp/git-daily-worship-test32075355 execID=006f2
level=error msg="Could not update application spec: could not find an image-name annotation for image daily-worship" application=daily-worship-test

Annotations:

annotations:
    argocd-image-updater.argoproj.io/image-list: <companyid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/helm.image-name: image.name
    argocd-image-updater.argoproj.io/helm.image-tag: image.tag

Relevant values:

image:
  name: <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
  tag: develop-v0.20.0-33

@burnb
Copy link
Author

burnb commented Jan 17, 2024

So I did the things correctly, but it's still not updating the application:

level=debug msg="found 32 from 32 tags eligible for consideration" image="<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.20.0-33"
level=info msg="Setting new image to <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.21.0-181" alias= application=daily-worship-test image_name=daily-worship image_tag=develop-v0.20.0-33 registry=<accountid>.dkr.ecr.us-east-1.amazonaws.com
level=debug msg="target parameters: image-spec= image-name=image.name, image-tag=image.tag" application=daily-worship-test image=<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
time="2024-01-16T18:22:45Z" level=info msg="Successfully updated image '<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.20.0-33' to '<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.21.0-181', but pending spec update (dry run=false)" alias= application=daily-worship-test image_name=daily-worship image_tag=develop-v0.20.0-33 registry=<accountid>.dkr.ecr.us-east-1.amazonaws.com
level=debug msg="Using commit message: build: automatic update of daily-worship-test\n\nupdates image daily-worship tag 'develop-v0.20.0-33' to 'develop-v0.21.0-181'\n"
level=info msg="Committing 1 parameter update(s) for application daily-worship-test" application=daily-worship-test
level=info msg="Initializing [email protected]:<company>/gitops.git to /tmp/git-daily-worship-test32075355"
level=info msg="rm -rf /tmp/git-daily-worship-test32075355" dir= execID=6aead
level=info msg=Trace args="[rm -rf /tmp/git-daily-worship-test32075355]" dir= operation_name="exec rm" time_ms=0.7052700000000001
level=warning msg="temporarily disabling strict host key checking (i.e. '-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null'), please don't use in production"
level=info msg="git fetch origin --tags --force" dir=/tmp/git-daily-worship-test32075355 execID=006f2
level=error msg="Could not update application spec: could not find an image-name annotation for image daily-worship" application=daily-worship-test

Annotations:

annotations:
    argocd-image-updater.argoproj.io/image-list: <companyid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/helm.image-name: image.name
    argocd-image-updater.argoproj.io/helm.image-tag: image.tag

Relevant values:

image:
  name: <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
  tag: develop-v0.20.0-33

in your case annotation should be

annotations:
    argocd-image-updater.argoproj.io/image-list: <companyid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/daily-worship.helm.image-name: image.name
    argocd-image-updater.argoproj.io/daily-worship.helm.image-tag: image.tag

@devopsidiot
Copy link

Thanks so much for responding: I did these things:

in your case annotation should be

annotations:
    argocd-image-updater.argoproj.io/image-list: <companyid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/daily-worship.helm.image-name: image.name
    argocd-image-updater.argoproj.io/daily-worship.helm.image-tag: image.tag

And the only error that I got was time="2024-01-17T15:35:19Z" level=warning msg="Event could not be sent: Event \"daily-worship-test.17ab2c6a6ce998d6\" is invalid: involvedObject.namespace: Invalid value: \"argo\": does not match event.namespace" application=daily-worship-test

Since these argo applications are in the argo namespace and not the argocd namespace, am I just SoL for the time being?

@burnb
Copy link
Author

burnb commented Jan 18, 2024

It's just a warning that it can't send event to the k8s. And at this step it successfully sent git update.

@devopsidiot
Copy link

You literally made me cry with relief. Thank you so much for all your hard work. This needs to get adopted. This makes it work.

strangnet pushed a commit to strangnet/argocd-image-updater that referenced this pull request Feb 2, 2024
@strangnet
Copy link

I've integrated this PR with #672 since I needed both features, and initial tests shows that it's working. This should be a prioritised feature to be included in a release not too far away. Great work @burnb

@tomjohnburton
Copy link

@burnb you are a hero! Finally ArgoCD somewhat resembles the power of Flux

@jnovick
Copy link
Contributor

jnovick commented Feb 29, 2024

Hey, I have been using this and it works great. I am cheering for this too. I can attest that it works when using aliases in the image list. It writes to a specified file as long as the directory of the file exists. The one piece that seemed to be missing is the ability to write to a directory that does not exist yet. I opened a separate PR for it since it does not directly relate to your PR, but if the maintainers prefer, they can combine our PRs.

@s-ledyakhov
Copy link

@burnb Could you take a minute and give me a hint too?
It seems your version is what I need, but I get an error:

Could not update application spec: could not find an image-name annotation for image emm/arm

Annotations:

    argocd-image-updater.argoproj.io/image-list: arm=docker-registry.<name>/emm/arm
    argocd-image-updater.argoproj.io/arm.helm.image-name: deployment.arm.image.repository
    argocd-image-updater.argoproj.io/arm.helm.image-tag: deployment.arm.image.tag
    argocd-image-updater.argoproj.io/write-back-method: git
    argocd-image-updater.argoproj.io/write-back-target: "helmvalues:values_dev.yaml"

Values:

...
.
deployment:
  arm:
    image:
      repository: docker-registry.<name>/emm/arm
      tag: "9.0-1"

If I remove the write-back-target, then the .argocd-source-dev.yaml file looks like this, everything works fine

helm:
  parameters:
  - name: deployment.arm.image.repository
    value: docker-registry.<name>/emm/arm
    forcestring: true
  - name: deployment.arm.image.tag
    value: 9.0-57
    forcestring: true

If I add the write-back-target: "helmvalues:values_dev.yaml", then I get the error "could not find an image-name"
I also tried
write-back-target: "helmvalues"
write-back-target: "helmvalues:../../values_dev.yaml"
write-back-target: "helmvalues:/helm/emm-helm/values_dev.yaml"

Just in case, tree looks like this:

helm/
├── emm-helm
│   ├── values.yaml
│   ├── values_dev.yaml
│   ├── ...

I will be eternally grateful if you have time to help with this!

@burnb
Copy link
Author

burnb commented Mar 6, 2024

@s-ledyakhov I guess the problem is with parsing of your image link by this function:

func ParseNormalizedNamed(s string) (Named, error) {
	if ok := anchoredIdentifierRegexp.MatchString(s); ok {
		return nil, fmt.Errorf("invalid repository name (%s), cannot specify 64-byte hexadecimal strings", s)
	}
	domain, remainder := splitDockerDomain(s)
	var remote string
	if tagSep := strings.IndexRune(remainder, ':'); tagSep > -1 {
		remote = remainder[:tagSep]
	} else {
		remote = remainder
	}
	if strings.ToLower(remote) != remote {
		return nil, fmt.Errorf("invalid reference format: repository name (%s) must be lowercase", remote)
	}

	ref, err := Parse(domain + "/" + remainder)
	if err != nil {
		return nil, err
	}
	named, isNamed := ref.(Named)
	if !isNamed {
		return nil, fmt.Errorf("reference %s has no name", ref.String())
	}
	return named, nil
}

from the "github.com/distribution/distribution/v3/reference" package, coz this error doesn't go to the logs and in that case, it parses image a bit differently, so alias doesn't go anywhere, and this is the root of the problem - it operates with the image name.

@s-ledyakhov
Copy link

@s-ledyakhov I guess the problem is with parsing of your image link by this function:

from the "github.com/distribution/distribution/v3/reference" package, coz this error doesn't go to the logs and in that case, it parses image a bit differently, so alias doesn't go anywhere, and this is the root of the problem - it operates with the image name.

Thanks for the answer
Do I understand correctly that this problem has been solved in your repository?
I've tried installing your manifest, but I haven't been able to solve the problem yet
Although my values-file look the same as the user's above

@burnb
Copy link
Author

burnb commented Mar 6, 2024

I think the problem is with your <name> in image. Maybe it is not in the lower case?

@s-ledyakhov
Copy link

I think the problem is with your <name> in image. Maybe it is not in the lower case?

No, I checked this point
docker-registry.<companyname>.com where is one word written together in lowercase letters

@mubarak-j
Copy link
Contributor

I can confirm this PR fixes the issues with nested image parameters, however, it does rewrite the value file to sort all yaml/helm keys alphabetically. was this introduced in this PR or in #636? Can we avoid re-arranging helm parameters?

@burnb
Copy link
Author

burnb commented Mar 22, 2024

@mubarak-j fixed, but I'm not quite sure about why linter alerting, I can't reproduce it locally and all the messages not about the PR code.

@zagr0
Copy link

zagr0 commented May 15, 2024

Hi, can the fix be moved forward to be merged?
Thanks you very much in advance!

@burnb burnb force-pushed the master branch 2 times, most recently from 3900089 to e1646e4 Compare May 23, 2024 16:01
@pasha-codefresh
Copy link
Collaborator

@mubarak-j could you please resolve conflicts ?

@mubarak-j
Copy link
Contributor

@mubarak-j could you please resolve conflicts?

@burnb can you please do the honor? 🙏🏼

IMHO I think it's worth adding an example in the docs for using write-back-target with helmvalues, currently docs only cover kustomize use case. I could also do that in a separate PR.

@pasha-codefresh
Copy link
Collaborator

@burnb i also prefer resolve different issues in different PRs, can we split all the problems to different PRs?
As first thing i am going to review this one #651

@pasha-codefresh
Copy link
Collaborator

Left comment #651 (review)

@chengfang
Copy link
Collaborator

@burnb can you please take another look, rebase and resolve conflicts based on recent review comments? Thanks!

@pasha-codefresh
Copy link
Collaborator

@chengfang i think we should merge this one first #725

Signed-off-by: Aleksandr Petrov <[email protected]>
Signed-off-by: Aleksandr Petrov <[email protected]>
@burnb
Copy link
Author

burnb commented Jun 14, 2024

@burnb can you please take another look, rebase and resolve conflicts based on recent review comments? Thanks!

Done, I hope didn't break anything, coz some of the fixes was merged and done a bit differently.

@chengfang
Copy link
Collaborator

chengfang commented Jun 26, 2024

@burnb thanks for your continued work on this important issue. Unfortunately, there are still conflicts due to some recent merges. I took a look and jotted down a few notes below. Would you be able to take another look, resolve the manual merges?

  • Dockerfile: only whitespaces, so we can safely revert this file.
  • go.mod, go.sum: This PR removes conflate lib, which has already been removed in master branch. And master now also contains other updates (retryhttphelper). So I think we should stick to master's version, and revert these 2 files.
  • argocd.go: only whitespace change, so we can revert this file too.
  • argocd_test.go: your change looks good and can be easily auto-merged.
  • update.go: master branch now contains some of the logics in this PR, and some other changes, so this file will need to be carefully examined and manually merged.
  • update_test.go: mostly your changes, but the master contains some recent fixes, so will need manual merge to preserver the existing fixes in master.

@mubarak-j
Copy link
Contributor

Does the recently merged #748 implement the same fix as this PR?

@burnb
Copy link
Author

burnb commented Jun 27, 2024

@chengfang Now I don't see any reason to go further with my RP and IMHO it looks weird that PR's merge goes out of order so PR made half year ago become useless because someone did double work 2 weeks ago and PR was quickly merged.

@chengfang
Copy link
Collaborator

@burnb sorry for being disorganized when merged PRs, and thanks for your deep investigation and the fixes. Even though this PR cannot be merged, it contributes a great deal towards resolving the issue. Thanks to everyone here for the great discussion and insights.

@chengfang chengfang closed this Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.