ArgoCD vault Plugin: line 5: syntax error near unexpected token `]'

[jauyzed]

Describe the bug
After a recent upgrade of ArgoCD operator and argoCD vault plugin on Openshift Cluster 4.16.15, we are seeing this error. Apps on the argocd are going into unknown state throwing an error as below

I'm not sure if the issue is with AVP or argoCD. I was asked to open an issue here as well.

Original issue: argoproj/argo-cd#20673

ComparisonError: Failed to load target state: failed to generate manifest for source 1 of 1: rpc error: code = Unknown desc = Manifest generation error (cached): plugin sidecar failed. error generating manifests in cmp: rpc error: code = Unknown desc = error generating manifests: `bash -c "helm template \"$ARGOCD_APP_NAME\" -n \"$ARGOCD_APP_NAMESPACE\" -f $(echo \"$ARGOCD_ENV_HELM_VALUES\") . | argocd-vault-plugin generate -"` failed exit status 2: /usr/local/bin/argocd-vault-plugin: line 5: syntax error near unexpected token `]' /usr/local/bin/argocd-vault-plugin: line 5: ` } ]' 

Here is the cmp-plugin.yaml Config map

kind: ConfigMap
apiVersion: v1
metadata:
  name: cmp-plugin
  namespace: openshift-gitops
  uid: 218a3b09-3a33-4438-9577-b520f1bd8266
  resourceVersion: '516875537'
  creationTimestamp: '2023-09-29T20:51:05Z'
  labels:
    app.kubernetes.io/instance: argocd-gitops-local-cluster
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{"avp.yaml":"apiVersion: argoproj.io/v1alpha1\nkind: ConfigManagementPlugin\nmetadata:\n  name: argocd-vault-plugin\nspec:\n  allowConcurrency: true\n  generate:\n    command: [\"bash\", \"-c\"]\n    args: ['helm template \"$ARGOCD_APP_NAME\" -n \"$ARGOCD_APP_NAMESPACE\" -f $(echo \"$ARGOCD_ENV_HELM_VALUES\") . | argocd-vault-plugin generate -']\n  lockRepo: false\n"},"kind":"ConfigMap","metadata":{"annotations":{},"labels":{"app.kubernetes.io/instance":"argocd-gitops-local-cluster"},"name":"cmp-plugin","namespace":"openshift-gitops"}}
  managedFields:
    - manager: argocd-controller
      operation: Update
      apiVersion: v1
      time: '2023-10-02T15:28:41Z'
      fieldsType: FieldsV1
      fieldsV1:
        'f:data': {}
        'f:metadata':
          'f:annotations':
            .: {}
            'f:kubectl.kubernetes.io/last-applied-configuration': {}
          'f:labels':
            .: {}
            'f:app.kubernetes.io/instance': {}
    - manager: Mozilla
      operation: Update
      apiVersion: v1
      time: '2024-10-29T22:40:35Z'
      fieldsType: FieldsV1
      fieldsV1:
        'f:data':
          'f:avp.yaml': {}
data:
  avp.yaml: |
    apiVersion: argoproj.io/v1alpha1
    kind: ConfigManagementPlugin
    metadata:
      name: argocd-vault-plugin
    spec:
      allowConcurrency: true
      generate:
        command: ["bash", "-c"]
        args: ['helm template "$ARGOCD_APP_NAME" -n "$ARGOCD_APP_NAMESPACE" -f $(echo "$ARGOCD_ENV_HELM_VALUES") . | argocd-vault-plugin generate -']
      lockRepo: false

To reproduce

There are few apps in argocd which are failing with the above error and the helm charts havent changed

Expected behavior
The apps on the argocd UI should sync normally and shouldn't go into unknown state throwing the above error

Screenshots/Verbose output

oc rsh -c avp openshift-gitops-repo-server-58fcd5cdc4-s8dmw
sh-4.4$ argocd-vault-plugin version
argocd-vault-plugin v1.18.1 (fc452cdd8d4727b412ce3de61ee0416efd75050d) BuildDate: 2024-06-07T03:17:37Z

If you've tried running argocd-vault-plugin generate with --verbose-sensitive-output to help debug, please include that output here after redacting any secrets.

Additional context