New vpc/subnet tag feature from #1170 clobbers existing tags when adopting existing resources

[jlee-idbydna]

What happened?

In PR #1170,
adoption of existing VPC or Subnets via external-name annotation will clobber existing tags. This is especially problematic when adopting resources with aws: prefixed tags, such as EKSCTL created VPCs/Subnets.

I expect this behavior to be optional as stated in the PR. Currently there does not appear to be a simple way to bypass this behavior.

How can we reproduce it?

Adopt any existing VPC or Subnet with pre-existing tags, via external-name annotation.

What environment did it happen in?

Crossplane version: 1.7.0
provider-aws version: 0.25.0
EKS/Kubernetes: 1.21
OS: Amazon Linux 2

[haarchri]

can you add the tags in the manifest definition before importing the resource ?

[jlee-idbydna]

can you add the tags in the manifest definition before importing the resource ?

That is a possible workaround, although somewhat unreasonable with dynamically generated tags for resources provisioned with tools such as EKSCTL. It seems to me the fix is quite simple: to optionally allow a merge instead of an override strategy for tags.

[dlydiard]

This is an issue if importing resources managed by another tool (such as Terraform). The auto-generated Terraform tags are clobbered by crossplane. Duplicating the tags is the manifest is also not a desired option as it forces to duplicate tags that were auto-generated from another tool. Crossplane should leave existing tags untouched and merge in the Crossplane related tags.

In addition, importing AWS subnets that have kubernetes.io/cluster/* tags using Subnet, when using EKS and OpenShift (ROSA) clusters will break networking for kube clusters running on AWS if those tags are removed.

[github-actions]

Crossplane does not currently have enough maintainers to address every issue and pull request. This issue has been automatically marked as stale because it has had no activity in the last 90 days. It will be closed in 14 days if no further activity occurs. Leaving a comment starting with /fresh will mark this issue as not stale.

[dlydiard]

We have a temporary workaround by denying crossplane access to create/delete VPC subnet tags.

        {
            "Effect": "Deny",
            "Action": [
                "ec2:DeleteTags",
                "ec2:CreateTags"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:vpc/*",
                "arn:aws:ec2:*:*:subnet/*"
            ]
        }

The AWS provider seems to be working (importing the resource) even though it can't tag the resource.

[MisterMX]

The general design idea in Crossplane is to have the managed resources as the single source of truth. Unless there are very specific technical reasons to merge external changes with MR settings, the controller should always overwrite settings in the external resource if they are different from the MR.

Preventing tag updates by denying access to the tagging API is not recommended since it leads to reconcile failures and will constantly trigger a rescheduling of that resource.

I am going to close this as this behaviour is by design. The best approach to this issue is to copy the auto-generated tags from the external resource and add them to the MR manually.