Truncate/Filter Diff Output

[whitleykeith]

I'd like to be able to truncate the output of diffs, specifically when the entire file is getting added/deleted

For instance, we use dyff to check what a PR will do to our K8s cluster. We also have certain workflows that consistently have deletes and creates of new manifests in our cluster. The rendering of those (especially deleted manifests) can be unweidly and generally provides little to no information on the actual change to the cluster.

We don't want to use brief output, because we take the diff of all the manifests at once and do want to see what changes are being made elsewhere. I've tried to use exclude-regexp and filter-regexp, but they didn't work. I think those only apply to excluding those paths from determining a diff and not output. Because deletes/creates are "root" level changes, they always show up unless I ignore all diffs.

A perfect use-case is that I never want to show the status or metadata.managedFields blocks on any diff, ever. Ideally, I could supress output paths --suppress status.*. I would also be fine if there was an output mode for "brief on large diffs"/"brief on new/deleted files".

Is there a way to do this today or is this a feature request?

[vladimir-avinkin]

You can use --exclude-regexp, for example dyff between x y --exclude-regexp '^/metadata.managedFields'

[whitleykeith]

@vladimir-avinkin this doesn't work if the entire file is getting deleted/added.

[oguzhan-yilmaz]

@whitleykeith I was able to get it working like this:

dyff between 1.yaml 2.yaml --exclude-regexp "spec.containers.*,spec.ports.*"
# or
dyff between --ignore-order-changes --exclude-regexp "metadata.labels.helm.sh.*,metadata.labels.app.kubernetes\.io.*"

Don't forget that it's actual regex syntax not UNIX-like. So you'd have to use .+ to select many characters.

[whitleykeith]

@oguzhan-yilmaz doesn't work if the file is getting deleted either. The issue is that the exclude-regexp doesn't filter a deleted file, and things like deleted K8s manifests can be long and not really useful in a dyff output

[HeavyWombat]

Thanks for the input. This is good. I am just thinking about it. Thing is, I wonder if it makes sense to add even more Kubernetes aware behavioral changes. I think I need to think about this a bit. So the idea would basically be that by default, dyff should omit things that do not serve any further insight or are volatile anyway. And you could have a flag that "forces" to show everything.

[whitleykeith]

That'd be great especially since there's already the --detect-kubernetes flag and most of the clutter comes from things like the metadata.managedFields and statusblocks. We could just add a default filter to apply on detected k8s manifests