Question: Tanka vision/roadmap

[olegmayko]

Hi,
As far as I understood, Tanka is replacement to ksonnet tool. Unfortunately, I did not use it and I wonder, how do you see the future of tanka. It's related somehow to #193, #75, #101 , #88 . What are minimal set of features would be considered for v1?
Thanks.

[sh0rez]

Hi! Yes, a clear roadmap is definitely a good thing to have.

Until we have come up with something to discuss publicly, these will probably be the focus points:

• Ecosystem: The current Jsonnet ecosystem is poor in comparison to the Helm one for example. Mostly related to the lack of libraries
  • Libraries: These need to be written. To do that, we need to get a guide together how to do that effectively, so that all libraries follow a common pattern
  • (Helm support): Easy win, cause we can use Charts until we have our own ones. Need to evaluate if we can do that in a nice way tho Done!
  • integrate https://github.com/jsonnet-bundler/jsonnet-bundler as tk pkg, so you only need a single binary
• Tooling: Jsonnet language tooling is pretty bad
  • Docs: Most important. Do ASAP. Work in progress: https://github.com/sh0rez/docsonnet
  • LSP: A language server to do getdoc, complete, highlight, etc would be super neat

Of course this is open for discussion and this list can be extended / changed if sensible :D

[sbarzowski]

I fully agree that we should work on this things (as a broader Jsonnet community - only "Helm support" and tk pkg are Tanka-specific.

Libraries: These need to be written. To do that, we need to get a guide together how to do that effectively, so that all libraries follow a common pattern

If you see a need for some specific library feel free to open an issue for that in https://github.com/google/jsonnet. I'm not saying that we will implement it as maintainers, but I think it's good to accumulate these needs and discussions in one place and work on these together as a community.

[olegmayko]

Thanks for answering. I like tanka for simplifying the building process with override feature, reasonable concept of environments and easy to get started tutorial, which simplifying the onboarding process.

Libraries: These need to be written. To do that, we need to get a guide together how to do that effectively, so that all libraries follow a common pattern

It could be hard to find a consensus on common style, just because the ksonnet style is quite different from style used by Bitnami. However, the style/structure used in kube-prometheus is straight-forward and can be used as a good reference; considering the number of people using/contributing to it.
I think the main issue is how/to create a "hub" similar to helm or terraform with a governance and a reliable support for community. I may assume that's why this project has been started.

integrate https://github.com/jsonnet-bundler/jsonnet-bundler as tk pkg, so you only need a single binary

It's nice to have but easy to overcome with make.
How about enhancing the deploying stage with features such as: "control/order flow" and a prune of the old resources?

LSP: A language server to do getdoc, complete, highlight, etc would be super neat

Totally agree.

If you see a need for some specific library feel free to open an issue for that in https://github.com/google/jsonnet.

@sbarzowski I'd like to see this merged google/jsonnet#665

[wdma]

I am learning Tanka now and I really happy!

Eventually, I would like to be able to contribute but, before that, would it be possible to add more documentation? In particular, I am interested in the how to use the utils. I find that I am guessing at what templates are available base on what is present in the broader Kubernetes ecosystem.

[Duologic]

You might be better of asking on Slack or mailing lists.

[bewest]

I've been enjoying tanka, and the jsonnet-libs. Been trying to evaluate kubecfg and tanka and this thread seemed to sum up some of my experience with tanka. How does tanka play with gitops and ci/cd or other tools in general? Is export meant to output the files for another tool to consume? kubernetes says the objects are managed by kubectl; does this suggest its part of a classic unix pipeline? However piping show to other tools requires a long switch; does this suggest tanka prefers some prioritized relationship to managing the workflow? Wondering how much to expect tanka to interop more with the system around it vs focusing on an interactive terminal, vs maintaining the utility, backwards compatibility, and workflow surrounding jsonnet-libs, k, et al? kubecfg doesn't seem to work with the jsonnet-lib stuff, but does facilitate getting started with components using yaml or json files as input (kubecfg show -o json previously_formatted.yaml is a good way to seed new jsonnet lib ready for factorizing). It's also been tricky dealing with different components that will create or consume different namespaces within the same environment. Is cloning lots of duplicate environments to manage different components expected?

I realize @sh0rez has been working on much needed documentation efforts. These are some of the questions rattling around after spending the summer with tanka. It's been great in assuring me that my resources are linked together appropriately and very much filling needs in this domain.

[sh0rez]

Hey @bewest, let me try to answer some of your questions.

How does tanka play with gitops and ci/cd or other tools in general? Is export meant to output the files for another tool to consume?

Yes it does! The tk export command is specifically meant to be used for handing the YAML resources to some sort of CD tooling. We pilot https://fluxcd.io/ internally and have promising results. @captncraig can tell more on that.

kubernetes says the objects are managed by kubectl; does this suggest its part of a classic unix pipeline? However piping show to other tools requires a long switch; does this suggest tanka prefers some prioritized relationship to managing the workflow

Tanka uses kubectl, not the go-library for cluster communication. Piping however is discouraged for safety reasons: kubectl must be pointed at a given cluster, which resources are sent do when doing kubectl apply. Not being careful about this can have fatal outcomes, e.g. when you apply to the wrong cluster. This happened before and wiped a cluster.
To combat this, tk apply and tk diff invoke kubectl with the --context flag, to force a context that matches the cluster of spec.json. However, when you pipe tk show to kubectl apply, Tanka can't do this for you. The long switch makes this responsibility clear.
In day-to-day operations there should be no need for piping tk show. If you want to export for CD, use tk export. For everything else (diff, apply, delete) Tanka has controlled commands. Does this make sense?

Wondering how much to expect tanka to interop more with the system around it vs focusing on an interactive terminal, vs maintaining the utility, backwards compatibility

Not sure whether I understand this question fully.

The Tanka workflow is primarily interactive: You edit your Jsonnet, tk diff it to verify the outcome and tk apply to make it happen. For automated deployments, tooling is available, notably FluxCD and ArgoCD. tk export provides a bridge to those.
We strongly value backwards compatibility, no core area of Tanka should ever break when upgrading.

and workflow surrounding jsonnet-libs, k, et al?

Let's try to clarify on the Kubernetes libraries.

In the beginning it was ksonnet-lib, which was a Jsonnet library for quickly piecing together Kubernetes objects. When the Heptio was acquired by VMWare, ksonnet was deprecated and thus all development stopped.

Due to a lack of alternative, we stayed on this library, even after we released Tanka as a replacement for ksonnet. Around February 2020, during FOSDEM, @metalmatze and I forked ksonnet-lib to https://github.com/kube-jsonnet/k, aiming to maintain it. Unfortunately we realized that the code-base was far more complex than we had time for to understand to be able to fix issues.

We re-wrote the generator, which formed https://github.com/jsonnet-libs/k8s. This library serves us well for over 2 months in prod now, and will be default in Tanka by the time we declare it GA.

A notable thing here is kausal.libsonnet, which is an extension on top of ksonnet-lib to do some new things we needed for ourselves. Most of what kausal provides is already merged into jsonnet-libs/k8s, so soon there will be only one library left.

kubecfg doesn't seem to work with the jsonnet-lib stuff, but does facilitate getting started with components using yaml or json files as input (kubecfg show -o json previously_formatted.yaml is a good way to seed new jsonnet lib ready for factorizing). It's also been tricky dealing with different components that will create or consume different namespaces within the same environment. Is cloning lots of duplicate environments to manage different components expected?

I can't tell too much about kubecfg, never really used it.

On Environments, they should usually be subset of a namespace. Say a environments/loki/us-central1 would deploy into the loki namespace of a us-central1 cluster. The reason resources are allowed into different namespaces is the fact that some operators require resources in certain namespaces they watch.

As this goes quite into detail, also requiring more information on your setup to answer properly, I suggest you join the #tanka channel on https://grafana.slack.com and ping the there!

[zerok]

Hi :) After nearly 4 years I think we can close this ticket. As far as Tanka's future: We are using it heavily internally and as far as I can tell that's not going to change in the foreseeable future.

Regarding a 1.0 roadmap: We don't have one (yet) but that might still come 🙂