-
|
Hello, After seeing there's a Pkl package for circleci CICD I have tried to get a dynamic pipeline up and running. My goal is to have a minimal circleci config that start a go program which will then generate a dynamic circleci config. I want to do this with Pkl to avoid having to generate a string with the circleci dynamic config. Having read the documentation I believe my best bet to get this up and running is by using the golang language binding in the following way:
However I've faced issues with both step 1 and 2. My issues boil down to not being able to create a Pkl go template from a package. I would be very grateful If someone with more knowledge on the language could guide me to an elegant solution to my problem. I've tried a couple of times but ultimately got stuck with the golang language bindings. Any help would be appreciated. AttemptsAttempt 1: Creating a go template from a circleci package importWhen trying to create a go template by leveraging the already created circleci package I get an error for a missing go module. The error message indicates it might be solved by using package mappings, I got stuck on this error trying different kinds of inputs for the package mapping but didn't manage to get it working. Workflow.pkl @go.Package {
name = "github.com/lopezm94/circleci_monorepo/pkl/lib/modules/cicd/workflow"
}
module lopezm94.circleci_monorepo.pkl.defs.modules.cicd.workflow
import "package://pkg.pkl-lang.org/pkl-go/[email protected]#/go.pkl"
import "package://pkg.pkl-lang.org/pkl-pantry/[email protected]#/Config.pkl" as circleci
...Error: –– Pkl Error ––
No go package defined for module `com.circleci.v2.Config` (https://github.com/apple/pkl-pantry/blob/[email protected]/packages/com.circleci.v2/Config.pkl#L20-20).
One may be defined by either:
* Declaring a `@go.Package` annotation on the module.
* Adding a `--mapping` flag when using the pkl-gen-go command.
* Adding a `packageMappings` clause to generator settings.
38 | ?? throw("""
^^^^^^^^^
at pkl.golang.Generator#getPackageName (https://github.com/apple/pkl-go/tree/v0.8.0/codegen/src/Generator.pkl#L38-L45)
77 | goPackage = getPackageName(it.enclosingDeclaration)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at pkl.golang.Generator#gatherClasses.<function#3>.goPackage (https://github.com/apple/pkl-go/tree/v0.8.0/codegen/src/Generator.pkl#L77-L77)
87 | .groupBy((it) -> it.goPackage)
^^^^^^^^^^^^
at pkl.golang.Generator#packages.<function#1> (https://github.com/apple/pkl-go/tree/v0.8.0/codegen/src/Generator.pkl#L87-L87)
86 | local packages = allMappings
^^^^^^^^^^^
at pkl.golang.Generator#packages (https://github.com/apple/pkl-go/tree/v0.8.0/codegen/src/Generator.pkl#L86-L87)
97 | for (_, package in packages) {
^^^^^^^^
at pkl.golang.Generator#output.files (https://github.com/apple/pkl-go/tree/v0.8.0/codegen/src/Generator.pkl#L97-L97)
1 | output.files.toMap().mapValues((_, it) -> it.text)
^^^^^^^^^^^^
at (repl:text)
pkl/defs/gen.go:3: running "pkl-gen-go": exit status 1Attempt 2: Creating a go template from copied circleci package and parsing an instance from itMy next attempt was downloading the circleci package and hardcode it in a file with the Workflow.pkl: name = "github.com/lopezm94/circleci_monorepo/pkl/lib/modules/cicd/circleci"
}
module lopezm94.circleci_monorepo.pkl.defs.modules.cicd.circleci
import "package://pkg.pkl-lang.org/pkl-go/[email protected]#/go.pkl"
// Contents: "package://pkg.pkl-lang.org/pkl-pantry/[email protected]#/Config.pkl"
// Source: https://github.com/apple/pkl-pantry/blob/com.circleci.v2%401.1.3/packages/com.circleci.v2/Config.pkl
// version: 1.1.3Go program: package main
import (
"context"
"github.com/lopezm94/circleci_monorepo/pkl/lib/modules/cicd/workflow"
)
func main() {
_, err := workflow.LoadFromPath(context.Background(), "pkl/defs/instance/cicd/tofuWorkflow.pkl")
if err != nil {
panic(err)
}
}tofuWorkflow.pkl: import "../../modules/cicd/Workflow.pkl" as circleci //
jobs: Mapping<String(matches(Regex("^[A-Za-z][A-Za-z\\s\\d_-]*$"))), circleci.Job> = new {
["init-validate"] = new circleci.Job {
steps {
"checkout"
new circleci.RunStep {
name = "OpenTofu Init and Validate"
command = """
tofu init
tofu validate
"""
}
}
docker {
new circleci.DockerImage { image = "ghcr.io/opentofu/opentofu:latest" }
}
resource_class = "small"
}
["plan"] = new circleci.Job {
steps {
"checkout"
new circleci.RunStep {
name = "OpenTofu Plan"
command = "tofu plan"
}
}
docker {
new circleci.DockerImage { image = "ghcr.io/opentofu/opentofu:latest" }
}
resource_class = "small"
}
["apply"] = new circleci.Job {
steps {
"checkout"
new circleci.RunStep {
name = "OpenTofu Apply"
command = "tofu apply -auto-approve tfplan"
}
}
docker {
new circleci.DockerImage { image = "ghcr.io/opentofu/opentofu:latest" }
}
resource_class = "small"
}
}
workflows: Mapping<String, circleci.Workflow> = new {
["opentofu-workflow-1"] = new circleci.Workflow {
jobs {
new circleci.WorkflowJob {
name = "init-validate"
}
new circleci.WorkflowJob {
name = "plan"
requires = List("init-validate")
}
new circleci.WorkflowJob {
type = "approval"
name = "approve-apply"
requires = List("plan")
}
new circleci.WorkflowJob {
name = "apply"
requires = List("approve-apply")
}
}
}
["opentofu-workflow-2"] = new circleci.Workflow {
jobs {
new circleci.WorkflowJob {
name = "init-validate"
}
new circleci.WorkflowJob {
name = "plan"
requires = List("init-validate")
}
new circleci.WorkflowJob {
type = "approval"
name = "approve-apply"
requires = List("plan")
}
new circleci.WorkflowJob {
name = "apply"
requires = List("approve-apply")
}
}
}
}Error: panic: –– Pkl Error ––
Cannot find type `Job` in module `lopezm94.circleci_monorepo.pkl.defs.modules.cicd.workflow`.
3 | jobs: Mapping<String(matches(Regex("^[A-Za-z][A-Za-z\\s\\d_-]*$"))), circleci.Job> = new {
^^^
at tofuWorkflow (file:///Users/lopezm/Projects/circleci-monorepo/pkl/defs/instance/cicd/tofuWorkflow.pkl)
goroutine 1 [running]:
main.main()
/Users/lopezm/Projects/circleci-monorepo/cmd/main.go:21 +0xac
exit status 2
Projects/circleci-monorepo - (main) > go run ./cmd
panic: –– Pkl Error ––
Cannot find type `Job` in module `lopezm94.circleci_monorepo.pkl.defs.modules.cicd.workflow`.
3 | jobs: Mapping<String(matches(Regex("^[A-Za-z][A-Za-z\\s\\d_-]*$"))), circleci.Job> = new {
^^^
at tofuWorkflow (file:///Users/lopezm/Projects/circleci-monorepo/pkl/defs/instance/cicd/tofuWorkflow.pkl)
goroutine 1 [running]:
main.main()
/Users/lopezm/Projects/circleci-monorepo/cmd/main.go:20 +0x5c
exit status 2 |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 3 replies
-
|
For your first problem: the error message describes the remediation steps here: The simplest approach is to use the For your second problem: Pkl is telling you that it doesn't know what Food for thought: if you goal is to generate a circleci config, you probably don't need to use codegen. You can write a Pkl program that outputs a CircleCI config based on some parameters. For example, here's a sample program that uses // myDynamicConfig.pkl
amends "package://pkg.pkl-lang.org/pkl-pantry/[email protected]#/Config.pkl"
jobs {
["something"] {
steps {
new RunStep {
command = read("prop:command")
}
}
}
}And you can simply evaluate the output text of this program: pkl eval -p command=cowsay myDynamicConfig.pklOr, if you're using pkl-go: evaluator, _ := pkl.NewEvaluator(context.TODO(), pkl.PreconfiguredOptions, func(options *pkl.EvaluatorOptions) {
options.Properties = map[string]string{
"command": "cowsay",
}
})
defer evaluator.Close()
outText, _ := evaluator.EvaluateOutputText(context.TODO(), pkl.FileSource("myDynamicConfig.pkl"))
fmt.Printf(outText) |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @bioball ! It was very useful, ended up using a mixture of This is because I want to create pipelines dynamically depending on whether a new folder was created in a repository, for this I need to detect in golang the new folders, and create pipelines from that. The only function I might be able to leverage is I think it would be ideal if there was a golang schema to pkl function, or a string split you can inject from a command. Is there a functionality I may be missing from Pkl? |
Beta Was this translation helpful? Give feedback.
-
|
After reading a bit more, it seems properties also support globbing. I think this will be enough. |
Beta Was this translation helpful? Give feedback.
-
|
You can definitely split strings in Pkl! Check out the API docs: https://pkl-lang.org/package-docs/pkl/current/base/String.html#split() |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @HT154. I keep getting surprised that it is actually a full fledged language, has recursion, package docs, etc. I was using the docs website as source of truth until now. |
Beta Was this translation helpful? Give feedback.
For your first problem: the error message describes the remediation steps here:
The simplest approach is to use the
--mappingflag, e.g.pkl-gen-go --mapping com.circleci.v2.Config= github.com/lopezm94/circleci_monorepo/pkl/lib/modules/cicd/circleciFor your second problem: Pkl is telling you that it doesn't know what
Jobis. Does modulelopezm94.circleci_monorepo.pkl.defs.modules.cicd.workflowexport this? Did you make a mistake here, and mean to importpackage://pkg.pkl-lang.or…