Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for generating files from command output #223

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/build_and_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,8 @@ jobs:
kubectl cp sidecar:/tmp/absolute/absolute.txt /tmp/absolute.txt
kubectl cp sidecar:/tmp/relative/relative.txt /tmp/relative.txt
kubectl cp sidecar:/tmp/500.txt /tmp/500.txt || true
kubectl cp sidecar:/tmp/command-output.txt /tmp/command-output.txt
kubectl cp sidecar:/tmp/hostname.txt /tmp/hostname.txt

echo "Downloading resource files from sidecar-5xx..."
kubectl cp sidecar-5xx:/tmp-5xx/hello.world /tmp/5xx/hello.world
Expand All @@ -119,6 +121,8 @@ jobs:
echo -n "This absolutely exists" | diff - /tmp/absolute.txt &&
echo -n "This relatively exists" | diff - /tmp/relative.txt &&
[ ! -f /tmp/500.txt ] && echo "No 5xx file created" &&
echo -n "This generated by script" | diff - /tmp/command-output.txt &&
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the example script this assertions is likely to fail (as the script generates random hex text, doesn't it?)

echo "sidecar" | diff - /tmp/hostname.txt &&
ls /tmp/script_result &&
echo -n "Hello World!" | diff - /tmp/5xx/hello.world &&
diff test/kubelogo.png /tmp/5xx/cm-kubelogo.png &&
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Both are identical multi-arch images built for `amd64`, `arm64`, `arm/v7`, `ppc6
- Update/Delete on change of configmap or secret
- Enforce unique filenames

# Usage
# Usage

Example for a simple deployment can be found in [`example.yaml`](./example.yaml). Depending on the cluster setup you have to grant yourself admin rights first:
```shell
Expand All @@ -48,6 +48,7 @@ metadata:
```

If the filename ends with `.url` suffix, the content will be processed as a URL which the target file contents will be downloaded from.
If the filename ends with `.command` suffix, the content will be processed as a shell command which will be executed. Stdout of the command will be stored in the file.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a small example (as yaml block) showing the 3 different variants here?
In addition we should add some words under which circumstances the script gets executed (and the http request gets made). I believe this is now "on creation and each (even unrelated) update of the containing configmap or secret".


## Configuration Environment Variables

Expand Down
14 changes: 13 additions & 1 deletion example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,19 @@ data:
# base64 encoded: my super cool \n multiline \ secret
secret.world: bXkgc3VwZXIgY29vbAptdWx0aWxpbmUKc2VjcmV0
---
apiVersion: v1
kind: ConfigMap
metadata:
name: output-of-command
labels:
findme: "yup"
data:
rand.sh: |
#!/bin/sh
dd if=/dev/random bs=4 count=1 | hexdump -v -e '/1 "%02X"'
random.txt.command: '/tmp/rand.sh'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this example is only reliable if the key-values of a configmap are actually sorted when accessed through the python client. Otherwise occasionally the command is executed before the shell script is stored. Could you double check this?

hostname.txt.command: '/bin/hostname'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
Expand All @@ -97,4 +110,3 @@ subjects:
- kind: ServiceAccount
name: sample-acc
namespace: default

6 changes: 4 additions & 2 deletions src/helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -158,11 +158,13 @@ def unique_filename(filename, namespace, resource, resource_name):
def execute(script_path):
logger.debug(f"Executing script from {script_path}")
try:
result = subprocess.run(["sh", script_path],
result = subprocess.run(script_path, shell=True,
capture_output=True,
check=True)
check=True,
text=True)
logger.debug(f"Script stdout: {result.stdout}")
logger.debug(f"Script stderr: {result.stderr}")
logger.debug(f"Script exit code: {result.returncode}")
except subprocess.CalledProcessError as e:
logger.error(f"Script failed with error: {e}")
return result
10 changes: 7 additions & 3 deletions src/resources.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,15 +43,18 @@ def signal_handler(signum, frame):
signal.signal(signal.SIGTERM, signal_handler)


def _get_file_data_and_name(full_filename, content, enable_5xx, content_type=CONTENT_TYPE_TEXT):
def _get_file_data_and_name(full_filename, content, enable_5xx, content_type=CONTENT_TYPE_TEXT, remove=False):
if content_type == CONTENT_TYPE_BASE64_BINARY:
file_data = base64.b64decode(content)
else:
file_data = content

if full_filename.endswith(".url"):
if full_filename.endswith(".url") and not remove:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change breaks removal as the returned filename would still contain the suffix .url which is not a part of the filename. To avoid running the request (and the script further down) on removals only setting file_data should be skipped, not filename.

filename = full_filename[:-4]
file_data = request(file_data, "GET", enable_5xx).text
elif full_filename.endswith(".command") and not remove:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comment above

filename = full_filename[:-8]
file_data = execute(file_data).stdout
else:
filename = full_filename

Expand Down Expand Up @@ -187,7 +190,8 @@ def _update_file(data_key, data_content, dest_folder, metadata, resource,
filename, file_data = _get_file_data_and_name(data_key,
data_content,
enable_5xx,
content_type)
content_type,
remove)
if unique_filenames:
filename = unique_filename(filename=filename,
namespace=metadata.namespace,
Expand Down
13 changes: 13 additions & 0 deletions test/resources/resources.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,16 @@ metadata:
findme: "yup"
data:
500.txt.url: "http://dummy-server/500"
---
apiVersion: v1
kind: ConfigMap
metadata:
name: command-configmap
labels:
findme: "yup"
data:
script.sh: |-
#!/bin/sh
echo -n "This generated by script"
command-output.txt.command: '/tmp/script.sh'
hostname.txt.command: '/bin/hostname'