Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add runner debug mode for extra logging #1164

Closed
elliotgunton opened this issue Aug 21, 2024 · 1 comment
Closed

Add runner debug mode for extra logging #1164

elliotgunton opened this issue Aug 21, 2024 · 1 comment
Labels
type:enhancement A general enhancement

Comments

@elliotgunton
Copy link
Collaborator

elliotgunton commented Aug 21, 2024

Is your feature request related to a problem? Please describe.
I've run into a problem when trying to load an artifact from S3 and using the runner, but the Artifact path comes out as None:

FileNotFoundError: [Errno 2] No such file or directory: '/tmp/hera-inputs/artifacts/None'
@script(constructor="runner")
def load_and_split_dataset(
    dataset_path: Annotated[
        Path,
        S3Artifact(
            endpoint="minio:9000",
            bucket="my-bucket",
            key="dataset.csv",
            access_key_secret=m.SecretKeySelector(
                name="my-minio-cred",
                key="accesskey",
            ),
            secret_key_secret=m.SecretKeySelector(
                name="my-minio-cred",
                key="secretkey",
            ),
            insecure=True,
            loader=None,
        ),
    ],
):
    print(dataset_path)
    data = pd.read_csv(dataset_path)

Describe the solution you'd like
A clear and concise description of what you want to happen.

I'd like some debug logs from within the runner loading stage which can be turned on via an environment variable on the script.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
printing debug logs myself but they are after the fact when the runner has already tried to load the artifact.

Additional context
Add any other context or screenshots about the feature request here.

The code works with an inline script. When using the runner the Artifact is retrieved as an input correctly, but seemingly not loaded by the runner:

image

@elliotgunton
Copy link
Collaborator Author

Dupe of #972

@elliotgunton elliotgunton closed this as not planned Won't fix, can't repro, duplicate, stale Aug 22, 2024
sambhav pushed a commit that referenced this issue Aug 27, 2024
Prompted by a couple of recent duplicate issues
#1164 and
#1174

Signed-off-by: Elliot Gunton <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:enhancement A general enhancement
Projects
None yet
Development

No branches or pull requests

1 participant