Setup checker pipeline on GitHub for users of ASAM standards

[andreaskern74]

Is your feature request related to a problem? Please describe.

The framework is designed also to run in a pipeline. This should be demonstrated.

Describe the solution you'd like

Setup a pipeline based on GitHub actions including running the Checker Library for OpenDRIVE, OpenSCENARIO XML or OTX in a standard configuration.

The user can upload its file, the pipeline should execute the selected Checker Library and the user can download the text based report and the machine-readable result file.

Additional idea: Provide a GitHub Action, which can be used by users pipelines (e.g. Open MSL).

Describe alternatives you've considered

Web-only user interface and runtime #21 --> this needs a lot of things to do before, so this feature request is starting point for running the checks in an environment, which don't need a installation.

[hoangtungdinh]

The following options were discussed for the demo pipeline.

Running the demo pipeline on ASAM public repo

In this option, we host the demo pipeline using workflow_dispatch, which can be triggered manually.

Pros

• Simple to configure
• Users don't need to execute anything locally

Cons

• Only users with "write" permission in the repo can trigger the job
• Users need to upload the input file somewhere and pass the public URL of the input file as the input for the workflow.
• The execution outputs are visible to everyone.

Running the demo pipeline on users' forked repo

In this option, we still create a demo pipeline using workflow_dispatch, but users will fork the public repo and run the GitHub action on their repo.

Pros

• Users don't need to execute anything locally
• Users can execute the demo with private data hosted in a private repo, by adding their github secret to their forked repo.

Cons

• In case of a public fork: Users can synchronize the fork with the main repo via web interface, but the output of actions will be visible to everyone.
• In case of a private fork: Users will need to synchronize the fork and the main repo using CLI, which is more complicated and requires more knowledge
• Users need to be familiar with Github, have Github account and have Github "credits" to run Actions.

Running as a web application

In this option, the QC Framework will be wrapped by a web application running serverless.

Pros

• Very simple for users to use

Cons

• Need a server
• Need to maintain the server and the web app. As most users of the QC Framework are not familiar with web app development and deployment, it can be a challenge.

Running as a docker container locally

In this option, we automatically build a docker image on our public git repo and push it to GitHub container registry (free for public repos). The docker image contains the demo pipeline. Users can use this docker image to run the pipeline on their local machine with a single command:

docker run —rm -v /user_input_path:/input_path ghcr.io/asam-ev/qc-framework:latest

Pros

• Simple for users to run locally

Cons

• Docker needs to be installed on users' local computers.
• Users need to interact with CLI.

Current conclusion

After evaluating all the trade-offs, we are currently in favor of the option Running as a docker container locally. Feedbacks and ideas are welcome.

[pmai]

Closed with merge of #86.