OpenShift Container Platform is built on top of Kubernetes, and offers a consistent hybrid cloud foundation for building and scaling containerized applications. To install standalone KFServing on OpenShift, the easiest way is to use the quick_install.sh
script provided in this repository. This assumes you do not already have Istio and Knative running on your cluster.
Note: These instructions were tested on OpenShift 4.5.15, with KFServing 0.4.1, Istio 1.6.2, and Knative 0.15.0 which are in the quick install script. Additionally, we have tested it with Kubeflow 1.2 recommended versions for Istio and Knative, i.e. Istio 1.3.1 and Knative 0.14.3.
git clone https://github.com/kubeflow/kfserving
Run the following to enable containers to run with UID 0 for Istio’s service accounts, as recommended on Istio's installation instructions for OpenShift
oc adm policy add-scc-to-group anyuid system:serviceaccounts:istio-system
From the root of the kfserving
directory, execute the following:
./hack/quick_install.sh
This script will install Istio, Knative, Cert Manager, and then finally KFServing.
Check that the KFserving controller is running:
oc get po -n kfserving-system
NAME READY STATUS RESTARTS AGE
kfserving-controller-manager-0 2/2 Running 0 2m28s
After installation is verified, expose an OpenShift route for the ingress gateway.
oc -n istio-system expose svc/istio-ingressgateway --port=http2
Now, create an inference service. From the root of the kfserving
directory, run:
oc create ns kfserving-test
oc apply -f docs/samples/sklearn/sklearn.yaml -n kfserving-test
Give it a minute, then check the InferenceService status:
oc get inferenceservices sklearn-iris -n kfserving-test
NAME URL READY DEFAULT TRAFFIC CANARY TRAFFIC AGE
sklearn-iris http://sklearn-iris.kfserving-test.example.com True 100 3m37s
Once the InferenceService is ready, try curling it for a prediction:
export INGRESS_HOST=$(oc get route istio-ingressgateway -n istio-system -ojsonpath='{.spec.host}')
export INGRESS_PORT=$(oc -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')
export SERVICE_HOSTNAME=$(oc get inferenceservice sklearn-iris -n kfserving-test -o jsonpath='{.status.url}' | cut -d "/" -f 3)
curl -v -H "Host: ${SERVICE_HOSTNAME}" http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/sklearn-iris:predict -d @./docs/samples/sklearn/iris-input.json
You should see a prediction output like:
{"predictions": [1, 1]}