Add runPrivileged/runAsUser options, for running on more restricted/s…

…ecured K8s clusters

These options make it possible to run the operator and es-clusters on a Kubernetes cluster,
that has Pod Security Policies in place, that:
- disallows running containers as root
- (and/or) disallows running containers in privileged mode

Note: the default elasticsearch image (upmcenterprises/docker-elasticsearch-kubernetes:6.1.3_0 as of writing)
will not work if you don't run it as root (uid 0), as its wrapper script tries a 'ulimit -l unlimited',
and eventually su-execs to elasticsearch user with uid 1000; both actions will fail.
Setting ulimit should not be neccessary with IPC_LOCK/SYS_RESOURCE capabilities,
however they get wiped when running a container as non-root.
So running this image will require some modifications,
e.g. chowning folders, setcap cap_ipc_lock=+ep on java binary + dependencies.

Most (recent) information on this topic that allowed me to solve the puzzle:
https://medium.com/@thejasongerard/resource-limits-mlock-and-containers-oh-my-cca1e5d1f259

Change-Id: I600e9dd4a49cab15a289fc50cc2a605c83ac3aa9

pkg/k8sutil/k8sutil_test.go

@@ -39,8 +39,10 @@ func TestSSLCertConfig(t *testing.T) {
 	}
 	clusterName := "test"
 	useSSL := false
+	runPrivileged := true
+	var runAsUser int64 = 0
 	statefulSet := buildStatefulSet("test", clusterName, "master", "foo/image", "test", "1G", "",
-		"", "", "", nil, &useSSL, resources, nil, "")
+		"", "", "", nil, &useSSL, resources, nil, "", &runPrivileged, &runAsUser)
 
 	for _, volume := range statefulSet.Spec.Template.Spec.Volumes {
 		if volume.Name == fmt.Sprintf("%s-%s", secretName, clusterName) {
@@ -50,7 +52,7 @@ func TestSSLCertConfig(t *testing.T) {
 
 	useSSL = true
 	statefulSet = buildStatefulSet("test", clusterName, "master", "foo/image", "test", "1G", "",
-		"", "", "", nil, &useSSL, resources, nil, "")
+		"", "", "", nil, &useSSL, resources, nil, "", &runPrivileged, &runAsUser)
 
 	found := false
 	for _, volume := range statefulSet.Spec.Template.Spec.Volumes {