Go sdk external data

[jjhwan-h]

hi, I have just started learning about OPA.

I am currently following the Go SDK example (https://www.openpolicyagent.org/docs/latest/integration/#integrating-with-the-go-sdk).

Is there a way to register data into the OPA instance, similar to how it's done via the REST API (https://www.openpolicyagent.org/docs/latest/rest-api/#create-or-overwrite-a-document),
so that it can be used during decision evaluation?

I tried adding an object to the Store as shown in the code below, but it doesn't seem to be the correct approach.

package main

import (
   "bytes"
   "context"
   "encoding/json"
   "fmt"
   "io"
   "log"
   "os"

   "github.com/open-policy-agent/opa/v1/sdk"
   sdktest "github.com/open-policy-agent/opa/v1/sdk/test"
   "github.com/open-policy-agent/opa/v1/storage"
   "github.com/open-policy-agent/opa/v1/storage/inmem"
)

func handleErr(err error) {
   if err != nil {
   	fmt.Println(err)
   	log.Fatal(err)
   }
}

func loadStorage() storage.Store {
   var data map[string]interface{}

   f, err := os.Open("./data.json")
   handleErr(err)
   defer f.Close()

   content, err := io.ReadAll(f)
   handleErr(err)

   decoder := json.NewDecoder(bytes.NewBufferString(string(content)))
   decoder.UseNumber()

   if err := decoder.Decode(&data); err != nil {
   	handleErr(err)
   }

   store := inmem.NewFromObject(data)

   return store
}

func main() {
   ctx := context.Background()

   // write policy
   f, err := os.Open("./policy.rego")
   handleErr(err)
   defer f.Close()

   content, err := io.ReadAll(f)
   handleErr(err)

   // create a mock HTTP bundle server
   server, err := sdktest.NewServer(sdktest.MockBundle("/bundles/bundle.tar.gz", map[string]string{
   	"policy.rego": string(content),
   }))
   handleErr(err)
   defer server.Stop()

   config := []byte(fmt.Sprintf(`{
   	"services": {
   		"test": {
   			"url": %q
   		}
   	},
   	"bundles": {
   		"test": {
   			"resource": "/bundles/bundle.tar.gz"
   		}
   	},
   	"decision_logs": {
   		"console": true
   	}
   }`, server.URL()))

   store_category := loadStorage()

   // create an instance of the OPA object
   opa, err := sdk.New(ctx, sdk.Options{
   	ID:     "opa-test-1",
   	Config: bytes.NewReader(config),
   	Store:  store_category,
   })
   handleErr(err)

   defer opa.Stop(ctx)

   f, err = os.Open("./input_sdk.json")
   handleErr(err)
   defer f.Close()

   content, err = io.ReadAll(f)
   handleErr(err)

   var input map[string]interface{}
   err = json.Unmarshal(content, &input)
   handleErr(err)

   if result, err := opa.Decision(ctx, sdk.DecisionOptions{Path: "/policy/allow", Input: input}); err != nil {
   	log.Fatalf("failed to decision: %v", err)
   } else if decision, ok := result.Result.(bool); !ok || !decision {
   } else {
   }

}

[jeyvison]

Hi @jjhwan-h , This is how we do it in our project, i hope it helps:

We load the json data from FS:

if *pDataFilePath != "" {
		data, err = os.ReadFile(*pDataFilePath)
		if err != nil {
			log.Fatalf("Error while reading data file '%s': %s", *pDataFilePath, err)
		}
	}

Then we transform it into a map and create a new Store object from it(we could do it directly from unmarshal but we have some more logic on that map)

var json map[string]interface{}
util.UnmarshalJSON([]byte(data), &json)
data := inmem.NewFromObject(json)

Then we provide the object as we create the rego instance:

rego := rego.New(
   ...
   rego.Store(data),
)

[anderseknert]

@jeyvison's answer is absolutely right for the low-level Rego API. For the high-level SDK that it looks like you're using, the idea is that just as when running a standalone OPA (opa run --server ...), both policies and data is provided via bundles (as fetched from the endpoints in your configuration). So if you want to add data to OPA, you'd add it to your bundle by placing one or more data.json files in the bundle where the data should be injected.