Skip to content

Commit

Permalink
Merge pull request #85 from Azure-Samples/sql_api
Browse files Browse the repository at this point in the history
Adding support for Azure CosmosDB SQL API
  • Loading branch information
chzbrgr71 committed Dec 8, 2023
2 parents 84b2868 + d1bbc9b commit d32e32a
Show file tree
Hide file tree
Showing 9 changed files with 708 additions and 286 deletions.
63 changes: 47 additions & 16 deletions src/makeline-service/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,22 +35,25 @@ export ORDER_QUEUE_NAME=orders
To run this against Azure Service Bus, you will need to create a Service Bus namespace and a queue. You can do this using the Azure CLI.

```bash
az group create --name <resource-group-name> --location <location>
az servicebus namespace create --name <namespace-name> --resource-group <resource-group-name>
az servicebus queue create --name orders --namespace-name <namespace-name> --resource-group <resource-group-name>
RGNAME=<resource-group-name>
LOCNAME=<location>

az group create --name $RGNAME --location $LOCNAME
az servicebus namespace create --name <namespace-name> --resource-group $RGNAME
az servicebus queue create --name orders --namespace-name <namespace-name> --resource-group $RGNAME
```

Once you have created the Service Bus namespace and queue, you will need to create a shared access policy with the **Listen** permission for the namespace.

```bash
az servicebus namespace authorization-rule create --name listener --namespace-name <namespace-name> --resource-group <resource-group-name> --rights Listen
az servicebus namespace authorization-rule create --name listener --namespace-name <namespace-name> --resource-group $RGNAME --rights Listen
```

Next, get the connection information for the Azure Service Bus queue and save the values to environment variables.

```bash
HOSTNAME=$(az servicebus namespace show --name <namespace-name> --resource-group <resource-group-name> --query serviceBusEndpoint -o tsv | sed 's/https:\/\///;s/:443\///')
PASSWORD=$(az servicebus namespace authorization-rule keys list --namespace-name <namespace-name> --resource-group <resource-group-name> --name listener --query primaryKey -o tsv)
HOSTNAME=$(az servicebus namespace show --name <namespace-name> --resource-group $RGNAME --query serviceBusEndpoint -o tsv | sed 's/https:\/\///;s/:443\///')
PASSWORD=$(az servicebus namespace authorization-rule keys list --namespace-name <namespace-name> --resource-group $RGNAME --name listener --query primaryKey -o tsv)
```

Finally, set the environment variables.
Expand Down Expand Up @@ -82,29 +85,57 @@ export ORDER_DB_COLLECTION_NAME=orders

To run this against Azure CosmosDB, you will need to create the CosmosDB account, the database, and collection. You can do this using the Azure CLI.

> Azure CosmosDB supports multiple APIs. This app supports both the MongoDB and SQL APIs. You will need to create the database and collection based on the API you want to use.
```bash
az group create --name <resource-group-name> --location <location>
az cosmosdb create --name <cosmosdb-account-name> --resource-group <resource-group-name> --kind MongoDB
az cosmosdb mongodb database create --account-name <cosmosdb-account-name> --name orderdb --resource-group <resource-group-name>
az cosmosdb mongodb collection create --account-name <cosmosdb-account-name> --database-name orderdb --name orders --resource-group <resource-group-name>
RGNAME=<resource-group-name>
LOCNAME=<location>
COSMOSDBNAME=<cosmosdb-account-name>

az group create --name $RGNAME --location $LOCNAME

# if database requires MongoDB API
# create the database and collection
az cosmosdb create --name $COSMOSDBNAME --resource-group $RGNAME --kind MongoDB
az cosmosdb mongodb database create --account-name $COSMOSDBNAME --name orderdb --resource-group $RGNAME
az cosmosdb mongodb collection create --account-name $COSMOSDBNAME --database-name orderdb --name orders --resource-group $RGNAME

# if database requires SQL API
# create the database and container
COSMOSDBPARTITIONKEY=storeId
az cosmosdb create --name $COSMOSDBNAME --resource-group $RGNAME --kind GlobalDocumentDB
az cosmosdb sql database create --account-name $COSMOSDBNAME --name orderdb --resource-group $RGNAME
az cosmosdb sql container create --account-name $COSMOSDBNAME --database-name orderdb --name orders --resource-group $RGNAME --partition-key-path /$COSMOSDBPARTITIONKEY
```

Next, get the connection information for the Azure Service Bus queue and save the values to environment variables.

```bash
COSMOSDBNAME=<cosmosdb-account-name>
USERNAME=<cosmosdb-account-name>
PASSWORD=$(az cosmosdb keys list --name <cosmosdb-account-name> --resource-group <resource-group-name> --query primaryMasterKey -o tsv)
COSMOSDBUSERNAME=$COSMOSDBNAME
COSMOSDBPASSWORD=$(az cosmosdb keys list --name $COSMOSDBNAME --resource-group $RGNAME --query primaryMasterKey -o tsv)
```

Finally, set the environment variables.

```bash
# if database requires MongoDB API
# set the following environment variables
export ORDER_DB_API=mongodb
export ORDER_DB_URI=mongodb://$COSMOSDBNAME.mongo.cosmos.azure.com:10255/?retryWrites=false
export ORDER_DB_NAME=orderdb
export ORDER_DB_COLLECTION_NAME=orders
export ORDER_DB_USERNAME=$USERNAME
export ORDER_DB_PASSWORD=$PASSWORD
export ORDER_DB_USERNAME=$COSMOSDBUSERNAME
export ORDER_DB_PASSWORD=$COSMOSDBPASSWORD

# if database requires SQL API
# set the following environment variables
export ORDER_DB_API=cosmosdbsql
export ORDER_DB_URI=https://$COSMOSDBNAME.documents.azure.com:443/
export ORDER_DB_NAME=orderdb
export ORDER_DB_CONTAINER_NAME=orders
export ORDER_DB_PASSWORD=$COSMOSDBPASSWORD
export ORDER_DB_PARTITION_KEY=$COSMOSDBPARTITIONKEY
export ORDER_DB_PARTITION_VALUE="pets"
```

> NOTE: With Azure CosmosDB, you must ensure the orderdb database and an unsharded orders collection exist before running the app. Otherwise you will get a "server selection error".
Expand Down Expand Up @@ -182,4 +213,4 @@ db.orders.find()

# get completed orders
db.orders.findOne({status: 1})
```
```
196 changes: 196 additions & 0 deletions src/makeline-service/cosmosdb.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,196 @@
package main

import (
"context"
"encoding/json"
"log"
"strings"

"github.com/Azure/azure-sdk-for-go/sdk/data/azcosmos"
"github.com/gofrs/uuid"
)

type PartitionKey struct {
Key string
Value string
}

type CosmosDBOrderRepo struct {
db *azcosmos.ContainerClient
partitionKey PartitionKey
}

func NewCosmosDBOrderRepo(cosmosDbEndpoint string, dbName string, containerName string, cosmosDbKey string, partitionKey PartitionKey) (*CosmosDBOrderRepo, error) {
cred, err := azcosmos.NewKeyCredential(cosmosDbKey)
if err != nil {
log.Printf("failed to create cosmosdb key credential: %v\n", err)
return nil, err
}

// create a cosmos client
client, err := azcosmos.NewClientWithKey(cosmosDbEndpoint, cred, nil)
if err != nil {
log.Printf("failed to create cosmosdb client: %v\n", err)
return nil, err
}

// create a cosmos container
container, err := client.NewContainer(dbName, containerName)
if err != nil {
log.Printf("failed to create cosmosdb container: %v\n", err)
return nil, err
}

return &CosmosDBOrderRepo{container, partitionKey}, nil
}

func (r *CosmosDBOrderRepo) GetPendingOrders() ([]Order, error) {
var orders []Order

pk := azcosmos.NewPartitionKeyString(r.partitionKey.Value)
opt := &azcosmos.QueryOptions{
QueryParameters: []azcosmos.QueryParameter{
{Name: "@status", Value: Pending},
},
}
queryPager := r.db.NewQueryItemsPager("SELECT * FROM o WHERE o.status = @status", pk, opt)

for queryPager.More() {
queryResponse, err := queryPager.NextPage(context.Background())
if err != nil {
log.Printf("failed to get next page: %v\n", err)
return nil, err
}

for _, item := range queryResponse.Items {
var order Order
err := json.Unmarshal(item, &order)
if err != nil {
log.Printf("failed to deserialize order: %v\n", err)
return nil, err
}
orders = append(orders, order)
}
}
return orders, nil
}

func (r *CosmosDBOrderRepo) GetOrder(id string) (Order, error) {
pk := azcosmos.NewPartitionKeyString(r.partitionKey.Value)
opt := &azcosmos.QueryOptions{
QueryParameters: []azcosmos.QueryParameter{
{Name: "@orderId", Value: id},
},
}
queryPager := r.db.NewQueryItemsPager("SELECT * FROM o WHERE o.orderId = @orderId", pk, opt)

for queryPager.More() {
queryResponse, err := queryPager.NextPage(context.Background())
if err != nil {
log.Printf("failed to get next page: %v\n", err)
return Order{}, err
}

for _, item := range queryResponse.Items {
var order Order
err := json.Unmarshal(item, &order)
if err != nil {
log.Printf("failed to deserialize order: %v\n", err)
return Order{}, err
}
return order, nil
}
}
return Order{}, nil
}

func (r *CosmosDBOrderRepo) InsertOrders(orders []Order) error {
var counter = 0

for _, o := range orders {
pk := azcosmos.NewPartitionKeyString(r.partitionKey.Value)

marshalledOrder, err := json.Marshal(o)
if err != nil {
log.Printf("failed to marshal order: %v\n", err)
return err
}

var order map[string]interface{}
err = json.Unmarshal(marshalledOrder, &order)
if err != nil {
log.Printf("failed to unmarshal order: %v\n", err)
return err
}

// add id with value of uuid.NewV4() to marhsalled order
uuidWithHyphen, err := uuid.NewV4()
if err != nil {
log.Printf("failed to generate uuid: %v\n", err)
return err
}
uuid := strings.Replace(uuidWithHyphen.String(), "-", "", -1)
order["id"] = uuid

order[r.partitionKey.Key] = r.partitionKey.Value

marshalledOrder, err = json.Marshal(order)
if err != nil {
log.Printf("failed to marshal order: %v\n", err)
return err
}

_, err = r.db.CreateItem(context.Background(), pk, marshalledOrder, nil)
if err != nil {
log.Printf("failed to create item: %v\n", err)
return err
}

// increment counter for each order inserted
counter++
}

log.Printf("Inserted %v documents into database\n", counter)

return nil
}

func (r *CosmosDBOrderRepo) UpdateOrder(order Order) error {
var existingOrderId string
pk := azcosmos.NewPartitionKeyString(r.partitionKey.Value)
opt := &azcosmos.QueryOptions{
QueryParameters: []azcosmos.QueryParameter{
{Name: "@orderId", Value: order.OrderID},
},
}
queryPager := r.db.NewQueryItemsPager("SELECT * FROM o WHERE o.orderId = @orderId", pk, opt)

for queryPager.More() {
queryResponse, err := queryPager.NextPage(context.Background())
if err != nil {
break
}

for _, item := range queryResponse.Items {
var order map[string]interface{}
err = json.Unmarshal(item, &order)
if err != nil {
log.Printf("failed to deserialize order: %v\n", err)
return err
}
existingOrderId = order["id"].(string)
break
}
}

patch := azcosmos.PatchOperations{}
patch.AppendReplace("/status", order.Status)

_, err := r.db.PatchItem(context.Background(), pk, existingOrderId, patch, nil)
if err != nil {
log.Printf("failed to replace item: %v\n", err)
return err
}

return nil
}
2 changes: 1 addition & 1 deletion src/makeline-service/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ services:
restart: always
environment:
- ORDER_SERVICE_URL=http://orderservice:3000/
- ORDERS_PER_HOUR=3600
- ORDERS_PER_HOUR=60
networks:
- backend_services
depends_on:
Expand Down
16 changes: 11 additions & 5 deletions src/makeline-service/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,27 @@ module aks-store-demo/makeline-service
go 1.20

require (
github.com/Azure/azure-sdk-for-go/sdk/data/azcosmos v0.3.6
github.com/Azure/go-amqp v1.0.1
github.com/gin-contrib/cors v1.4.0
github.com/gin-gonic/gin v1.9.1
github.com/gofrs/uuid v4.4.0+incompatible
go.mongodb.org/mongo-driver v1.11.7
)

require (
github.com/Azure/azure-sdk-for-go v68.0.0+incompatible // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.9.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.5.1 // indirect
github.com/bytedance/sonic v1.9.1 // indirect
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311 // indirect
github.com/gabriel-vasile/mimetype v1.4.2 // indirect
github.com/gin-contrib/sse v0.1.0 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.14.0 // indirect
github.com/go-playground/validator/v10 v10.16.0 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/gofrs/uuid/v5 v5.0.0 // indirect
github.com/golang/snappy v0.0.1 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.13.6 // indirect
Expand All @@ -36,11 +42,11 @@ require (
github.com/xdg-go/stringprep v1.0.3 // indirect
github.com/youmark/pkcs8 v0.0.0-20181117223130-1be2e3e5546d // indirect
golang.org/x/arch v0.3.0 // indirect
golang.org/x/crypto v0.14.0 // indirect
golang.org/x/net v0.17.0 // indirect
golang.org/x/crypto v0.16.0 // indirect
golang.org/x/net v0.19.0 // indirect
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c // indirect
golang.org/x/sys v0.13.0 // indirect
golang.org/x/text v0.13.0 // indirect
golang.org/x/sys v0.15.0 // indirect
golang.org/x/text v0.14.0 // indirect
google.golang.org/protobuf v1.30.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
Loading

0 comments on commit d32e32a

Please sign in to comment.