extend.Rmd

---
title: "Extending AzureRMR"
Author: Hong Ooi
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Extending AzureRMR}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{utf8}
---

AzureRMR provides a generic framework for managing Azure resources. While you can use it as provided to work with any Azure service, you may also want to extend it to provide more features for a particular service. This vignette describes the process of doing so.

We'll use examples from some of the other AzureR packages to show how this works.

## Subclass resource/template classes

Create subclasses of `az_resource` and/or `az_template` to represent the resources used by this service. For example, the AzureStor package provides a new class, `az_storage`, that inherits from `az_resource`. This class represents a storage accounts and has new methods specific to storage, such as listing access keys, generating a shared access signature (SAS), and creating a client endpoint object. Here is a simplified version of the `az_storage` class.

```{r, eval=FALSE}
az_storage <- R6::R6Class("az_storage", inherit=AzureRMR::az_resource,

public=list(

    list_keys=function()
    {
        keys <- named_list(private$res_op("listKeys", http_verb="POST")$keys, "keyName")
        sapply(keys, `[[`, "value")
    },

    get_blob_endpoint=function(key=self$list_keys()[1], sas=NULL)
    {
        blob_endpoint(self$properties$primaryEndpoints$blob, key=key, sas=sas)
    },

    get_file_endpoint=function(key=self$list_keys()[1], sas=NULL)
    {
        file_endpoint(self$properties$primaryEndpoints$file, key=key, sas=sas)
    }
))
```

In most cases, you can rely on the default `az_resource$initialize` method to handle object construction. You can override this method if your resource class contains new data fields that have to be initialised.

A more complex example of a custom class is the `az_vm_template` class in the AzureVM package. This represents the resources used by a virtual machine, or cluster of virtual machines, in Azure. The initialisation code not only handles the details of deploying or getting the template used to create the VM(s), but also retrieves the individual resource objects themselves.

```{r, eval=FALSE}
az_vm_template <- R6::R6Class("az_vm_template", inherit=AzureRMR::az_template,

public=list(
    disks=NULL,
    status=NULL,
    ip_address=NULL,
    dns_name=NULL,
    clust_size=NULL,

    initialize=function(token, subscription, resource_group, name, ...)
    {
        super$initialize(token, subscription, resource_group, name, ...)

        # fill in fields that don't require querying the host
        num_instances <- self$properties$outputs$numInstances
        if(is_empty(num_instances))
        {
            self$clust_size <- 1
            vmnames <- self$name
        }
        else
        {
            self$clust_size <- as.numeric(num_instances$value)
            vmnames <- paste0(self$name, seq_len(self$clust_size) - 1)
        }

        private$vm <- sapply(vmnames, function(name)
        {
            az_vm_resource$new(self$token, self$subscription, self$resource_group,
                type="Microsoft.Compute/virtualMachines", name=name)
        }, simplify=FALSE)

        # get the hostname/IP address for the VM
        outputs <- unlist(self$properties$outputResources)
        ip_id <- grep("publicIPAddresses/.+$", outputs, ignore.case=TRUE, value=TRUE)
        ip <- lapply(ip_id, function(id)
            az_resource$new(self$token, self$subscription, id=id)$properties)

        self$ip_address <- sapply(ip, function(x) x$ipAddress)
        self$dns_name <- sapply(ip, function(x) x$dnsSettings$fqdn)

        lapply(private$vm, function(obj) obj$sync_vm_status())
        self$disks <- lapply(private$vm, "[[", "disks")
        self$status <- lapply(private$vm, "[[", "status")

        NULL
    }

    # ... other VM-specific methods ...
),

private=list(
    # will store a list of VM objects after initialisation
    vm=NULL

    # ... other private members ...
)
))
```


## Add accessor functions

Once you've created your new class(es), you should add accessor functions to `az_resource_group` (and optionally `az_subscription` as well, if your service has subscription-level API calls) to create, get and delete resources. This allows the convenience of _method chaining_:

```{r, eval=FALSE}
res <- az_rm$new("tenant_id", "app_id", "secret") $
    get_subscription("subscription_id") $
    get_resource_group("resgroup") $
    get_my_resource("myresource")
```

Note that if you are writing a package that extends AzureRMR, these methods _must_ be defined in the package's `.onLoad` function. This is because the methods must be added at runtime, when the user loads your package, rather than at compile time, when it is built or installed.

The `create_storage_account`, `get_storage_account` and `delete_storage_account` methods from the AzureStor package are defined like this. Note that calls to your class methods should include the `pkgname::` qualifier, to ensure they will work even if your package is not attached.

```{r, eval=FALSE}

# all methods adding methods to classes in external package must go in .onLoad
.onLoad <- function(libname, pkgname)
{
    AzureRMR::az_resource_group$set("public", "create_storage_account", overwrite=TRUE,
    function(name, location,
             kind="Storage",
             sku=list(name="Standard_LRS", tier="Standard"),
             ...)
    {
        AzureStor::az_storage$new(self$token, self$subscription, self$name,
            type="Microsoft.Storage/storageAccounts", name=name, location=location,
            kind=kind, sku=sku, ...)
    })

    AzureRMR::az_resource_group$set("public", "get_storage_account", overwrite=TRUE,
    function(name)
    {
        AzureStor::az_storage$new(self$token, self$subscription, self$name,
            type="Microsoft.Storage/storageAccounts", name=name)
    })

    AzureRMR::az_resource_group$set("public", "delete_storage_account", overwrite=TRUE,
    function(name, confirm=TRUE, wait=FALSE)
    {
        self$get_storage_account(name)$delete(confirm=confirm, wait=wait)
    })

    # ... other startup code ...
}
```

The corresponding accessor functions for AzureVM's `az_vm_template` class are more complex, as might be imagined. Here is a fragment of that package's `onLoad` function showing the `az_resource_group$create_vm_cluster` method.

```{r, eval=FALSE}
.onLoad <- function(libname, pkgname)
{
    AzureRMR::az_resource_group$set("public", "create_vm_cluster", overwrite=TRUE,
    function(name, location,
             os=c("Windows", "Ubuntu"), size="Standard_DS3_v2",
             username, passkey, userauth_type=c("password", "key"),
             ext_file_uris=NULL, inst_command=NULL,
             clust_size, template, parameters,
             ..., wait=TRUE)
    {
        os <- match.arg(os)
        userauth_type <- match.arg(userauth_type)

        if(missing(parameters) && (missing(username) || missing(passkey)))
            stop("Must supply login username and password/private key", call.=FALSE)

        # find template given input args
        if(missing(template))
            template <- get_dsvm_template(os, userauth_type, clust_size,
                                          ext_file_uris, inst_command)

        # convert input args into parameter list for template
        if(missing(parameters))
            parameters <- make_dsvm_param_list(name=name, size=size,
                username=username, userauth_type=userauth_type, passkey=passkey,
                ext_file_uris=ext_file_uris, inst_command=inst_command,
                clust_size=clust_size, template=template)

        AzureVM::az_vm_template$new(self$token, self$subscription, self$name, name,
            template=template, parameters=parameters, ..., wait=wait)
    })

    # ... other startup code ...
}
```

### Adding documentation

Documenting methods added to a class in this way can be problematic. R's .Rd help format is designed around traditional functions, and R6 classes and methods are usually not a good fit. The popular Roxygen format also (as of October 2018) doesn't deal very well with R6 classes. The fact that we are adding methods to a class defined in an external package is an additional complication.

Here is an example documentation skeleton in Roxygen format, copied from AzureStor. You can add this as a separate block in the source file where you define the accessor method(s). The block uses Markdown formatting, so you will need to have installed roxygen2 version 6.0.1 or later.

```{r, eval=FALSE}
#' Get existing Azure resource type 'foo'
#'
#' Methods for the [AzureRMR::az_resource_group] and [AzureRMR::az_subscription] classes.
#'
#' @rdname get_foo
#' @name get_foo
#' @aliases get_foo list_foos
#'
#' @section Usage:
#' ```
#' get_foo(name)
#' list_foos()
#' ```
#' @section Arguments:
#' - `name`: For `get_foo()`, the name of the resource.
#'
#' @section Details:
#' The `AzureRMR::az_resource_group` class has both `get_foo()` and `list_foos()` methods, while the `AzureRMR::az_subscription` class only has the latter.
#'
#' @section Value:
#' For `get_foo()`, an object of class `az_foo` representing the foo resource.
#'
#' For `list_foos()`, a list of such objects.
#'
#' @seealso
#' [create_foo], [delete_foo], [az_foo]
NULL
```

We note the following:
- The `@aliases` tag includes all the names that will bring up this page when using the `?` command, _including_ the default name.
- Rather than using the standard `@usage`, `@param`, `@details` and `@return` tags, the block uses `@section` to create sections with the appropriate titles (including one named 'Arguments').
- The usage block is explicitly formatted as fixed-width using Markdown backticks.
- The arguments are formatted as a (bulleted) list rather than the usual table format for function arguments.

These changes are necessary because what we're technically documenting is not a standalone function, but a method inside a class. The `@usage`, `@param` tags et al only apply to functions, and if you use them here, `R CMD check` will generate a warning when it can't find a function with the given name. This can be important if you want to publish your package on CRAN.


## Add client-facing interface

The AzureRMR class framework allows you to work with resources at the _Azure_ level, via Azure Resource Manager. If a service exposes a _client_ endpoint that is independent of ARM, you may also want to create a separate R interface for the endpoint.

As the client interface is independent of the ARM interface, you have flexibility to tailor its design. For example, rather than using R6, the AzureStor package uses S3 classes to represent storage endpoints and individual containers and shares within an endpoint. It further defines (S3) methods for these classes to perform common operations like listing directories, uploading and downloading files, and so on. This is consistent with most other data access and manipulation packages in R, which usually stick to S3.

```{r, eval=FALSE}
# blob endpoint for a storage account
blob_endpoint <- function(endpoint, key=NULL, sas=NULL, api_version=getOption("azure_storage_api_version"))
{
    if(!is_endpoint_url(endpoint, "blob"))
        stop("Not a blob endpoint", call.=FALSE)

    obj <- list(url=endpoint, key=key, sas=sas, api_version=api_version)
    class(obj) <- c("blob_endpoint", "storage_endpoint")
    obj
}


# S3 generic and methods to create an object representing a blob container within an endpoint
blob_container <- function(endpoint, ...)
{
    UseMethod("blob_container")
}

blob_container.character <- function(endpoint, key=NULL, sas=NULL,
                                     api_version=getOption("azure_storage_api_version"))
{
    do.call(blob_container, generate_endpoint_container(endpoint, key, sas, api_version))
}

blob_container.blob_endpoint <- function(endpoint, name)
{
    obj <- list(name=name, endpoint=endpoint)
    class(obj) <- "blob_container"
    obj
}


# download a file from a blob container
download_blob <- function(container, src, dest, overwrite=FALSE, lease=NULL)
{
    headers <- list()
    if(!is.null(lease))
        headers[["x-ms-lease-id"]] <- as.character(lease)
    do_container_op(container, src, headers=headers, config=httr::write_disk(dest, overwrite))
}
```