To setup CloudGraph in development mode, first clone the CLI repo.
TODO: update to correct url
git clone https://github.com/cloudgraphdev/cli.gitNext, if you are doing updates to an existing provider module, clone that as well. For example cg-provider-aws
git clone https://github.com/cloudgraphdev/cloudgraph-provider-aws.gitcd into the provider repo and run the repos build command. For cg-provider-aws this would be:
yarn build
In order to have the CLI pick up changes you have made locally, you must link the two repos. In the provider repo, run:
yarn linkThe output of yarn link will tell you what command to run within the CLI repo. For example:
yarn link cg-provider-awsNext, make your changes within the provider repo and run yarn build again. And that's it! Now the CLI will pick up your changes when it pulls in the provider client.
To create a new provider, you must create a new NPM module that is publicly available within the NPM registry and conforms to the naming convention @${yourOrgName}/cg-provider-${providerName}. For example @myOrg/cg-provider-pivotal. The module must export a client for your provider that extends the Client class found in @cloudgraph/sdk shown below and defines the functions configure, getSchema, and getData . We will describe what each function should do below.
export default abstract class Provider {
constructor(config: any) {
this.logger = config.logger
this.config = config.provider
}
interface = inquirer
logger: Logger
config: any
async configure(flags: any): Promise<any> {
throw new Error('Function configure has not been defined')
}
getSchema(): string {
throw new Error('Function getSchema has not been defined')
}
async getData({ opts }: { opts: Opts }): Promise<any> {
throw new Error('Function getData has not been defined')
}
}
To add a custom naming for the resources' schemas, export a schemasMap property at the provider. Following the structure:
{
[serviceName]: "schemaName",
}
The configure function is called by @cloudgraph/cli in the INIT command to allow each provider to control its own configuration. This configuration will then be passed to the provider client's constructor as config.provider. The provider client must call super(config) within its constructor to allow the @cloudgraph/sdk client to set the this.config which can then be consumed within the provider. The configure function should return an Object containing all the properties and values the provider wants to allow the end user to set. Here is an example configuration for aws
{
"regions": "us-east-1,us-east-2,us-west-1",
"resources": "alb,lambda,ebs"
}
You may prompt the user to enter values using this.interface which is an instance of Inquirer.js https://github.com/SBoudrias/Inquirer.js
The getSchema function should return the stringified GraphQL schema that will be used by your provider. You can add any valid Dgraph directive to your schema in order to control the results of the schema generated by Dgraph. Below is an example implementation of getSchema used in @cloudgraph/cg-provider-aws.
NOTE: You will only need to define the GraphQL types that describe your schema and Dgraph will automatically generate the queries and mutations to access those types.
/**
* getSchema is used to get the schema for provider
* @returns A string of graphql sub schemas
*/
getSchema(): string {
const typesArray = loadFilesSync(path.join(__dirname), {
recursive: true,
extensions: ['graphql'],
})
return print(mergeTypeDefs(typesArray))
}
The getData function is responsible for collecting and returning all the provider data that you would like to be query-able by the end user. @cloudgraph/cli creates nodes in the graph through the concept of entities and edges in the graph through the concept of connections. entities are the provider data objects themselves as described by the defined GraphQL schema for the provider. connections are objects that describe how the tool should make connections between entities in the provider data. The data structure returned by the getData function should match the ProviderData interface below:
Note: Please see the @cloudgraph/cg-template-provider (TODO: update with link to actual template) for an example on how to create entities and connections for a provider
export interface ServiceConnection {
id: string // The id of the entity to make a connection to
resourceType?: string // [Optional] The name of the connection
relation?: string // [Optional] The relation beteen the entity and its connection
field: string // The property on the parent schema this connected entity should be added to
}
export interface Entity {
name: string, // The name of the entity
mutation: string, // The GraphQL mutation that should be called to push this entity to Dgraph
/**
* An array of the entity data supplied by the provider
* that matches the GraphQL schema of that entity
* (except for connections)
*/
data: any[]
}
export interface ProviderData {
entities: Entity[], // An array of objects matching the Entity interface
/**
* An object where the keys are the ids of parent entities
* to make connections to and where the values are an array of ServiceConnection
* objects denoting which child entities the parent is connected to.
*/
connections: {[key: string]: ServiceConnection[]}
}
To add a new entity (i.e. adding RDS to AWS) to an existing provider, (i.e. @cloudgraph/cg-provider-aws), you must create a new GraphQL sub-schema for that entity. This GraphQL schema should define the type(s) for the new entity and add any directives wanted. You must then define the functions the provider requires to query, format, and form connections for the new entity. In the case of officially supported providers under the @cloudgraph org, this would be done by creating the functions defined in the Service interface below.
NOTE: community supported providers could handle entities differently, consult with the creators of those providers if the way to add new entities is unclear.
export interface Service {
/**
* function that formats an entity to match the GraphQL schema for that entity
*/
format: ({
service,
region,
account,
}: {
service: any
region: string
account: string
}) => any
/**
* [Optional] function that returns the connections for an entity
*/
getConnections?: ({
service,
region,
account,
data,
}: {
service: any
region: string
account: string
data: any
}) => {[key: string]: ServiceConnection[],
mutation: string, // GraphQL mutation used to insert this entity into the DB
/**
* Function to get the RAW entity data from the provider (such as the aws-sdk)
*/
getData: ({
regions,
credentials,
opts,
}: {
regions: string
credentials: any
opts: Opts
}) => any
}
You then must ensure that the getData function for the provider client knows about the new entity. In the case of @cloudgraph/cg-provider-aws this would be done by updating the ServiceMap Object and services.js file to include the new entity. For example, if you created a new entity MyEntity, you would first update the services.js file to include your new entity.
export default {
alb: 'alb',
cloudwatch: 'cloudwatch',
ebs: 'ebs',
...
myEntity: 'myEntity', // The new entity you are adding
...
subnet: 'subnet',
vpc: 'vpc',
}
You would then update the ServiceMap to point to the new entity's class as seen below:
export const ServiceMap = {
[services.alb]: ALB,
[services.cloudwatch]: CloudWatch,
...
[services.myEntity]: MyEntity, // The new entity class you have created
[services.vpc]: VPC,
...
[services.ebs]: EBS,
}
In order to add new data to an existing entity for Officially supported providers, you must update the entity's schema, format function, and getData function. Lets say you have an entity called MyEntity with the following schema, getData and format.
type MyEntity {
id: String!
name: String!
someDataPoint: String
}
function getData() => {
return {
id: 'fakeId',
name: 'fakeName',
someDataFieldToChange: 'isADataPoint'
}
}
function format(rawData) => {
return {
id: rawData.id,
name: rawData.name,
someDataPoint: rawData.someDataFieldToChange
}
}
and you wanted to add a new attribute called myNewData. You would update the entity like so:
type MyEntity {
id: String!
name: String!
someDataPoint: String
myNewData: String // or whatever type the new data is
}
function getData() => {
return {
id: 'fakeId',
name: 'fakeName',
someDataFieldToChange: 'isADataPoint',
myNewData: 'myNewDataToAdd'
}
}
function format(rawData) => {
return {
id: rawData.id,
name: rawData.name,
someDataPoint: rawData.someDataFieldToChange,
myNewData: rawData.myNewData
}
}
And that's it! The CLI will now pick up the new data point and push it to the DB.
If you have any ideas for how to make this contribution guide more effective or easier to work with please let us know, we would love to hear your feedback.