Comments and Documentation not sufficient for implementing new cloud environments.

[deathly809]

We want to add support for AzureStack, Microsoft's hybrid cloud. However the documentation in the Wiki on doing this is not clear. Also, all the comments are TBD for each cloud operation.

[maugustosilva]

Hello... yes, this documentation is a work in progress, certainly! Question: do you intend to add this adapters through libcloud or with a native python client?

[deathly809]

Native python. I am trying to guess on what some of the functions do. And also what should be set and what you should expect to be passed in.

[maugustosilva]

Alright, I have started populating the short description in https://github.com/ibmcb/cbtool/wiki/FAQ-D with a little bit more detail, regarding the different methods. I was busy with other projects the past week, but should be free to quickly answer any questions that you have this coming week.

[deathly809]

Awesome, I just back to work from Surgery so I will be in and out for the week. I will take a look and let you know if I have any questions.

[deathly809]

I have started a blank template here. The only question I have is how do you know what will is required/optional in obj_attr_list? Also, how do you know what to set in it as well?

Also, is there any documentation on the config/templates? I am not 100% sure on what some of the properties mean.

[maugustosilva]

What is required and what is optional on the obj_attr_list is cloud-dependent. This dictionary gets populated in https://github.com/ibmcb/cbtool/blob/master/lib/operations/base_operations.py#L925, with the values that it reads from the config/templates. Documentation on the config/templates, is I will freely admit, non-existent at the moment (it has been on my todo list for a while).

[deathly809]

So I "think" I have everything filled in but it is complaining with this,

unknown error: Your configuration file is missing the following configuration:
        [USER-DEFINED : CLOUDOPTION_TESTAZSCLOUD]
        AZS_LOGIN = XXXXX

Do you know what this is and where it comes from? I saw similar configuration type files under configs/regression but I don't see any for the current Azure configuration.

[maugustosilva]

Hmmm... just making sure that you have a sane configuration. So, taking SoftLayer as an example... https://github.ibm.com/marcios/cloudbench/blob/master/configs/templates/_slr.txt... every attribute that has a value of need_to_be_configured_by_user (e.g. https://github.ibm.com/marcios/cloudbench/blob/master/configs/templates/_slr.txt#L7) needs to be present on your private configuration file while trying to attach a cloud...

For reference, here is the example private configuration file for a SoftLayer cloud: https://github.ibm.com/marcios/cloudbench/blob/master/configs/cloud_definitions.txt#L94

Does it help?

[deathly809]

I can't access those links.

[maugustosilva]

Argh, my apologies, I used the links from our internal repo! Here it goes again, with the correct links:

So, taking SoftLayer as an example... https://github.com/ibmcb/cbtool/blob/master/configs/templates/_slr.txt ... every attribute that has a value of need_to_be_configured_by_user (e.g. https://github.com/ibmcb/cbtool/blob/master/configs/templates/_slr.txt#L7) needs to be present on your private configuration file while trying to attach a cloud...

For reference, here is the example private configuration file for a SoftLayer cloud: https://github.com/ibmcb/cbtool/blob/master/configs/cloud_definitions.txt#L85

Does it help?

[deathly809]

I filled those in for AzureStack [link]. Where does CLOUDOPTION_TESTAZSCLOUD come from?

[maugustosilva]

That comes from https://github.com/deathly809/cbtool/blob/8ef87595f3ce9631e8a4d736c3d18c5607392288/configs/cloud_definitions.txt#L24

Also, make sure that the STARTUP_CLOUD in https://github.com/deathly809/cbtool/blob/8ef87595f3ce9631e8a4d736c3d18c5607392288/configs/cloud_definitions.txt#L20 is MYAZS

[deathly809]

I am trying to convert the sizes in the Azure text file to more modern values, but I don't know what size:1gb is, in fact I can't find any reference to it in LibCloud.

[maugustosilva]

A mistake (from my part, when I copied from Digital Ocean).... please use "Large"....

[deathly809]

Will make the change. Also, there are a few functions that are required but not listed in the Appendix as required.

get_ssh_keys
get_security_groups
create_ssh_key

EDIT:

Also, when I create a VM how can I tell if I should use a private vs. public IP address? I don't think I can create a public IP address for every VM it might want to create. Or does CBTool on create a single VM for each benchmarch?

[maugustosilva]

Hello again. Thanks will add all 3 functions to the list of mandatory functions.

Regarding the public vs private IPs, please let me know if this answers your question: https://github.com/ibmcb/cbtool/wiki/FAQ-S#sq6

I must also point out that, for the current libcloud-based Azure adapter, the behavior seems to be, when the first instance is created, a new Public IP gets allocated, and subsequent instances are SSH-accessible through a different port using the same Public IP. (e.g., https://github.com/ibmcb/cbtool/blob/master/lib/clouds/libcloud_common.py#L710)

[deathly809]

Thanks, I will take a look. Does CBTOOL know to use the different ports? How would I report that information to CBTOOL?

I have some other questions on things that are not clear.

1. On first load you instantiate a cloud object and test the VMC connection and create some SSH keys. However, you don't reuse this object so I cannot store the keys in the object which requires me to save them in KeyVault. Is this the expected flow? EC2 automatically creates their keys so they don't have to worry about this as they are always available.
2. I am not sure what vmccleanup should do. It is called on the initial vmcregister but if I do cleanup all resources the SSH keys are lost and therefore cannot connect to any VMs. If I don't delete resources here where should I delete all my resources? If I am supposed to delete resources here how do I get SSH keys from CBTOOL?
3. It is not clear what an image means for the functions get_images/check_images. In Azure/AzureStack tenants cannot upload custom images, they can only upload VHDs and then attach VMs to those. I am also not sure on how once I prepare a VM I can make CBTOOL work, unless I overload CBTOOL to look for VHDs in a storage account.
4. When I create a VM with a public IP address and try to prepare it using cbssh I get an error that the IP address is blank. Is there some value I need to be setting? I am following EC2 pretty close.
5. Is there a way to automate CBTOOL including VM preparation?

[maugustosilva]

CBTOOL will know use whatever port in is specified on prov_cloud_port to ssh into the instances... an example here https://github.com/ibmcb/cbtool/blob/master/lib/clouds/libcloud_common.py#L712

Regarding the other question, here are my answers:

1. Hmmm... If I understood your question properly, I believe I had a similar problem writing the Google Compute Engine adapter. My solution on that case was to explicitly add the keys to the metadata section of each instance created... example: https://github.com/ibmcb/cbtool/blob/master/lib/clouds/gce_cloud_ops.py#L1017

2. vmccleanup should remove only Instances and Volumes (whose names have the "cb-" + obj_attr_list["username"] + '-' + obj_attr_list["cloud_name"].lower() prefix) left from previous executions (i.e., CBTOOL processes were killed and no vmdetach all/aiattach all were issued). It should not delete SSH keys.

3. You mean, they cannot upload bootable custom images? That seems so restrictive! Anyway, if there is a list of images obtainable through an API call, then purpose of check_images is to basically ensure - during the initial cloud attachment - that every image specified on the cloud configuration file exists (and has a corresponding UUID from the cloud provider), while the goal of get_images is to ensure, before an instance creation is issued, that the image UUID specified on the CBTOOL's VM attribute imageid1 is present on the cloud.

4. If the attributes cloud_ip, run_cloud_ip and prov_cloud_ip are populated, then you should have no problem invoking cbssh.

5. Hmmm... there is an automated installer for both Orchestrator and Workload images: https://github.com/ibmcb/cbtool/wiki/HOWTO:-Preparing-a-VM-to-be-used-with-CBTOOL-on-a-real-cloud