merging autorest to master

Author: amarzavery

.gitignore

@@ -1,12 +1,19 @@
 projects/*
 targets/*
+packages/*
 
 # Visual Studio #
 *.suo
 *.user
+.ntvs_analysis.dat
+obj/*
+examples/obj
+examples/.ntvs_analysis.dat
+examples/.vs
+examples/bin
+.vs
 
 # Node #
-test/recordings/
 node_modules/
 npm-debug.log
 azure_error
@@ -33,6 +40,9 @@ docs/
 main.conf.json
 child.conf.json
 
+# temporary test artifacts
+test/tmp/*
+
 # Nuget packages and corresponding codegen sources
 .nuget
 packages

.npmignore

@@ -46,3 +46,6 @@ gruntfile.js
 .nuget/
 packages/
 packages.config
+
+# VS #
+.ntvs_analysis.*

.travis.yml

@@ -1,8 +1,9 @@
 language: node_js
+sudo: false
 node_js:
+  - "4.2"
   - "0.12"
   - "0.10"
-  - "0.8"
 
 matrix:
   allow_failures:

Documentation/Authentication.md

@@ -0,0 +1,126 @@
+
+# Creating a ServicePrincipal for scripting scenarios
+One does not want to login interactively all the time. Azure provides service principal authentication as a secure way for silent login.
+
+## Via Portal
+[This article](https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/) provides detailed steps on creating a service principal via portal.
+
+## Via XplatCLI
+Pre-requisite:
+- Install the latest version of cli from the [latest github release](https://github.com/Azure/azure-xplat-cli/releases) or from [npm](https://npmjs.com/package/azure-cli).
+
+```bash
+npm uninstall azure-cli -g
+npm cache clear -g
+npm install azure-cli -g
+```
+- Login to the azure-cli via azure login command to create a service principal for future use.
+  - For 2FA enabled user accounts  `azure login`
+  - For non 2FA enabled user accounts `azure login -u [email protected]`
+
+After successful login, please follow the steps mentioned below to create a serviceprincipal:
+
+#### Step 1. Create an aplication
+
+Let us create an application with a password. The default start-date witll be the day of application creation and the default end-date will be 1 year from the day of creation.
+```
+D:\sdk>azure ad app create -n testap908 home-page http://www.bing.com --identifier-uris https://testap674.com/home -p P@ssw0rd
+info:    Executing command ad app create
++ Creating application testap674
+data:    AppId:                   56894bd4-0fde-41d8-a0d7-5bsslccety2
+data:    ObjectId:                g565675e8-7c30-908t-8548-87r98ew7rej
+data:    DisplayName:             testap674
+data:    IdentifierUris:          0=https://testap674.com/home
+data:    ReplyUrls:
+data:    AvailableToOtherTenants:  False
+data:    AppPermissions:
+data:                             claimValue:  user_impersonation
+data:                             description:  Allow the application to access testap674 on behalf of the signed-in user.
+data:                             directAccessGrantTypes:
+data:                             displayName:  Access testap674
+data:                             impersonationAccessGrantTypes:  impersonated=User, impersonator=Application
+data:                             isDisabled:
+data:                             origin:  Application
+data:                             permissionId:  12345698ui-fa71-4ab0-b647-fdajfhdakfh789
+data:                             resourceScopeType:  Personal
+data:                             userConsentDescription:  Allow the application to access testap674 on your behalf.
+data:                             userConsentDisplayName:  Access testap674
+data:                             lang:
+info:    ad app create command OK
+```
+
+#### Step 2. Create a ServicePrincipal with the applicationId
+
+Let us create a serviceprincipal using the applicationId from the previous step.
+```
+D:\sdk>azure ad sp create --aplicationId 56894bd4-0fde-41d8-a0d7-5bsslccety2
+info:    Executing command ad sp create
++ Creating service principal for application 56894bd4-0fde-41d8-a0d7-5bsslccety2
+data:    Object Id:               weewrerer-e329-4e9b-98c6-7878787
+data:    Display Name:            testap674
+data:    Service Principal Names:
+data:                             56894bd4-0fde-41d8-a0d7-5bsslccety2
+data:                             https://testap674.com/home
+info:    ad sp create command OK
+```
+
+#### Step 3. Assigning a role to a ServicePrincipal by using the spn
+
+- You can get a list of available roles by ```azure role list```
+- In this example we are creating the serviceprincipal as a Contributor at the subscription level.
+  - Definition of a Contributor
+```
+data:    Name             : Contributor
+data:    Actions          : 0=*
+data:    NotActions       : 0=Microsoft.Authorization/*/Delete, 1=Microsoft.Authorization/*/Write
+data:    IsCustom         : false
+```
+This will associate the serviceprincipal to your current subscription
+```
+D:\sdk>azure role assignment create --spn 56894bd4-0fde-41d8-a0d7-5bsslccety2 -o Contributor
+info:    Executing command role assignment create
++ Finding role with specified name
+/data:    RoleAssignmentId     : /subscriptions/abcdefgh-1234-4cc9-89b5-12345678/providers/Microsoft.Authorization/roleAssignments/987654-ea85-40a5-80c2-abcdferghtt
+data:    RoleDefinitionName   : Contributor
+data:    RoleDefinitionId     : jhfskjf-6180-42a0-ab88-5656eiu677e23e
+data:    Scope                : /subscriptions/abcdefgh-1234-4cc9-89b5-12345678
+data:    Display Name         : testap674
+data:    SignInName           :
+data:    ObjectId             : weewrerer-e329-4e9b-98c6-7878787
+data:    ObjectType           : ServicePrincipal
+data:
++
+info:    role assignment create command OK
+```
+
+#### Step 4. Login as a serviceprincipal
+```
+D:\sdk>azure login -u 56894bd4-0fde-41d8-a0d7-5bsslccety2 -p P@ssw0rd --tenant <a guid OR your domain(contosocorp.com) --service-principal
+info:    Executing command login
+\info:    Added subscription TestSubscription
++
+info:    login command OK
+```
+
+## Using serviceprincipal authentication in your node.js script
+```javascript
+ var msrestAzure = require('ms-rest-azure');
+ //service principal authentication
+ 'your-client-id' - is the spn ('56894bd4-0fde-41d8-a0d7-5bsslccety2')
+ 'your-domain' - is the tenant id (a guid) or the part **after @** in your username (user1@**contosocorp.com**) ('contosocorp.com')
+ 'your-secret' - is the password you created for the serviceprincipal ('P@ssw0rd')
+ var credentials = new msRestAzure.ApplicationTokenCredentials('your-client-id', 'your-domain', 'your-secret');
+ ```
+
+## Using user authentitcation in your node.js script
+Currently, the node sdk only supports users with org-id (organizational account) and have 2FA disabled.
+ ```javascript
+ var msrestAzure = require('ms-rest-azure');
+ //user authentication
+ 'your-client-id' - is the id provided by Azure Active Directory for your application
+ 'your-domain' - is the tenant id (a guid) or the part **after @** in your username (user1@**contosocorp.com**) ('contosocorp.com')
+ 'your-username' - is your username ('[email protected]')
+ 'your-password' - password associated with the username
+ 'your-redirect-uri' - is the redirect uri for your application. Providing 'http://localhost:8080' should also be fine.
+ var credentials = new msRestAzure.UserTokenCredentials('your-client-id', 'your-domain', 'your-username', 'your-password', 'your-redirect-uri');
+ ```

Documentation/Debugging.md

@@ -12,6 +12,26 @@ console.log(">>>>>>>>>>>>>> Some Identifier   " + util.inspect(some_object, {dep
 ```
 Providing **{depth: null}** is optional. By default, it will dump object upto 3 levels deep. Setting depth  to null will dump the complete object.
 
+## Using Visual Studio
+The Visual Studio plugin for node.js can be downloaded from [here](https://github.com/Microsoft/nodejstools/releases).
+
+### Setting up the project
+* Open the **nodeSDK.sln** file present at the root of the cloned repo.
+* Set the Tab size and Indentation to 2 spaces by going to 
+  * Tools --> Options --> TextEditor --> Node.js --> Tabs --> [Tab size: 2, Indent size: 2]
+  * Tools --> Options --> TextEditor --> Javascript --> Tabs --> [Tab size: 2, Indent size: 2]
+
+### For debugging the tests:
+  * From the menu bar go to Project --> Properties and set the environment variables required for running tests.
+    * The list of environment variables can be found over [here](./EnvironmentVariables.md)
+    * If the need arises to add a new environment variable for tests please update the [Documentation](EnvironmentVariables.md) to keep the list current
+    * Visual Studio can also be opened from **"Visual Studio Debugger Cmd Prompt"** usually located at "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\Tools\Shortcuts\Developer Command Prompt for VS2013.lnk" to set the environment variables. Then open the .sln file from the prompt.
+  * In the Solution Explorer, click on a test that needs to be debugged. For example: "azure-sdk-for-node\test\services\storageManagement\storageManagementClient-tests.js"
+  * In the **Properties** pane select **"Mocha"** as the Test Framework. Save All the changes. 
+  * The tests shall be seen in the "Test Explorer". Right Click on any Test and Select "Debug Selected Test".
+* **Note:** If the test file has any errors then you might not see the tests in the TestExplorer. In the Properties pane select **ExportRunner** as the test framework and save the solution. You will see the error in the output pane. Once the errors are resolved change the test framework back to **Mocha** in the Properties pane for that test file. When you save the solution, you should see your tests in the test explorer.
+  * You can also run ```npm -s run-script jshint``` in the cmd prompt from the root of the repo to find errors if any. 
+
 ##  Using node inspector
 This will open a debugging session in a browser (chrome or opera). It is super easy to use.
 Steps to get it working:
@@ -52,38 +72,3 @@ Runner.prototype.runTest = function(fn){
 };
 ```
 * Set a breakpoint in your test which should be located under "azure-sdk-for-node/test/commands" directory
-
-
-## Using Visual Studio
-The Visual Studio plugin for node.js can be downloaded from [here](http://nodejstools.codeplex.com/).
-
-### Setting up the project
-* File --> New --> Project
-* On the left pane Installed --> Templates --> Javascript --> Node.js
-* From the available options Select "From Existing Node.js Code"
-  * Provide a name to your project "xplat" and a name to the solution "xplat"
-  * The location of the project would be the location of your cloned repo. Example - "D:\sdk\xplat\azure-sdk-tools-xplat"
-* Next --> Enter the filter to include files: In the end append the following string "; *._js"
-* Next --> Including node_modules in the project is optional. (It can always be include later, if the need arises).
-* Next --> Location for the project file - "D:\sdk\xplat\azure-sdk-tools-xplat\xplat.njsproj" --> Finish
-* In some time the solution explorer shows the source code files.
-* For better performance, it is advisable to **disable** intellisense in VisualStudio for Node.js projects by going to
-  * Tools --> Options --> TextEditor --> Node.js --> Intellisense -->No Intellisense.
-* Set the Tab size and Indentation to 2 spaces by going to 
-  * Tools --> Options --> TextEditor --> Node.js --> Tabs --> [Tab size: 2, Indent size: 2]
-  * Tools --> Options --> TextEditor --> Javascript --> Tabs --> [Tab size: 2, Indent size: 2]
-
-### For debugging the service:
-  * Create a sample.js file that instantiates the service client you intend to debug.
-  * Right Click the "azure-sdk-for-node\sample.js" file and set it as Node.js startup file.
-  * Set breakpoints at desired locations and Press F5 for happy debugging
-  * At times, files with extension "._js" do not hit the breakpoint. It is flaky and nothing can be done about it. At such times, console.log() is your best buddy :+1:
-
-### For debugging the tests:
-  * From the menu bar go to Project --> Properties and set the environment variables required for running tests.
-    * The list of environment variables can be found over [here](./EnvironmentVariables.md)
-    * If the need arises to add a new environment variable for tests please update the [Documentation](EnvironmentVariables.md) to keep the list current
-    * Visual Studio can also be opened from **"Visual Studio Debugger Cmd Prompt"** usually located at "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\Tools\Shortcuts\Developer Command Prompt for VS2013.lnk" to set the environment variables. Then open the .sln file from the prompt.
-  * In the Solution Explorer, click on a test that needs to be debugged. For example: "azure-sdk-for-node\test\services\sql\sqlmanagementservice-tests.js"
-  * In the **Properties** pane select **"Mocha"** as the Test Framework. Save All the changes. 
-  * The tests shall be seen in the "Test Explorer". Right Click on any Test and Select "Debug Selected Test".

Documentation/EnvironmentVariables.md

@@ -1,5 +1,34 @@
 ## Environment Variables Setup
 
+
+### For clients targeting the ARM (V2) Azure API following environment variables need to be setup
+From an admin cmd console/terminal, at the root directory of your cloned repo, run the following for environment setup:
+* **Windows**
+```
+set AZURE_SUBSCRIPTION_ID=<A Guid>
+set CLIENT_ID=<A Guid> # Application Id provided by Azure Active Directory (SPN for service principal auth)
+set DOMAIN=<A Guid or the domain name of your org> contosoCorp.com
+set AZURE_USERNAME=<Your org-id user name> [email protected] # Only set this if you are using user authentication
+set AZURE_PASSWORD=<Your Password> # Only set this if you are using user authentication
+set APPLICATION_SECRET=<Your service principal password or secret> # Only set this if you are using service principal auth
+set NOCK_OFF=true
+set AZURE_NOCK_RECORD=
+```
+
+* **OS X**, **Linux**
+```
+export AZURE_SUBSCRIPTION_ID=<A Guid>
+export CLIENT_ID=<A Guid> # Application Id provided by Azure Active Directory (SPN for service principal auth)
+export DOMAIN=<A Guid or the domain name of your org> contosoCorp.com
+export AZURE_USERNAME=<Your org-id user name> [email protected] # Only set this if you are using user authentication
+export AZURE_PASSWORD=<Your Password> # Only set this if you are using user authentication
+export APPLICATION_SECRET=<Your service principal password or secret> # Only set this if you are using service principal auth
+export NOCK_OFF=true
+export AZURE_NOCK_RECORD=
+```
+
+### For clients targeting the ASM (V1) Azure API following environment variables need to be setup
+
 From an admin cmd console/terminal, at the root directory of your cloned repo, run the following for environment setup:
 * **Windows**
 ```

Documentation/README.md

@@ -1,10 +1,12 @@
 # Azure-SDK-For-Node Documentation
 
 1. [Setting up the repo](./SetupRepo.md)
+2. [Login using ServicePrincipal](./Authentication.md)
+3. [Regenerating code for your service](./RegeneratingCode.md)
 4. Testing the SDK
     - [Setting up the Environment Variables](./EnvironmentVariables.md)
     - [Test Modes & Recording Infrastructure](./TestModes.md)
     - [Running Tests](./RunTests.md)
 5. [Release Process](./ReleaseProcess.md)
-5. [Debugging the SDK](./Debugging.md)
-6. [Generating Self Signed Certificates for Testing Purpose](./DummyCerts.md)
+6. [Debugging the SDK](./Debugging.md)
+7. [Generating Self Signed Certificates for Testing Purpose](./DummyCerts.md)

Documentation/RegeneratingCode.md

@@ -0,0 +1,15 @@
+# Regenerating Code
+
+- Please execute `npm install` locally from the root of the cloned repo (This will install gulp locally for the repo).
+- Please install gulp globally `npm install gulp -g`
+- If you need the latest version of Autorest, then it needs to be specified in [gulpfile.js](https://github.com/Azure/azure-sdk-for-node/blob/autorest/gulpfile.js#L59)
+  * Determining latest Autorest version
+    * A beta version is published every night to a myget feed . Hence, set the autoRestVersion to yesterday's date. Always increment the version of Autorest by 1 from the publicly available version on [nuget](http://www.nuget.org/packages/AutoRest/). While this doc is being written the publicly available version is 0.14.0
+    * ```var autoRestVersion=0.15.0-Nightly20160219```
+- Make sure you have updated the [mappings object](https://github.com/Azure/azure-sdk-for-node/blob/autorest/gulpfile.js#L6) correctly for generating the source code for your service
+- To list all the tasks execute `gulp -T` from the root of the repo
+- Regenertion command options
+  - If you want to regenerate all the services then execute `gulp codegen`
+  - If you want to generate for your project then execute `gulp codegen --project <your project name a.k.a the key of the mappings object>`
+  - If you want to use a local file then save the file locally say "D:\sdk" and make sure to have the same path as defined in the source of your porject in the mappings object. Execute `gulp codegen --spec-root "D:\sdk" --project <your-project-name>`.
+- If generation is successful then you will see the generated code in the lib folder under `lib/services/<YourServiceName>`

Documentation/SetupRepo.md

@@ -18,9 +18,6 @@ not polluted with your current changes.
 git checkout -b mybranch 
 git push origin mybranch
 ```
-* Setup the environment
-  * On a Windows machine, open a cmd prompt as an Administrator and run the following command: ```npm run-script setup```
-  * On a Mac or Linux machine, at the terminal run the following command: ```sudo npm run-script setup```
 
 * Installing the required npm modules from the root directory```npm install```
 

Documentation/TestModes.md

@@ -31,19 +31,19 @@ The recordings will get saved in azure-xplat-cli/test/recordings/{test-suite} di
 #### Recording tests related to a specific service/feature
 If you plan on adding some tests / features and do not need to regenerate the full set of test recordings, you can open the file: 
 ```
-tests/testlist.txt (if you are writing tests for commands in asm mode)
-tests/testlistarm.txt (if you are writing tests for commands in arm mode)
+tests/testlist.txt (if you are writing tests for clients in asm mode)
+tests/testlistarm.txt (if you are writing tests for clients in arm mode)
 ```
 and comment out the tests you do not wish to run during the recording process.
 
 To do so, use a leading \# character. i.e.:
 
-\# commands/cli.cloudservice-tests.js <br />
-\# commands/cli.deployment-tests.js <br />
-commands/cli.site-tests.js <br />
-\# commands/cli.site.appsetting-tests <br />
+\# services/resourceManagement/resourceManagementClient-tests.js <br />
+\# services/resourceManagement/authorizationClient-tests.js <br />
+services/storageManagement/storageManagementClient-tests.js <br />
+\# services/resourceManagement/featureClient-tests.js <br />
 
-In the above example only the cli.site-tests.js tests would be run.
+In the above example only the storageManagementClient-tests.js tests would be run.
 
 #### Recording a particular test in a suite