Skip to content

Commit

Permalink
Migrated merge process to Power Automate (#18)
Browse files Browse the repository at this point in the history
* Development solution changes now published on import
* Authentication via client secret with staging environment
* Fixed issue that caused merging flow to not trigger
* Migrated to flow, added solution version entity, replaced notes with file fields
* Updated documentation
* Updated sample script to perform a squash merge
  • Loading branch information
ewingjm authored Jul 15, 2020
1 parent aa5f27e commit 381988c
Show file tree
Hide file tree
Showing 101 changed files with 3,479 additions and 3,751 deletions.
44 changes: 17 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ The Development Hub brings continuous integration to Power Apps development by a

## Features

- Peer review
- Solution merging
- Automatic semantic versioning
- Automatic source control
- Peer review for configuration
- Merging for individual bugs and features
- Automated semantic versioning
- Automated source control

## Prerequisites

Expand All @@ -47,30 +47,23 @@ The package can be deployed to your development environment using the Package De

### Register an app

Access to the Common Data Service Web API is required in order to merge development solutions with the solution(s) in the master instance. Follow Microsoft's guide on registering an app with access to the Common Data Service Web API [here](https://docs.microsoft.com/en-us/powerapps/developer/common-data-service/walkthrough-register-app-azure-active-directory).

Once registered:

- Grant admin consent for the tenant for the Dynamics CRM API permissions
- Navigate to _Authentication_ and set _Treat application as a public client_ to _Yes_.
- Make a note of the client ID and tenant ID
Access to the Common Data Service Web API is required in order to merge development solutions with the solution(s) in the master instance. Follow Microsoft's guide on registering an app with access to the Common Data Service Web API [here](https://docs.microsoft.com/en-us/powerapps/developer/common-data-service/use-single-tenant-server-server-authentication#azure-application-registration). You will need to use the client ID, tenant ID, and a client secret for the app you register in later steps.

### Configure plug-in steps

Use the plug-in registration tool to set the secure configuration parameters necessary for authentication for the Web API. The steps to set the secure configuration for are under the InjectSecureConfig plug-in within the DevelopmentHub.Develop assembly.

The secure configuration is a JSON object with the client ID, tenant ID, and user credentials:
The secure configuration is a JSON object with the client ID, tenant ID, and client secret:

```json
{
"ClientId": "<from app registration>",
"TenantId": "<from app registration>",
"Username": "<service account email>",
"Password": "<service account password>"
"ClientSecret": "<from app registration>",
}
```

**Note: the user must have admin permissions in the master instance.**
**Note: the application user must should have admin permissions in both the master instance and the development instance.**

### Configure Azure DevOps

Expand All @@ -82,16 +75,9 @@ Navigate to _Project Settings -> Repositories_ in the Azure DevOps project that
- Contribute
- Create branch

A build definition capable of extracting solutions is required. Refer to the [samples](./samples) folder for a possible build configuration. If you use the sample files as is, the Cake build script assumes that your folder structure is similar to that of this repository i.e. you have a _solutions_ folder at the root, folders within this that match your solutions' unique names, a _solution.json_ within each of these that provides the development environment URL, and an _Extract_ folder alongside it to contain the unpacked solution zip file fragments.

If you have an existing folder structure which is different, the _build.cake_ file will probably require tweaking to how the `outputPath` variable is assigned as well as the path used to retrieve the _solution.json_ file within `GetConnectionString`. The _azure-pipelines-extract.yml_ file shouldn't need to be changed.
A build definition capable of extracting solutions is required. There are several files in the [samples](./samples) folder to help you with this. If you use the sample files as is, copy the _scripts_ folder and _azure-pipelines-extract.yml_ file into your repository. The sample build script assumes that your repository structure is that you have a _src_ folder at the root, a _solutions_ folder within, and then folders that match your solutions' unique names. In addition, it expects a _solution.json_ within each of these solution folders that provides the development environment URL (see the _solution.json_ in the samples folder), and an _extract_ folder alongside it to contain the unpacked solution zip file fragments. You will also need to create a _Development Hub_ variable group that contains three variables - `Client ID`, `Tenant ID`, and `Client Secret`. These should be taken from the app registration created earlier.

You will need [Cake](https://cakebuild.net/) installed in your repository to use the sample build files. You can do this easily within VS Code by installing the Cake extension and running the _Cake: Install to workspace_ task. If no _build.ps1_ bootstrapper file has been created in the root of the repository, you can create this using the _Cake: Install a bootstrapper_ task.
The sample _build.cake_ file can then be used.

The sample build requires that a variable group named 'Cake' exists and that it contains two variables - _dynamicsUsername_ and _dynamicsPassword_. These will be used by the build to connect to the development instance when extracting the post-merge solution zip.

**Note: the Common Data Service package Yeoman generator will scaffold a build and repository compatible with the Development Hub.**
If you have an existing folder structure which is different, the _Merge-SolutionVersion.ps1_ script will require tweaking - but the _azure-pipelines-extract.yml_ file shouldn't need to be changed.

### Set solution environment variables

Expand All @@ -106,10 +92,12 @@ The build definition ID is the numeric ID given to the build definition by Azure

### Set flow connections

There are two flows located in the _devhub_DevelopmentHub_AzureDevOps_ solution that must be set in order to trigger extract builds. The flows to set the connections on are:
There are four flows located in the _devhub_DevelopmentHub_Develop_ and _devhub_DevelopmentHub_AzureDevOps_ solutions that must be set. The flows to set the connections on are:

- When a solution merge is approved -> Merge the solution
- When a solution merge is merged -> Approve the first queued solution merge
- Environment Variable Key -> Environment Variable Value
- When a solution is merged - Commit changes to source control
- When a solution is merged -> Commit changes to source control

## Configuration

Expand Down Expand Up @@ -153,7 +141,7 @@ Once the issue has been developed, a solution merge record can be created. This

Once approved, the development solution will be merged into the target solution. If multiple solution merges have been approved, they will enter a queue. This means that an 'Approved' solution merge will transition to either a 'Merging' or 'Queued' status.

A successful solution merge will transition to an inactive 'Merged' status. The 'Version History' tab on the target solution record will also contain two new attachments with the post-merge unmanaged and managed solution zips. The new solution version is based on the type of issue merged. A feature issue will increment the minor version and a bug issue will increment the patch version. Major version changes must be done manually.
A successful solution merge will transition to an inactive 'Merged' status. The 'Version History' tab on the target solution record will also contain a new record with the post-merge unmanaged and managed solution zips available. The new solution version is based on the type of issue merged. A feature issue will increment the minor version and a bug issue will increment the patch version. Major version changes must be done manually.


![Solution merge](./docs/images/solutionmerge.png)
Expand All @@ -168,6 +156,8 @@ Specifying that there are manual merge activities on the solution merge record w

When the merging process is in a state where manual merge activities can begin, the solution merge will transition to an 'Awaiting Manual Merge Activities' status. If you are deleting components from the solution in the master instance, it is recommended to update the major version of the solution record in the Development Hub during this period.

To notify the flow that the manual merge activities are complete, navigate to _Action items -> Approvals_ within Power Automate and set the approval status to merged.

### Handle a failed merge

If the merging process failed (e.g. due to missing dependencies) then the solution merge will transition to a 'Failed' status. A *Retry* button is available after the necessary steps have been taken. Failure reason will be attached as a note to the solution merge record.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,17 +88,17 @@ protected override void Execute(CodeActivityContext context)
[ExcludeFromCodeCoverage]
private static IODataClient GetNewODataClient(Uri targetInstance, CodeActivityContext context, IWorkflowContext workflowContext, ILogWriter logWriter)
{
var passwordGrantRequest = GetPasswordGrantRequest(workflowContext, logWriter);
passwordGrantRequest.Resource = targetInstance;
var oAuthGrantRequest = GetOAuthGrantRequest(workflowContext, logWriter);
oAuthGrantRequest.Resource = targetInstance;

logWriter.Log(Severity.Info, Tag, $"Making password grant OAuth request for {passwordGrantRequest.Resource} as {passwordGrantRequest.Username}");
logWriter.Log(Severity.Info, Tag, $"Making client credentials grant OAuth request for {oAuthGrantRequest.Resource}.");

var token = (context.GetExtension<IOAuthTokenRepository>() ?? new OAuthTokenRepository()).GetAccessToken(passwordGrantRequest).Result;
var token = (context.GetExtension<IOAuthTokenRepository>() ?? new OAuthTokenRepository()).GetAccessToken(oAuthGrantRequest).Result;

return new ODataClient(targetInstance, token);
}

private static OAuthPasswordGrantRequest GetPasswordGrantRequest(IWorkflowContext workflowContext, ILogWriter logWriter)
private static OAuthClientCredentialsGrantRequest GetOAuthGrantRequest(IWorkflowContext workflowContext, ILogWriter logWriter)
{
if (!workflowContext.SharedVariables.ContainsKey(SharedVariablesSecureConfigKey))
{
Expand All @@ -109,13 +109,13 @@ private static OAuthPasswordGrantRequest GetPasswordGrantRequest(IWorkflowContex

logWriter.Log(Severity.Info, Tag, $"Deserializing password grant request from secure configuration.");

OAuthPasswordGrantRequest passwordGrantRequest;
OAuthClientCredentialsGrantRequest clientCredentialsGrantRequest;
using (var ms = new MemoryStream(Encoding.UTF8.GetBytes(secureConfig)))
{
passwordGrantRequest = (OAuthPasswordGrantRequest)new DataContractJsonSerializer(typeof(OAuthPasswordGrantRequest)).ReadObject(ms);
clientCredentialsGrantRequest = (OAuthClientCredentialsGrantRequest)new DataContractJsonSerializer(typeof(OAuthClientCredentialsGrantRequest)).ReadObject(ms);
}

return passwordGrantRequest;
return clientCredentialsGrantRequest;
}

private void SetError(CodeActivityContext context, TracingServiceLogWriter logWriter, string message)
Expand Down
2 changes: 1 addition & 1 deletion common/DevelopmentHub.Model/DevelopmentHub.Model.projitems
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
<Compile Include="$(MSBuildThisFileDirectory)ODataClientResponse.cs" />
<Compile Include="$(MSBuildThisFileDirectory)ODataEntity.cs" />
<Compile Include="$(MSBuildThisFileDirectory)ODataHeaders.cs" />
<Compile Include="$(MSBuildThisFileDirectory)Requests\OAuthClientCredentialsGrantRequest.cs" />
<Compile Include="$(MSBuildThisFileDirectory)Requests\OAuthPasswordGrantRequest.cs" />
<Compile Include="$(MSBuildThisFileDirectory)OAuthToken.cs" />
<Compile Include="$(MSBuildThisFileDirectory)ODataError.cs" />
Expand All @@ -22,6 +23,5 @@
</ItemGroup>
<ItemGroup>
<Folder Include="$(MSBuildThisFileDirectory)Responses\" />
<Folder Include="$(MSBuildThisFileDirectory)Requests\" />
</ItemGroup>
</Project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
namespace DevelopmentHub.Model.Requests
{
using System;
using System.Runtime.Serialization;

/// <summary>
/// Data contract for an OAuth password grant request.
/// </summary>
[DataContract]
public class OAuthClientCredentialsGrantRequest
{
/// <summary>
/// The OAuth grant type associated with this request.
/// </summary>
[IgnoreDataMember]
public const string GrantType = "client_credentials";

/// <summary>
/// Initializes a new instance of the <see cref="OAuthClientCredentialsGrantRequest"/> class.
/// </summary>
public OAuthClientCredentialsGrantRequest()
{
}

/// <summary>
/// Initializes a new instance of the <see cref="OAuthClientCredentialsGrantRequest"/> class.
/// </summary>
/// <param name="clientId">The client ID.</param>
/// <param name="tenantId">The tenant ID.</param>
/// <param name="resource">The resource.</param>
/// <param name="clientSecret">The client secret.</param>
public OAuthClientCredentialsGrantRequest(Guid clientId, Guid tenantId, Uri resource, string clientSecret)
{
this.ClientId = clientId;
this.TenantId = tenantId;
this.Resource = resource;
this.ClientSecret = clientSecret;
}

/// <summary>
/// Gets or sets the application's client ID.
/// </summary>
[DataMember]
public Guid ClientId { get; set; }

/// <summary>
/// Gets or sets the application's tenant ID.
/// </summary>
[DataMember]
public Guid TenantId { get; set; }

/// <summary>
/// Gets or sets the resource to request access to.
/// </summary>
[DataMember]
public Uri Resource { get; set; }

/// <summary>
/// Gets or sets the client secret.
/// </summary>
[DataMember]
public string ClientSecret { get; set; }
}
}
8 changes: 4 additions & 4 deletions common/DevelopmentHub.Repositories/IOAuthTokenRepository.cs
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@
public interface IOAuthTokenRepository
{
/// <summary>
/// Get an OAuth access token using the password grant.
/// Gets an access token for a resource using client credentials.
/// </summary>
/// <param name="request">The password grant request.</param>
/// <returns>An OAuth token./returns>.
Task<OAuthToken> GetAccessToken(OAuthPasswordGrantRequest request);
/// <param name="request">The password grant request parameters.</param>
/// <returns>The OAuth token.</returns>
Task<OAuthToken> GetAccessToken(OAuthClientCredentialsGrantRequest request);
}
}
7 changes: 3 additions & 4 deletions common/DevelopmentHub.Repositories/OAuthTokenRepository.cs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ public class OAuthTokenRepository : IOAuthTokenRepository
/// </summary>
/// <param name="request">The password grant request parameters.</param>
/// <returns>The OAuth token.</returns>
public async Task<OAuthToken> GetAccessToken(OAuthPasswordGrantRequest request)
public async Task<OAuthToken> GetAccessToken(OAuthClientCredentialsGrantRequest request)
{
byte[] response;

Expand All @@ -31,9 +31,8 @@ public async Task<OAuthToken> GetAccessToken(OAuthPasswordGrantRequest request)
{
{ "resource", request.Resource.ToString() },
{ "client_id", request.ClientId.ToString() },
{ "grant_type", OAuthPasswordGrantRequest.GrantType },
{ "username", request.Username },
{ "password", request.Password },
{ "grant_type", OAuthClientCredentialsGrantRequest.GrantType },
{ "client_secret", request.ClientSecret },
};

response = await client.UploadValuesTaskAsync(tokenEndpoint, data).ConfigureAwait(false);
Expand Down
35 changes: 8 additions & 27 deletions samples/azure-pipelines-extract.yml
Original file line number Diff line number Diff line change
@@ -1,34 +1,15 @@
name: $(solution) - $(commitMessage) - $(triggeredBy)
name: $(solution) $(commitMessage)
pool:
vmImage: windows-latest
trigger: none
steps:
- checkout: self
persistCredentials: true
- powershell: >-
$env:GIT_REDIRECT_STDERR = '2>&1';
git config --global user.email $env:TRIGGEREDBYEMAIL;
git config --global user.name $env:TRIGGEREDBY;
git checkout master;
if ($env:SOURCEBRANCH)
{
git merge origin/$env:SOURCEBRANCH --no-commit;
}
Invoke-Expression "&$(Build.SourcesDirectory)\build.ps1 -Target ExtractSolutionFromDevelopmentHub -ScriptArgs `"--solution=$env:SOLUTION`",`"--unmanagedNoteId=$env:UNMANAGEDNOTEID`",`"--managedNoteId=$env:MANAGEDNOTEDID`" -Verbosity Diagnostic";
git add .;
git reset -- NuGet.config;
$commitMessage = $env:COMMITMESSAGE;
if ($env:WORKITEMID)
{
$commitMessage += " #$env:WORKITEMID";
}
git commit -m $commitMessage;
git push origin;
displayName: Extract and commit solution
env:
CAKE_DYNAMICS_PASSWORD: $(dynamicsPassword)
CAKE_DYNAMICS_USERNAME: $(dynamicsUsername)
- task: PowerShell@2
inputs:
workingDirectory: $(Build.SourcesDirectory)
filePath: 'scripts/Merge-SolutionVersion.ps1'
arguments: '-ClientId "$(Client ID)" -TenantId "$(Tenant ID)" -ClientSecret (ConvertTo-SecureString "$(Client Secret)" -AsPlainText -Force) -SolutionVersionId "$(solutionVersionId)" -Solution "$(solution)" -CommitUserEmailAddress "$(triggeredByEmail)" -CommitUserName "$(triggeredBy)" -CommitMessage "$(commitMessage)" -SourceBranch "$(sourceBranch)" -WorkItemId "$(workItemId)"'
displayName: Extract and commit
variables:
- group: Cake
- name: workItemId
value: ""
- group: Development Hub
35 changes: 0 additions & 35 deletions samples/build.cake

This file was deleted.

Loading

0 comments on commit 381988c

Please sign in to comment.