Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --git-archive-path and --metadata-archive-path options #1294

Merged
merged 15 commits into from
Nov 21, 2024

Conversation

jfine
Copy link
Contributor

@jfine jfine commented Nov 7, 2024

Adds the ability to migrate a local file utilizing the --git-archive-path and --metadata-archive-path options. This change is similar to the --git-archive-url and --metadata-archive-url options though instead of starting the migration directly it uploads the archives (only one upload if the same archive is specified) to the storage option selected and then starts the migration.

  • Did you write/update appropriate tests
  • Release notes updated (if appropriate)
  • Appropriate logging output
  • Issue linked
  • Docs updated (or issue created)
  • New package licenses are added to ThirdPartyNotices.txt (if applicable)

@jfine jfine self-assigned this Nov 7, 2024
Copy link

github-actions bot commented Nov 7, 2024

Unit Test Results

840 tests   840 ✅  22s ⏱️
  1 suites    0 💤
  1 files      0 ❌

Results for commit c6d693a.

♻️ This comment has been updated with latest results.

@jfine jfine marked this pull request as ready for review November 21, 2024 14:53
@jfine jfine changed the title Add --git-archive-file-path and --metadata-archive-file-path options Add --git-archive-path and --metadata-archive-path options Nov 21, 2024
@jfine
Copy link
Contributor Author

jfine commented Nov 21, 2024

Manual tests (using Azure Storage)

Using two different archives

$ ./src/gei/bin/Debug/net6.0/gei migrate-repo \
  --git-archive-path /workspaces/gh-gei/jfine-sandbox-attachment-test-6492762.tar.gz \
  --metadata-archive-path /workspaces/gh-gei/jfine-sandbox-attachment-test-6492762-2.tar.gz \
  --github-target-org jfine-sandbox \
  --target-repo jfine-sandbox-attachment-different-archive \
  --github-source-org jfine-sandbox \
  --source-repo attachment-test \
  --github-target-pat ghp_*** \
  --azure-storage-connection-string "DefaultEndpointsProtocol=https;AccountName=***;AccountKey=***;EndpointSuffix=core.windows.net"
[2024-11-21 15:34:17] [INFO] GITHUB SOURCE ORG: jfine-sandbox
[2024-11-21 15:34:17] [INFO] SOURCE REPO: attachment-test
[2024-11-21 15:34:17] [INFO] GITHUB TARGET ORG: jfine-sandbox
[2024-11-21 15:34:17] [INFO] TARGET REPO: jfine-sandbox-attachment-different-archive
[2024-11-21 15:34:17] [INFO] AZURE STORAGE CONNECTION STRING: ***
[2024-11-21 15:34:17] [INFO] GIT ARCHIVE PATH: /workspaces/gh-gei/jfine-sandbox-attachment-test-6492762.tar.gz
[2024-11-21 15:34:17] [INFO] METADATA ARCHIVE PATH: /workspaces/gh-gei/jfine-sandbox-attachment-test-6492762-2.tar.gz
[2024-11-21 15:34:17] [INFO] GITHUB TARGET PAT: ***
[2024-11-21 15:34:17] [INFO] Since github-target-pat is provided, github-source-pat will also use its value.
[2024-11-21 15:34:17] [INFO] Migrating Repo...
[2024-11-21 15:34:18] [INFO] Uploading archive 2024-11-21_15-34-18-git_archive.tar.gz to Azure Blob Storage
[2024-11-21 15:34:19] [INFO] Upload complete
[2024-11-21 15:34:19] [INFO] Uploading archive 2024-11-21_15-34-18-metadata_archive.tar.gz to Azure Blob Storage
[2024-11-21 15:34:19] [INFO] Upload complete
[2024-11-21 15:34:19] [INFO] Archive(s) uploaded to blob storage, now starting migration...
[2024-11-21 15:34:21] [INFO] Migration in progress (ID: RM_kgDaACRkNmFlZDRhYS0zMTYzLTQ1ODktOTAzZS0wYTg4Yzk4OTgxZTM). State: PENDING_VALIDATION. Waiting 10 seconds...
[2024-11-21 15:34:42] [INFO] Migration in progress (ID: RM_kgDaACRkNmFlZDRhYS0zMTYzLTQ1ODktOTAzZS0wYTg4Yzk4OTgxZTM). State: IN_PROGRESS. Waiting 10 seconds...
...
[2024-11-21 15:38:17] [INFO] Migration completed (ID: RM_kgDaACRkNmFlZDRhYS0zMTYzLTQ1ODktOTAzZS0wYTg4Yzk4OTgxZTM)! State: SUCCEEDED

Using the same archive

$ ./src/gei/bin/Debug/net6.0/gei migrate-repo \
  --git-archive-path /workspaces/gh-gei/jfine-sandbox-attachment-test-6492762.tar.gz \
  --metadata-archive-path /workspaces/gh-gei/jfine-sandbox-attachment-test-6492762.tar.gz \
  --github-target-org jfine-sandbox \
  --target-repo jfine-sandbox-attachment-same-archive \
  --github-source-org jfine-sandbox \
  --source-repo attachment-test \
  --github-target-pat ghp_*** \
  --azure-storage-connection-string "DefaultEndpointsProtocol=https;AccountName=***;AccountKey=***;EndpointSuffix=core.windows.net"
[2024-11-21 15:06:20] [INFO] GITHUB SOURCE ORG: jfine-sandbox
[2024-11-21 15:06:20] [INFO] SOURCE REPO: attachment-test
[2024-11-21 15:06:20] [INFO] GITHUB TARGET ORG: jfine-sandbox
[2024-11-21 15:06:20] [INFO] TARGET REPO: jfine-sandbox-attachment-same-archive
[2024-11-21 15:06:20] [INFO] AZURE STORAGE CONNECTION STRING: ***
[2024-11-21 15:06:20] [INFO] GIT ARCHIVE PATH: /workspaces/gh-gei/jfine-sandbox-attachment-test-6492762.tar.gz
[2024-11-21 15:06:20] [INFO] METADATA ARCHIVE PATH: /workspaces/gh-gei/jfine-sandbox-attachment-test-6492762.tar.gz
[2024-11-21 15:06:20] [INFO] GITHUB TARGET PAT: ***
[2024-11-21 15:06:20] [INFO] Since github-target-pat is provided, github-source-pat will also use its value.
[2024-11-21 15:06:20] [INFO] Migrating Repo...
[2024-11-21 15:06:21] [INFO] Uploading archive 2024-11-21_15-06-21-archive.tar.gz to Azure Blob Storage
[2024-11-21 15:06:22] [INFO] Upload complete
[2024-11-21 15:06:22] [INFO] Archive(s) uploaded to blob storage, now starting migration...
[2024-11-21 15:06:24] [INFO] Migration in progress (ID: RM_kgDaACQ0YTU5MmFjYi0yYmE1LTQzNzAtOGFhOS05MTA0MTFkYzVhODI). State: PENDING_VALIDATION. Waiting 10 seconds...
[2024-11-21 15:06:35] [INFO] Migration in progress (ID: RM_kgDaACQ0YTU5MmFjYi0yYmE1LTQzNzAtOGFhOS05MTA0MTFkYzVhODI). State: IN_PROGRESS. Waiting 10 seconds...
...
[2024-11-21 15:10:42] [INFO] Migration completed (ID: RM_kgDaACQ0YTU5MmFjYi0yYmE1LTQzNzAtOGFhOS05MTA0MTFkYzVhODI)! State: SUCCEEDED

Copy link
Collaborator

@ArinGhazarian ArinGhazarian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good 🎉 The only missing thing is updating the releasenotes.md

@jfine jfine enabled auto-merge November 21, 2024 17:52
Copy link

Code Coverage

Package Line Rate Branch Rate Complexity Health
gei 81% 73% 578
ado2gh 84% 78% 627
Octoshift 87% 76% 1300
bbs2gh 79% 74% 680
Summary 84% (7081 / 8463) 76% (1613 / 2134) 3185

@jfine jfine merged commit 66fde68 into main Nov 21, 2024
30 checks passed
@jfine jfine deleted the add-git-archive-file-git-metadata-file-options branch November 21, 2024 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants