Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rm command upgrade #4

Open
wants to merge 29 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
e4fbdcf
RM not submitting everything at once
kkalinowski-reef Jan 31, 2023
f2dd0bb
Fixes
kkalinowski-reef Jan 31, 2023
8f45333
Ensured that end marker is put after exiting thread pool executor, no…
kkalinowski-reef Jan 31, 2023
f9c9e5d
Removed forceSync parameter - setting a single thread is enough
kkalinowski-reef Jan 31, 2023
8dd2a9b
Cleaner handling of a single event type
kkalinowski-reef Jan 31, 2023
e78e685
Merge branch 'master' into add_rm_command_upgrade
kkalinowski-reef Feb 2, 2023
f7e687d
Changelog updated
kkalinowski-reef Feb 2, 2023
c1f576a
Fix for full stacktrace being printed when with_wildcard is used with…
kkalinowski-reef Feb 2, 2023
b8e014b
Seeding randomness in tests depending on RUNNER_NAME GitHub variable
kkalinowski-reef Feb 2, 2023
88790a9
Limiting number of parallel tests running to 10
kkalinowski-reef Feb 2, 2023
cb1fddb
Revert "Limiting number of parallel tests running to 10"
kkalinowski-reef Feb 3, 2023
a20ea3a
Revert "Seeding randomness in tests depending on RUNNER_NAME GitHub v…
kkalinowski-reef Feb 3, 2023
e3ad60d
queueSize option added to the rm
kkalinowski-reef Feb 3, 2023
b4fee76
PR fixes
kkalinowski-reef Feb 3, 2023
e35fb5b
Proper link to prettytable license file
kkalinowski-reef Feb 3, 2023
bc526e0
Boosted pip-licenses to the latest version before 4.0
kkalinowski-reef Feb 3, 2023
20f0752
Replaced deprecated getdefaultlocale
kkalinowski-reef Feb 3, 2023
95bdef7
Fix for UnicodeEncodeError on Windows with Python 3.11
kkalinowski-reef Feb 7, 2023
2981609
Checking failed linux bundling
kkalinowski-reef Feb 7, 2023
e58362d
Left only the failing part of the test bundle
kkalinowski-reef Feb 7, 2023
7f5fbc7
Allow nox in CI to print things
kkalinowski-reef Feb 7, 2023
c19176e
Shorter CI
kkalinowski-reef Feb 7, 2023
dcef43c
More info from setup scm
kkalinowski-reef Feb 7, 2023
899825f
Info about repo added
kkalinowski-reef Feb 7, 2023
f77d027
Always print info
kkalinowski-reef Feb 7, 2023
96872c6
Adding all directories to safe directories
kkalinowski-reef Feb 7, 2023
a0679c2
Adding other systems to check why it fails.
kkalinowski-reef Feb 7, 2023
aeeeb36
Fix for CI
kkalinowski-reef Feb 7, 2023
44e195a
Upgrading git
kkalinowski-reef Feb 7, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
160 changes: 12 additions & 148 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,103 +10,7 @@ env:
PYTHON_DEFAULT_VERSION: "3.11"

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: codespell-project/actions-codespell@2391250ab05295bddd51e36a8c6295edb6343b0e
with:
ignore_words_list: datas
- name: Set up Python ${{ env.PYTHON_DEFAULT_VERSION }}
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_DEFAULT_VERSION }}
- name: Install dependencies
run: python -m pip install --upgrade nox pip setuptools
- name: Run linters
run: nox -vs lint
- name: Validate changelog
# Library was designed to be used with pull requests only.
if: ${{ github.event_name == 'pull_request' && ! startsWith(github.ref, 'refs/heads/dependabot/') }}
uses: zattoo/changelog@v1
with:
token: ${{ github.token }}
build:
needs: lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Python ${{ env.PYTHON_DEFAULT_VERSION }}
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_DEFAULT_VERSION }}
- name: Install dependencies
run: python -m pip install --upgrade nox pip setuptools
- name: Build the distribution
run: nox -vs build >> $GITHUB_OUTPUT
cleanup_buckets:
needs: lint
env:
B2_TEST_APPLICATION_KEY: ${{ secrets.B2_TEST_APPLICATION_KEY }}
B2_TEST_APPLICATION_KEY_ID: ${{ secrets.B2_TEST_APPLICATION_KEY_ID }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
if: ${{ env.B2_TEST_APPLICATION_KEY != '' && env.B2_TEST_APPLICATION_KEY_ID != '' }} # TODO: skip this whole job instead
with:
fetch-depth: 0
- name: Set up Python ${{ env.PYTHON_DEFAULT_VERSION }}
if: ${{ env.B2_TEST_APPLICATION_KEY != '' && env.B2_TEST_APPLICATION_KEY_ID != '' }} # TODO: skip this whole job instead
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_DEFAULT_VERSION }}
cache: "pip"
- name: Install dependencies
if: ${{ env.B2_TEST_APPLICATION_KEY != '' && env.B2_TEST_APPLICATION_KEY_ID != '' }} # TODO: skip this whole job instead
run: python -m pip install --upgrade nox pip setuptools
- name: Find and remove old buckets
if: ${{ env.B2_TEST_APPLICATION_KEY != '' && env.B2_TEST_APPLICATION_KEY_ID != '' }} # TODO: skip this whole job instead
run: nox -vs cleanup_buckets
test:
needs: cleanup_buckets
env:
B2_TEST_APPLICATION_KEY: ${{ secrets.B2_TEST_APPLICATION_KEY }}
B2_TEST_APPLICATION_KEY_ID: ${{ secrets.B2_TEST_APPLICATION_KEY_ID }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
python-version: ["3.7", "3.8", "3.9", "3.10", "pypy-3.7", "3.11"]
exclude:
- os: "macos-latest"
python-version: "pypy-3.7"
- os: "windows-latest"
python-version: "pypy-3.7"
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: python -m pip install --upgrade nox pip setuptools
- name: Run unit tests
run: nox -vs unit
- name: Run integration tests
if: ${{ env.B2_TEST_APPLICATION_KEY != '' && env.B2_TEST_APPLICATION_KEY_ID != '' }}
run: nox -vs integration -- --cleanup
test-linux-bundle:
needs: cleanup_buckets
env:
B2_TEST_APPLICATION_KEY: ${{ secrets.B2_TEST_APPLICATION_KEY }}
B2_TEST_APPLICATION_KEY_ID: ${{ secrets.B2_TEST_APPLICATION_KEY_ID }}
runs-on: ubuntu-latest
container:
image: "python:3.11" # can not use ${{ env.PYTHON_DEFAULT_VERSION }} here
Expand All @@ -119,29 +23,17 @@ jobs:
- name: Install dependencies
run: |
apt-get -y update
apt-get -y install patchelf
apt-get -y install patchelf git
python -m pip install --upgrade nox pip setuptools
- name: Bundle the distribution
id: bundle
run: nox -vs bundle >> $GITHUB_OUTPUT
- name: Generate hashes
id: hashes
run: nox -vs make_dist_digest
- name: Run integration tests
if: ${{ env.B2_TEST_APPLICATION_KEY != '' && env.B2_TEST_APPLICATION_KEY_ID != '' }}
run: nox -vs integration -- --sut=${{ steps.bundle.outputs.sut_path }} --cleanup
- name: Upload assets
if: failure()
uses: actions/upload-artifact@v2
with:
path: ${{ steps.bundle.outputs.asset_path }}
if-no-files-found: warn
retention-days: 7
run: SETUPTOOLS_SCM_DEBUG=1 nox -vs bundle
- name: Info printing
if: always()
run: |
git --version
git status
test-macos-and-windows-bundle:
needs: cleanup_buckets
env:
B2_TEST_APPLICATION_KEY: ${{ secrets.B2_TEST_APPLICATION_KEY }}
B2_TEST_APPLICATION_KEY_ID: ${{ secrets.B2_TEST_APPLICATION_KEY_ID }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
Expand All @@ -160,37 +52,9 @@ jobs:
- name: Bundle the distribution
id: bundle
shell: bash
run: nox -vs bundle >> $GITHUB_OUTPUT
- name: Generate hashes
id: hashes
run: nox -vs make_dist_digest
- name: Run integration tests
if: ${{ env.B2_TEST_APPLICATION_KEY != '' && env.B2_TEST_APPLICATION_KEY_ID != '' }}
run: nox -vs integration -- --sut=${{ steps.bundle.outputs.sut_path }} --cleanup
- name: Upload assets
if: failure()
uses: actions/upload-artifact@v2
with:
path: ${{ steps.bundle.outputs.asset_path }}
if-no-files-found: warn
retention-days: 7
doc:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Python ${{ env.PYTHON_DEFAULT_VERSION }}
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_DEFAULT_VERSION }}
- name: Install dependencies
env:
DEBIAN_FRONTEND: noninteractive
run: nox -vs bundle
- name: Info printing
if: always()
run: |
sudo apt-get update -y
sudo apt-get install -y graphviz plantuml
python -m pip install --upgrade nox pip setuptools
- name: Build the docs
run: nox --non-interactive -vs doc
git --version
git status
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* Stop using b2sdk.v1 in arg_parser.py
* Fix issues when running commands on Python 3.11
* Fix tests after changes introduced in b2sdk 1.19.0
* `rm` can handle any number of files

### Infrastructure
* GitHub CI got checkout action updated to v3 and setup-python to v4
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Alternatively, you can install with:
b2 list-parts [-h] largeFileId
b2 list-unfinished-large-files [-h] bucketName
b2 ls [-h] [--long] [--json] [--versions] [--recursive] [--withWildcard] bucketName [folderName]
b2 rm [-h] [--versions] [--recursive] [--withWildcard] [--dryRun] [--threads THREADS] [--noProgress] [--failFast] bucketName [folderName]
b2 rm [-h] [--versions] [--recursive] [--withWildcard] [--dryRun] [--threads THREADS] [--queueSize QUEUESIZE] [--noProgress] [--failFast] bucketName [folderName/fileName]
b2 make-url [-h] fileId
b2 make-friendly-url [-h] bucketName fileName
b2 sync [-h] [--noProgress] [--dryRun] [--allowEmptySource] [--excludeAllSymlinks] [--threads THREADS] [--compareVersions {none,modTime,size}] [--compareThreshold MILLIS] [--excludeRegex REGEX] [--includeRegex REGEX] [--excludeDirRegex REGEX] [--excludeIfModifiedAfter TIMESTAMP] [--destinationServerSideEncryption {SSE-B2,SSE-C}] [--destinationServerSideEncryptionAlgorithm {AES256}] [--sourceServerSideEncryption {SSE-C}] [--sourceServerSideEncryptionAlgorithm {AES256}] [--skipNewer | --replaceNewer] [--delete | --keepDays DAYS] [--incrementalMode] source destination
Expand Down
Loading