Skip to content

Commit

Permalink
Prepare for v4.3.0-beta1. (#823)
Browse files Browse the repository at this point in the history
Many small things including:

* Update release notes.
* Update pyktx README.md with note about macOS build warning.
* Fix many small issues with pyktx deployment.
* Fix pyktx version number to include normalized tweak.
* Add tweak to version number in Android package names.
* Fix typo in example in documentation.
* Fix Emscripten build to meet new requirement to enable
  GetProcAddress via compile option.
* Fix #851: image corruption on loading.
* Prepare for publishing pyktx to PyPI.
  • Loading branch information
MarkCallow authored Dec 22, 2023
1 parent d3ef5ed commit 29255c5
Show file tree
Hide file tree
Showing 24 changed files with 345 additions and 117 deletions.
126 changes: 126 additions & 0 deletions .github/workflows/publish-pyktx.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# Copyright 2023 Shukant Pal
# SPDX-License-Identifier: Apache-2.0

name: Publish Python 🐍 distribution 📦 to PyPI or TestPyPI

# https://github.com/pypa/gh-action-pypi-publish strongly recommends a
# separate publishing job when building platform-specific distribution
# packages, hence this.
#
# This should not be run until the release artifacts have been deployed.
# The only way I can see to trigger this automatically is to have all
# platform builds in a single workflow file and use on: `workflow_run`
# so this is triggered when that workflow completes. Once all platform
# builds are moved to GitHub Actions we can try making each platform
# workflow reusable and making another workflow that calls each of
# them via `uses` and use `on: workflow_run` here.
#
# We also have to figure out how to determine if the workflow deployed
# to releases. Maybe can use the GitHub REST API to get an artifact
# from the triggering workflow that is set only when deploying
# releases. See https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#using-data-from-the-triggering-workflow.
# Alternatively maybe there is some way of querying what triggered
# the workflow that we can test in an `if:` in the job as noted below.

on:
workflow_dispatch:
inputs:
# repository-url:
# description: 'destination repository'
# required: true
# default: 'https://pypi.org'
# type: choice
# options:
# - https://test.pypi.org
# - https://pypi.org
test-pypi:
type: boolean
description: 'Deploy to test pypi registry'
required: true
default: false

# Example to try in future.
# workflow_run:
# workflows: [KTX-Software Build All]
# types:
# - completed
# An unknown is how to limit the trigger to when deploy is run in
# Windows CI which is happens when that workflow is triggered by
# a push with the tags below.
# push:
# # Trigger on push of release tags. There is no way to limit the trigger
# # to a specific branch. A later build step checks for the main branch.
# tags:
# - 'v[0-9]+\.[0-9]+\.[0-9]+'
# - 'v[0-9]+\.[0-9]+\.[0-9]+-*'

jobs:
publish-to-pypi:
name: Publish Python 🐍 distribution 📦 to PyPI
if: test-pypi == false
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/pyktx
permissions:
id-token: write # IMPORTANT: this permission is mandatory for trusted publishing
# When using workflow_run it is necessary to check for successful
# completion of the workflow with
#if: ${{ github.event.workflow_run.conclusion == 'success' }}
# Maybe it is possible to test for a release tag here too.
steps:
- name: Download latest release metadata
run: |
curl \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
-o latest.json \
https://api.github.com/repos/KhronosGroup/KTX-Software/releases/latest
- name: Filter out pyktx- assets
run: |
jq '.assets | map(select(.name | startswith("pyktx-"))) | map(.browser_download_url)' < latest.json > pyktx.json
- name: Create dist directory
run: |
mkdir dist
- name: Download assets
run: |
cd dist; jq -c '.[]' ../pyktx.json | xargs -L 1 curl -O -J; cd..
- name: Publish distribution 📦 to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.PYPI_API_TOKEN }}

publish-to-testpypi:
name: Publish Python 🐍 distribution 📦 to TestPyPI
if: test-pypi
runs-on: ubuntu-latest
environment:
name: testpypi
url: https://pypi.org/p/pyktx
permissions:
id-token: write # IMPORTANT: this permission is mandatory for trusted publishing
# When using workflow_run it is necessary to check for successful
# completion of the workflow with
#if: ${{ github.event.workflow_run.conclusion == 'success' }}
# Maybe it is possible to test for a release tag here too.
steps:
- name: Download latest release metadata
run: |
curl \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
-o latest.json \
https://api.github.com/repos/KhronosGroup/KTX-Software/releases/latest
- name: Filter out pyktx- assets
run: |
jq '.assets | map(select(.name | startswith("pyktx-"))) | map(.browser_download_url)' < latest.json > pyktx.json
- name: Create dist directory
run: |
mkdir dist
- name: Download assets
run: |
cd dist; jq -c '.[]' ../pyktx.json | xargs -L 1 curl -O -J; cd..
- name: Publish distribution 📦 to TestPyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.PYPI_API_TOKEN }}
38 changes: 0 additions & 38 deletions .github/workflows/publish.yml

This file was deleted.

10 changes: 6 additions & 4 deletions .github/workflows/windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,7 @@ jobs:
# is spawned from some parent Powershell so it does not pick up
# the changes from the registry. Use Chocolatey's helper to
# pull in the changes.
# After import `refreshenv` is an alias for
# After import, `refreshenv` is an alias for
# Update-SessionEnvironment. Without the import refreshenv will end
# up calling the cmd.exe version, which won't help Powershell, and
# Update-SessionEnvironment will not exist.
Expand Down Expand Up @@ -263,18 +263,20 @@ jobs:
path: ${{env.BUILD_DIR}}/KTX-Software-*.exe*

- name: Upload pyktx artifacts
if: matrix.options.package == 'YES' && matrix.options.py
if: matrix.options.package == 'YES' && matrix.options.py == 'ON'
uses: kittaakos/upload-artifact-as-is@v0
with:
path: ${{env.BUILD_DIR}}/interface/python_binding/dist/

- name: Upload To Release
- name: Upload to Release
uses: softprops/action-gh-release@v1
if: matrix.options.package == 'YES' && matrix.toolset == 'CLangCL' && github.event_name == 'push' && github.ref_type == 'tag'
with:
draft: true
prerelease: true
files: ${{env.BUILD_DIR}}/KTX-Software-*.exe*
files: |
${{env.BUILD_DIR}}/KTX-Software-*.exe*
${{env.BUILD_DIR}}/interface/python_binding/dist/pyktx-*
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,6 @@ cmake-build-*
# Temps used for Windows signing on CI
appveyor-tools
the_khronos_group_inc.p12

# Place for random developer test files.
/testground/
16 changes: 15 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -396,11 +396,25 @@ deploy:

- provider: script
edge: true
script: ruby ci_scripts/github_release.rb -s ${GITHUB_TOKEN} -r ${TRAVIS_REPO_SLUG} -c $REL_DESC_FILE -t ${TRAVIS_TAG} --overwrite true --draft true --prerelease true $BUILD_DIR/KTX-Software-*-* $BUILD_DIR/interface/python_binding/dist/*.whl
# nullglob is to force a null string when there are no *.whl files.
script: ruby ci_scripts/github_release.rb -s ${GITHUB_TOKEN} -r ${TRAVIS_REPO_SLUG} -c $REL_DESC_FILE -t ${TRAVIS_TAG} --overwrite true --draft true --prerelease true $BUILD_DIR/KTX-Software-*-*
on:
tags: true
#branch: master
condition: $PACKAGE = "YES" && $TRAVIS_TAG =~ ^v[0-9].*

# Separate deploy for pyktx because the linux runners use sh not bash
# for the "script" so `shopt -s nullglob; ci_scripts/github_release.rb ...`
# does not work to avoid passing `.../*.whl` to github_release when
# FEATURE_PY not ON.
- provider: script
edge: true
script: ruby ci_scripts/github_release.rb -s ${GITHUB_TOKEN} -r ${TRAVIS_REPO_SLUG} -c $REL_DESC_FILE -t ${TRAVIS_TAG} --overwrite true --draft true --prerelease true $BUILD_DIR/interface/python_binding/dist/*.whl
on:
tags: true
#branch: master
condition: $PACKAGE = "YES" && $TRAVIS_TAG =~ ^v[0-9].* && $FEATURE_PY = "ON"

# ---------------------------------------------------------

# vim:ai:ts=4:sts=2:sw=2:expandtab
1 change: 1 addition & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -877,6 +877,7 @@ if(EMSCRIPTEN)
"SHELL:-s EXPORT_NAME=LIBKTX"
"SHELL:-s EXPORTED_RUNTIME_METHODS=[\'GL\']"
"SHELL:-s GL_PREINITIALIZED_CONTEXT=1"
"SHELL:-s GL_ENABLE_GET_PROC_ADDRESS=1" # For Emscripten 3.1.51+
)
set_target_properties( ktx_js PROPERTIES OUTPUT_NAME "libktx")

Expand Down
104 changes: 97 additions & 7 deletions RELEASE_NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,31 @@
<!-- SPDX-License-Identifier: Apache-2.0 -->
Release Notes
=============
## Version 4.3.0-alpha3
### New Features
## Version 4.3.0-beta1
### New Features in Version 4.3.0
#### Command Line Tools Suite

v4.3.0 contains a new suite of command line tools accessed via an umbrella `ktx` command.

| Tool | Description | Equivalent old tool |
| :--- | ----------- | ------------------- |
| `ktx create` | Create a KTX2 file from various input files | `toktx` |
| `ktx extract` | Export selected images from a KTX2 file | - |
| `ktx encode` | Encode a KTX2 file | `ktxsc` |
| `ktx encode` | Encode a KTX2 file to a Basis Universal format | `ktxsc` |
| `ktx transcode` | Transcode a KTX2 file | - |
| `ktx info` | Prints information about a KTX2 file | `ktxinfo` |
| `ktx validate` | Validate a KTX2 file | `ktx2check` |
| `ktx help` | Display help information about the ktx tools | - |

Equivalent old tools will be removed in the next release.
Equivalent old tools will be removed in a subsequent release soon.

Some features of the old tools are not currently available in the new equivalent.

| Old Tool | New Tool | Missing Features |
| :------: | :------: | ---------------- |
| `toktx` | `create` | JPEG and NBPM input and scaling/resizing. |
| `ktxsc` | `encode` | ASTC encoding. This can be done in `create`. |
| `toktx` | `create` | JPEG and NBPM input and scaling/resizing of input images. |
| `ktxsc` | `encode` | ASTC encoding of a KTX2 file. This can be done in `create`. <br>Deflation of a KTX2 file with zlib or zstd.|


The command-line syntax and semantics differ from the old tools including, but not limited to:

Expand All @@ -43,12 +45,17 @@ The command-line syntax and semantics differ from the old tools including, but n

Please refer to the manual pages or use the `--help` command-line option for further details on the options available and associated semantics for each individual command.

#### Python Binding
A Python binding for `libktx` has been added. Applications written in Python can now use `libktx` functions. Huge thanks to @ShukantPal. Please download the appropriate `pyktx` package from Releases as publishing to PyPI is not yet established.

### Changes

* `libktx` has been made much more robust to errors KTX files.

* `libktx` now validates checksums when present in a Zstd data stream.
* `libktx` has two new error codes it can return: `KTX_DECOMPRESS_LENGTH_ERROR` and `KTX_DECOMPRESS_CHECKSUM_ERROR`.
* All tools and `libktx` now correctly process on all platforms utf8 file names
with multi-byte code-points. Previously such names did not work on Windows.
* The Vulkan texture uploader can now optionally be used with an extenal memory allocator such as [VulkanMemoryAllocator](https://gpuopen.com/vulkan-memory-allocator/).

### Known Issues

Expand All @@ -63,6 +70,89 @@ Please refer to the manual pages or use the `--help` command-line option for fur
* Neither the Vulkan nor GL loaders support depth/stencil textures.


### Changes since v4.3.0-alpha3 (by part)
### libktx

* git subrepo push lib/dfdutils (ab9c27707) (@MarkCallow)

* Reenable build of loadtest apps on Windows arm64 CI (#802) (6c131f75f) (@MarkCallow)

* Utf-8/unicode support in legacy tools and lib. (#800) (1c5dc9cf6) (@MarkCallow)

* Do not redefine NOMINMAX (#801) (6dbb24643) (@corporateshark)

* libktx: update ktxTexture2\_setImageFromStream to allow setting the entire level's data in one call (#794) (88fc7a6e9) (@AlexRouSg)

* Update dfdutils-included vulkan\_core.h. (#783) (9c223d950) (@MarkCallow)

* Support for A8 and A1B5G5R5 formats (#785) (eeac6206c) (@aqnuep)

* Major non-content documentation fixes. (#773) (e6a6a3be9) (@MarkCallow)

* Fix vendor-specific/tied memory property flag detection (#771) (a10021758) (@toomuchvoltage)

* Return KTX\_NOT\_FOUND when a GPU proc is not found. (#770) (aeca5e695) (@MarkCallow)

* Support for external allocators: (#748) (6856fdb0d) (@toomuchvoltage)

* Fix ktx\_strncasecmp (#741) (1ae04f897) (@VaderY)

* Use correct counter for indexing sample. (#739) (3153e94e8) (@MarkCallow)

### Tools

* Disallow ASTC options when format is not ASTC (#809) (d3ef5ed8b) (@aqnuep)

* Remove unnecessary nullptr checks. (#807) (072a4eb25) (@MarkCallow)

* Set tools and tests rpath on Linux. (#804) (928612a71) (@MarkCallow)

* Utf-8/unicode support in legacy tools and lib. (#800) (1c5dc9cf6) (@MarkCallow)

* Support building of loadtest apps with locally installed dependencies (#799) (84ee59dd2) (@MarkCallow)

* Update dfdutils-included vulkan\_core.h. (#783) (9c223d950) (@MarkCallow)

* Add UTF-8 filename support on Windows to ktxtools (#788) (7b6eab5dc) (@aqnuep)

* Support for A8 and A1B5G5R5 formats (#785) (eeac6206c) (@aqnuep)

* KTXwriterScParams support (#779) (f8691ff05) (@aqnuep)

* Use \-- in doc. to get -- not n-dash in output. (#767) (724790094) (@MarkCallow)

* Document convert\_primaries option in toktx. (#765) (3049f5b5e) (@MarkCallow)

* Do target\_type changes only in toktx (#757) (2cf053c19) (@MarkCallow)

* Add support for fewer components in input files (#755) (adcccf152) (@aqnuep)

* Fix --convert-primaries (#753) (e437ec45f) (@aqnuep)

* Fix legacy app input from pipes on Windows. (#749) (3e7fd0af6) (@MarkCallow)

* Improve output determinism and add internal ktxdiff tool for comparing test outputs (#745) (7f67af7e0) (@VaderY)

* Add KTX\_WERROR config option (#746) (dab32db90) (@MarkCallow)

* Fix incorrect index calculations in image conversions (#735) (682f456de) (@VaderY)

* Color-space and documentation related improvements for ktx create (#732) (8b12216f7) (@VaderY)





### Java Wrapper

* Update dfdutils-included vulkan\_core.h. (#783) (9c223d950) (@MarkCallow)




## Version 4.3.0-alpha3


### Changes since v4.3.0-alpha2 (by part)
### libktx

Expand Down
2 changes: 1 addition & 1 deletion cmake/version.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ function( create_version_header dest_path target )
endfunction()

function( create_version_file )
file(WRITE ${PROJECT_BINARY_DIR}/ktx.version "${KTX_VERSION}")
file(WRITE ${PROJECT_BINARY_DIR}/ktx.version "${KTX_VERSION_FULL}")
endfunction()

# vim:ai:ts=4:sts=4:sw=2:expandtab:textwidth=70
Loading

0 comments on commit 29255c5

Please sign in to comment.