Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Temporary PR to compare #971 directly against main #972

Closed
wants to merge 90 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
40806a2
Add ability for images to be shared between glTFs
azrogers Jul 10, 2024
7e5270a
Merge branch 'main' of github.com:CesiumGS/cesium-native into shared-…
azrogers Jul 10, 2024
a0954e3
Add test for shared images
azrogers Jul 23, 2024
dd8e3d0
Merge branch 'main' of github.com:CesiumGS/cesium-native into shared-…
azrogers Jul 23, 2024
8d2c5e1
clang-format
azrogers Jul 23, 2024
9bf3ac4
Fix unnecessary reference
azrogers Jul 23, 2024
2f622d7
Unused param
azrogers Jul 23, 2024
4e5bff4
Changes for Unreal implementation
azrogers Aug 16, 2024
e79b313
Merge with main
azrogers Aug 16, 2024
34ec18d
Revert adding extern folder back
azrogers Aug 16, 2024
8ae2150
Narrowing in on the correct set of extern files
azrogers Aug 16, 2024
0018493
Fix GltfReader I broke in merge
azrogers Aug 16, 2024
7e36d6c
Refactor to use IntrusivePointer for AssetContainer
azrogers Aug 23, 2024
2f852d0
Re-add thread safety that I accidentally reverted
azrogers Aug 23, 2024
a5829ef
Fix missing std::move
azrogers Sep 3, 2024
96cd924
Update based on review
azrogers Sep 11, 2024
8d38927
Merge with main
azrogers Sep 11, 2024
0e44d92
clang-format
azrogers Sep 11, 2024
8e664a1
Fix warnings breaking Unreal build
azrogers Sep 13, 2024
d612b32
Use template for factory so it can be moved into the future
azrogers Sep 17, 2024
5ae5539
Clean up SharedAssetDepot
azrogers Sep 18, 2024
0ce17c8
Fix wonky namespaces
azrogers Sep 18, 2024
4851ed2
Fix test
azrogers Sep 18, 2024
31d0048
Add timer for deleting assets
azrogers Sep 25, 2024
45365b2
Merge remote-tracking branch 'origin/main' into shared-assets
kring Sep 25, 2024
97909b8
Add AttributeSemantics.
kring Sep 26, 2024
e781882
Merge remote-tracking branch 'origin/all-with-void-futures' into shar…
kring Sep 26, 2024
6de6b38
Merge remote-tracking branch 'origin/main' into shared-assets-kring
kring Sep 29, 2024
86995ff
Allow constructor parameters to pass to ExtensibleObject::addExtension.
kring Sep 30, 2024
9fe3b4e
WIP error reporting for raster overlays.
kring Sep 2, 2024
95840fe
Fix compile errors in tests.
kring Sep 5, 2024
8ec646c
Propagate raster overlay load errors to the user.
kring Sep 5, 2024
03b60ef
Don't let failing overlays stop rendering completely.
kring Sep 6, 2024
f601680
errors -> errorList.
kring Sep 6, 2024
e83222c
Update CHANGES.md.
kring Sep 6, 2024
407be44
Fix clang error.
kring Sep 6, 2024
05e1af1
Show tileset if overlay tile provider can't be created.
kring Sep 17, 2024
d7bef24
Merge remote-tracking branch 'origin/main' into shared-assets
kring Sep 30, 2024
be51458
Merge remote-tracking branch 'origin/shared-assets' into shared-asset…
kring Sep 30, 2024
91f5b5f
Merge remote-tracking branch 'origin/main' into shared-assets
kring Oct 2, 2024
82fb7e8
Merge remote-tracking branch 'origin/shared-assets' into shared-asset…
kring Oct 2, 2024
e781a9c
AttributeSemantics -> VertexAttributeSemantics
kring Oct 3, 2024
cee6f21
Make SharedAsset a base class an eliminate AssetContainer.
kring Oct 4, 2024
ab21441
Fix water mask bug, add assets to depot.
kring Oct 4, 2024
275bac0
Better naming.
kring Oct 4, 2024
a88d83f
Move constructor for SharedAsset.
kring Oct 4, 2024
80502ad
Merge pull request #959 from CesiumGS/shared-assets-kring
azrogers Oct 4, 2024
e3e5665
Rework deletion
azrogers Oct 4, 2024
d6c7785
Merge branch 'shared-assets-image-cesium-only' of github.com:CesiumGS…
azrogers Oct 4, 2024
9574924
Format
azrogers Oct 4, 2024
aa81cbd
Merge pull request #960 from CesiumGS/shared-assets-image-cesium-only
azrogers Oct 4, 2024
924b000
Add accessors for deletion stats
azrogers Oct 8, 2024
21ecab9
Fix asset lifetime, avoid circular reference counting.
kring Oct 9, 2024
ee4b391
Remove TODO.
kring Oct 9, 2024
1b0934e
Don't use an iterator after it's erased.
kring Oct 9, 2024
8595163
getSizeBytes can use sizeInBytes or pixelData.size().
kring Oct 10, 2024
c8d9612
Remove unnecessary SharedAssetSystem::deletionTick.
kring Oct 10, 2024
6459729
Raster overlays store heap-allocated ImageCesiums.
kring Oct 10, 2024
36783ad
SharedAsset improvements...
kring Oct 10, 2024
e397591
Even independent assets can be shared.
kring Oct 10, 2024
a9516a0
Move SharedAsset[Depot] to CesiumAsync.
kring Oct 10, 2024
7f79837
GltfSharedAssetSystem
kring Oct 10, 2024
526e660
default -> getDefault
kring Oct 10, 2024
df1f887
Doc comments, rename ImageCesium
azrogers Oct 10, 2024
883e36f
Rename ImageAssetMipPosition
azrogers Oct 10, 2024
a230812
Merge pull request #965 from CesiumGS/shared-assets-tweaks
azrogers Oct 10, 2024
3e5eb71
Merge pull request #964 from CesiumGS/shared-assets-lifetime
azrogers Oct 10, 2024
38b4811
Pass factory to asset depot constructor
azrogers Oct 11, 2024
b220031
Doubly linked list for deletion candidates
azrogers Oct 11, 2024
083cb92
WIP refactoring.
kring Oct 25, 2024
727a68d
Introduce AssetEntry.
kring Oct 28, 2024
c75831f
Fix some problems that only showed up in Unreal.
kring Oct 28, 2024
c5533b7
Cleanup.
kring Oct 28, 2024
d16e3ef
Add doc.
kring Oct 28, 2024
49f1d63
Introduce TilesetSharedAssetSystem.
kring Oct 28, 2024
d43edce
Add ResultPointer.
kring Oct 29, 2024
7df4dc4
Keep depot alive if any live assets it manages are alive.
kring Oct 29, 2024
feaf448
Flesh out utility functions slightly.
kring Oct 29, 2024
c73f5b5
staleAssetSizeLimit -> inactiveAssetSizeLimitBytes
kring Oct 29, 2024
ff87332
Formatting.
kring Oct 29, 2024
fd601bc
Fix clang errors.
kring Oct 29, 2024
e9fac59
Fix incorrect comparison.
kring Oct 29, 2024
0961978
Fix a clang warning.
kring Oct 29, 2024
ce212af
Use brace initialization for base class.
kring Oct 29, 2024
172ce88
Merge remote-tracking branch 'origin/main' into shared-assets
kring Oct 29, 2024
117b65f
Merge branch 'shared-assets' into shared-assets-wip
kring Oct 29, 2024
1eee18f
Fix recursive mutex locking bug.
kring Oct 29, 2024
323de96
Fix clang warning.
kring Oct 29, 2024
a9d6c6a
Fix another clang warning.
kring Oct 29, 2024
c654653
Fix small typo
azrogers Oct 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ struct Rectangle;
}

namespace CesiumGltf {
struct ImageCesium;
struct ImageAsset;
struct Model;
} // namespace CesiumGltf

Expand Down
10 changes: 10 additions & 0 deletions Cesium3DTilesSelection/include/Cesium3DTilesSelection/Tileset.h
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,12 @@
#include <vector>

namespace Cesium3DTilesSelection {

class TilesetContentManager;
class TilesetMetadata;
class TilesetHeightQuery;
class TilesetHeightRequest;
class TilesetSharedAssetSystem;

/**
* @brief A <a
Expand Down Expand Up @@ -181,6 +183,14 @@ class CESIUM3DTILESSELECTION_API Tileset final {
/** @copydoc Tileset::getOverlays() */
const RasterOverlayCollection& getOverlays() const noexcept;

/**
* @brief Returns the {@link SharedAssetDepot} of this tileset.
*/
TilesetSharedAssetSystem& getSharedAssetSystem() noexcept;

/** @copydoc Tileset::getSharedAssetSystem() */
const TilesetSharedAssetSystem& getSharedAssetSystem() const noexcept;

/**
* @brief Updates this view but waits for all tiles that meet sse to finish
* loading and ready to be rendered before returning the function. This method
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#pragma once

#include <CesiumGltfReader/GltfSharedAssetSystem.h>

namespace Cesium3DTilesSelection {

/**
* @brief Contains assets that are potentially shared across multiple Tilesets.
*/
class TilesetSharedAssetSystem
: public CesiumGltfReader::GltfSharedAssetSystem {
public:
static CesiumUtility::IntrusivePointer<TilesetSharedAssetSystem> getDefault();

virtual ~TilesetSharedAssetSystem() = default;
};

} // namespace Cesium3DTilesSelection
4 changes: 3 additions & 1 deletion Cesium3DTilesSelection/src/Tile.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,9 @@ int64_t Tile::computeByteSize() const noexcept {

// sizeBytes is set in TilesetContentManager::ContentKindSetter, if not
// sooner (e.g., by the renderer implementation).
bytes += image.cesium.sizeBytes;
if (image.pCesium) {
bytes += image.pCesium->sizeBytes;
}
}
}

Expand Down
3 changes: 3 additions & 0 deletions Cesium3DTilesSelection/src/TileContentLoadInfo.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ TileContentLoadInfo::TileContentLoadInfo(
const std::shared_ptr<IPrepareRendererResources>&
pPrepareRendererResources_,
const std::shared_ptr<spdlog::logger>& pLogger_,
const CesiumUtility::IntrusivePointer<TilesetSharedAssetSystem>&
pSharedAssetSystem_,
const TilesetContentOptions& contentOptions_,
const Tile& tile)
: asyncSystem(asyncSystem_),
Expand All @@ -18,6 +20,7 @@ TileContentLoadInfo::TileContentLoadInfo(
tileID(tile.getTileID()),
tileBoundingVolume(tile.getBoundingVolume()),
tileContentBoundingVolume(tile.getContentBoundingVolume()),
pSharedAssetSystem{pSharedAssetSystem_},
tileRefine(tile.getRefine()),
tileGeometricError(tile.getGeometricError()),
tileTransform(tile.getTransform()),
Expand Down
4 changes: 4 additions & 0 deletions Cesium3DTilesSelection/src/TileContentLoadInfo.h
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
#include <Cesium3DTilesSelection/TileID.h>
#include <Cesium3DTilesSelection/TileRefine.h>
#include <Cesium3DTilesSelection/TilesetOptions.h>
#include <Cesium3DTilesSelection/TilesetSharedAssetSystem.h>
#include <CesiumAsync/AsyncSystem.h>
#include <CesiumAsync/IAssetAccessor.h>
#include <CesiumGeometry/Axis.h>
Expand All @@ -24,6 +25,8 @@ struct TileContentLoadInfo {
const std::shared_ptr<IPrepareRendererResources>&
pPrepareRendererResources,
const std::shared_ptr<spdlog::logger>& pLogger,
const CesiumUtility::IntrusivePointer<TilesetSharedAssetSystem>&
pSharedAssetSystem,
const TilesetContentOptions& contentOptions,
const Tile& tile);

Expand All @@ -40,6 +43,7 @@ struct TileContentLoadInfo {
BoundingVolume tileBoundingVolume;

std::optional<BoundingVolume> tileContentBoundingVolume;
CesiumUtility::IntrusivePointer<TilesetSharedAssetSystem> pSharedAssetSystem;

TileRefine tileRefine;

Expand Down
8 changes: 8 additions & 0 deletions Cesium3DTilesSelection/src/Tileset.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,14 @@ const RasterOverlayCollection& Tileset::getOverlays() const noexcept {
return this->_pTilesetContentManager->getRasterOverlayCollection();
}

TilesetSharedAssetSystem& Tileset::getSharedAssetSystem() noexcept {
return *this->_pTilesetContentManager->getSharedAssetSystem();
}

const TilesetSharedAssetSystem& Tileset::getSharedAssetSystem() const noexcept {
return *this->_pTilesetContentManager->getSharedAssetSystem();
}

static bool
operator<(const FogDensityAtHeight& fogDensity, double height) noexcept {
return fogDensity.cameraHeight < height;
Expand Down
23 changes: 19 additions & 4 deletions Cesium3DTilesSelection/src/TilesetContentManager.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -49,12 +49,15 @@ struct ContentKindSetter {

void operator()(CesiumGltf::Model&& model) {
for (CesiumGltf::Image& image : model.images) {
if (!image.pCesium)
continue;

// If the image size hasn't been overridden, store the pixelData
// size now. We'll be adding this number to our total memory usage soon,
// and remove it when the tile is later unloaded, and we must use
// the same size in each case.
if (image.cesium.sizeBytes < 0) {
image.cesium.sizeBytes = int64_t(image.cesium.pixelData.size());
if (image.pCesium->sizeBytes < 0) {
image.pCesium->sizeBytes = int64_t(image.pCesium->pixelData.size());
}
}

Expand Down Expand Up @@ -562,6 +565,9 @@ postProcessContentInWorkerThread(
tileLoadInfo.contentOptions.ktx2TranscodeTargets;
gltfOptions.applyTextureTransform =
tileLoadInfo.contentOptions.applyTextureTransform;
if (tileLoadInfo.pSharedAssetSystem) {
gltfOptions.pSharedAssets = tileLoadInfo.pSharedAssetSystem;
}

auto asyncSystem = tileLoadInfo.asyncSystem;
auto pAssetAccessor = tileLoadInfo.pAssetAccessor;
Expand Down Expand Up @@ -599,12 +605,12 @@ postProcessContentInWorkerThread(
"Warning when resolving external gltf buffers from "
"{}:\n- {}",
result.pCompletedRequest->url(),
CesiumUtility::joinToString(gltfResult.errors, "\n- "));
CesiumUtility::joinToString(gltfResult.warnings, "\n- "));
} else {
SPDLOG_LOGGER_ERROR(
tileLoadInfo.pLogger,
"Warning resolving external glTF buffers:\n- {}",
CesiumUtility::joinToString(gltfResult.errors, "\n- "));
CesiumUtility::joinToString(gltfResult.warnings, "\n- "));
}
}

Expand Down Expand Up @@ -660,6 +666,7 @@ TilesetContentManager::TilesetContentManager(
_tileLoadsInProgress{0},
_loadedTilesCount{0},
_tilesDataUsed{0},
_pSharedAssets(TilesetSharedAssetSystem::getDefault()),
_destructionCompletePromise{externals.asyncSystem.createPromise<void>()},
_destructionCompleteFuture{
this->_destructionCompletePromise.getFuture().share()},
Expand Down Expand Up @@ -689,6 +696,7 @@ TilesetContentManager::TilesetContentManager(
_tileLoadsInProgress{0},
_loadedTilesCount{0},
_tilesDataUsed{0},
_pSharedAssets(TilesetSharedAssetSystem::getDefault()),
_destructionCompletePromise{externals.asyncSystem.createPromise<void>()},
_destructionCompleteFuture{
this->_destructionCompletePromise.getFuture().share()},
Expand Down Expand Up @@ -840,6 +848,7 @@ TilesetContentManager::TilesetContentManager(
_tileLoadsInProgress{0},
_loadedTilesCount{0},
_tilesDataUsed{0},
_pSharedAssets(TilesetSharedAssetSystem::getDefault()),
_destructionCompletePromise{externals.asyncSystem.createPromise<void>()},
_destructionCompleteFuture{
this->_destructionCompletePromise.getFuture().share()},
Expand Down Expand Up @@ -985,6 +994,7 @@ void TilesetContentManager::loadTileContent(
this->_externals.pAssetAccessor,
this->_externals.pPrepareRendererResources,
this->_externals.pLogger,
this->_pSharedAssets,
tilesetOptions.contentOptions,
tile};

Expand Down Expand Up @@ -1227,6 +1237,11 @@ TilesetContentManager::getTilesetCredits() const noexcept {
return this->_tilesetCredits;
}

const CesiumUtility::IntrusivePointer<TilesetSharedAssetSystem>&
TilesetContentManager::getSharedAssetSystem() const noexcept {
return this->_pSharedAssets;
}

int32_t TilesetContentManager::getNumberOfTilesLoading() const noexcept {
return this->_tileLoadsInProgress;
}
Expand Down
8 changes: 8 additions & 0 deletions Cesium3DTilesSelection/src/TilesetContentManager.h
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@

namespace Cesium3DTilesSelection {

class TilesetSharedAssetSystem;

class TilesetContentManager
: public CesiumUtility::ReferenceCountedNonThreadSafe<
TilesetContentManager> {
Expand Down Expand Up @@ -115,6 +117,9 @@ class TilesetContentManager

const std::vector<CesiumUtility::Credit>& getTilesetCredits() const noexcept;

const CesiumUtility::IntrusivePointer<TilesetSharedAssetSystem>&
getSharedAssetSystem() const noexcept;

int32_t getNumberOfTilesLoading() const noexcept;

int32_t getNumberOfTilesLoaded() const noexcept;
Expand Down Expand Up @@ -167,6 +172,9 @@ class TilesetContentManager
int32_t _loadedTilesCount;
int64_t _tilesDataUsed;

// Stores assets that might be shared between tiles.
CesiumUtility::IntrusivePointer<TilesetSharedAssetSystem> _pSharedAssets;

CesiumAsync::Promise<void> _destructionCompletePromise;
CesiumAsync::SharedFuture<void> _destructionCompleteFuture;

Expand Down
1 change: 1 addition & 0 deletions Cesium3DTilesSelection/src/TilesetJsonLoader.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1005,4 +1005,5 @@ void TilesetJsonLoader::addChildLoader(
std::unique_ptr<TilesetContentLoader> pLoader) {
this->_children.emplace_back(std::move(pLoader));
}

} // namespace Cesium3DTilesSelection
2 changes: 2 additions & 0 deletions Cesium3DTilesSelection/src/TilesetJsonLoader.h
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

#include <Cesium3DTilesSelection/TilesetContentLoader.h>
#include <Cesium3DTilesSelection/TilesetExternals.h>
#include <Cesium3DTilesSelection/TilesetSharedAssetSystem.h>
#include <CesiumAsync/Future.h>
#include <CesiumAsync/IAssetAccessor.h>

Expand Down Expand Up @@ -55,6 +56,7 @@ class TilesetJsonLoader : public TilesetContentLoader {
private:
std::string _baseUrl;
CesiumGeospatial::Ellipsoid _ellipsoid;
CesiumUtility::IntrusivePointer<TilesetSharedAssetSystem> _pSharedAssets;

/**
* @brief The axis that was declared as the "up-axis" for glTF content.
Expand Down
34 changes: 34 additions & 0 deletions Cesium3DTilesSelection/src/TilesetSharedAssetSystem.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
#include <Cesium3DTilesSelection/TilesetSharedAssetSystem.h>

using namespace Cesium3DTilesSelection;
using namespace CesiumAsync;
using namespace CesiumGltf;
using namespace CesiumGltfReader;
using namespace CesiumUtility;

namespace {

CesiumUtility::IntrusivePointer<TilesetSharedAssetSystem> createDefault() {
CesiumUtility::IntrusivePointer<TilesetSharedAssetSystem> p =
new TilesetSharedAssetSystem();

CesiumUtility::IntrusivePointer<GltfSharedAssetSystem> pGltf =
GltfSharedAssetSystem::getDefault();

p->pImage = pGltf->pImage;

return p;
}

} // namespace

namespace Cesium3DTilesSelection {

/*static*/ CesiumUtility::IntrusivePointer<TilesetSharedAssetSystem>
TilesetSharedAssetSystem::getDefault() {
static CesiumUtility::IntrusivePointer<TilesetSharedAssetSystem> pDefault =
createDefault();
return pDefault;
}

} // namespace Cesium3DTilesSelection
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ class SimplePrepareRendererResource
}

virtual void* prepareRasterInLoadThread(
CesiumGltf::ImageCesium& /*image*/,
CesiumGltf::ImageAsset& /*image*/,
const std::any& /*rendererOptions*/) override {
return new AllocationResult{totalAllocation};
}
Expand Down
Loading
Loading