Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support 3D Tile I3dm legacy tile format #854

Merged
merged 39 commits into from
May 29, 2024
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
3ba003a
Make CESIUM_DEBUG_PREFIX and CESIUM_RELEASE_PREFIX cache variables
timoore Apr 7, 2024
ff427ac
Fix CesiumGltf::Model::merge to handle EXT_mesh_gpu_instancing
timoore Apr 7, 2024
dd563d5
Add support for I3dm 3D Tile instance files
timoore Apr 8, 2024
fe1f01d
use npm format properly
timoore Apr 9, 2024
72a8fb5
Fix convert calls in tests
timoore Apr 10, 2024
9b2bde2
remove const from static_cast to satisfy gcc
timoore Apr 11, 2024
b1c9f94
More formatting fun
timoore Apr 11, 2024
37f3684
Preserve double values from i3dm, WIP
timoore Apr 11, 2024
806d0ed
WIP glTF converter functions return a Future
timoore Apr 17, 2024
81bb2ff
Change ConverterSubprocessor arguments from pointer to reference
timoore Apr 17, 2024
7db60d6
Add std::move in response to compiler warning
timoore Apr 17, 2024
0a4a5a9
Merge branch 'main' into i3dm-2024
timoore Apr 17, 2024
616bf74
Rebase instance positions to their mean
timoore Apr 19, 2024
80ade66
I3dm: Support the EAST_NORTH_UP global semantic
timoore Apr 26, 2024
9661e40
Refactor and support existing instances in GLB
timoore Apr 26, 2024
4658109
Refactor with helpers for converting glTF values to glm.
timoore Apr 26, 2024
2cef550
Merge branch 'main' into i3dm-2024
timoore Apr 26, 2024
ab70a66
Merge branch 'main' into i3dm-2024
timoore May 2, 2024
db9637d
Merge branch 'main' into i3dm-2024
timoore May 3, 2024
1cd4ab7
Modify Cesium3DTilesContent tests to use ConverterSubprocessor
timoore May 7, 2024
5731d48
Squash some CI compilation errors
timoore May 7, 2024
d238025
Add a perpVec function and use it
timoore May 7, 2024
aca7686
Merge remote-tracking branch 'origin/main' into i3dm-2024
kring May 9, 2024
ea79bb8
Reponse to feedback: change ConverterSubprocessor to AssetFetcher
timoore May 16, 2024
068f70e
WIP responding to review comments
timoore May 17, 2024
598fd2e
Break i3dm JSON parsing out into a helper function
timoore May 17, 2024
7369873
Rework applyRTC and meshGpuTransforms
timoore May 17, 2024
88a3b8d
Use gsl:span instead of raw pointers
timoore May 17, 2024
be323a4
Move rotation function to CesiumUtility/Math.h
timoore May 17, 2024
c6b5321
Merge remote-tracking branch 'origin/i3dm-2024' into i3dm-2024
timoore May 17, 2024
ed9bfa8
Merge branch 'tmp/quatFunc' into i3dm-2024
timoore May 17, 2024
e60e936
format fixes
timoore May 17, 2024
edc22cb
Test compilation and gcc fixes
timoore May 20, 2024
9af4cd7
Add tests for CesiumUtility::Math::rotation and fix a bug
timoore May 23, 2024
d822cf4
Use a fail() local function in ImplicitOctreeLoader
timoore May 23, 2024
029d0ab
Renamed LegacyUtilities to GltfConverterUtility
timoore May 23, 2024
2a284f5
formatting and name change
timoore May 23, 2024
80a5091
More changes in response to code review
timoore May 24, 2024
ba68601
Renaming and simplification
timoore May 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
- Added `getNodeTransform`, `setNodeTransform`, `removeUnusedTextures`, `removeUnusedSamplers`, `removeUnusedImages`, `removeUnusedAccessors`, `removeUnusedBufferViews`, and `compactBuffers` methods to `GltfUtilities`.
- Added `postprocessGltf` method to `GltfReader`.
- `Model::merge` now merges the `EXT_structural_metadata` and `EXT_mesh_features` extensions. It also now returns an `ErrorList`, used to report warnings and errors about the merge process.
- Added support for I3dm 3D Tile content files.

##### Fixes :wrench:

Expand Down
8 changes: 4 additions & 4 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ if (NOT DEFINED GLOB_USE_CONFIGURE_DEPENDS)
)
endif()

set(CESIUM_DEBUG_POSTFIX "d")
set(CESIUM_RELEASE_POSTFIX "")
set(CESIUM_DEBUG_POSTFIX "d" CACHE STRING "debug postfix for cesium native")
set(CESIUM_RELEASE_POSTFIX "" CACHE STRING "release postfix for cesium native")

set(CMAKE_DEBUG_POSTFIX ${CESIUM_DEBUG_POSTFIX})
set(CMAKE_RELEASE_POSTFIX ${CESIUM_RELEASE_POSTFIX})
Expand Down Expand Up @@ -137,8 +137,8 @@ endfunction()
add_subdirectory(extern EXCLUDE_FROM_ALL)

# These libraries override the debug postfix, so re-override it.
set_target_properties(spdlog PROPERTIES DEBUG_POSTFIX ${CESIUM_DEBUG_POSTFIX})
set_target_properties(tinyxml2 PROPERTIES DEBUG_POSTFIX ${CESIUM_DEBUG_POSTFIX})
set_target_properties(spdlog PROPERTIES DEBUG_POSTFIX "${CESIUM_DEBUG_POSTFIX}")
set_target_properties(tinyxml2 PROPERTIES DEBUG_POSTFIX "${CESIUM_DEBUG_POSTFIX}")

# Public Targets
add_subdirectory(CesiumUtility)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

#include "GltfConverterResult.h"

#include <CesiumAsync/Future.h>
#include <CesiumGltf/Model.h>
#include <CesiumGltfReader/GltfReader.h>

Expand All @@ -10,9 +11,12 @@
#include <optional>

namespace Cesium3DTilesContent {
struct AssetFetcher;

struct B3dmToGltfConverter {
static GltfConverterResult convert(
static CesiumAsync::Future<GltfConverterResult> convert(
const gsl::span<const std::byte>& b3dmBinary,
const CesiumGltfReader::GltfReaderOptions& options);
const CesiumGltfReader::GltfReaderOptions& options,
const AssetFetcher& assetFetcher);
};
} // namespace Cesium3DTilesContent
Original file line number Diff line number Diff line change
@@ -1,20 +1,27 @@
#pragma once

#include <Cesium3DTilesContent/GltfConverterResult.h>
#include <CesiumAsync/Future.h>
#include <CesiumGltfReader/GltfReader.h>

#include <gsl/span>

#include <cstddef>

namespace Cesium3DTilesContent {
struct AssetFetcher;

struct BinaryToGltfConverter {
public:
static GltfConverterResult convert(
static CesiumAsync::Future<GltfConverterResult> convert(
const gsl::span<const std::byte>& gltfBinary,
const CesiumGltfReader::GltfReaderOptions& options);
const CesiumGltfReader::GltfReaderOptions& options,
const AssetFetcher& assetFetcher);

private:
static GltfConverterResult convertImmediate(
const gsl::span<const std::byte>& gltfBinary,
const CesiumGltfReader::GltfReaderOptions& options);
static CesiumGltfReader::GltfReader _gltfReader;
};
} // namespace Cesium3DTilesContent
Original file line number Diff line number Diff line change
@@ -1,16 +1,20 @@
#pragma once

#include <Cesium3DTilesContent/GltfConverterResult.h>
#include <CesiumAsync/Future.h>
#include <CesiumGltfReader/GltfReader.h>

#include <gsl/span>

#include <cstddef>

namespace Cesium3DTilesContent {
struct AssetFetcher;

struct CmptToGltfConverter {
static GltfConverterResult convert(
static CesiumAsync::Future<GltfConverterResult> convert(
const gsl::span<const std::byte>& cmptBinary,
const CesiumGltfReader::GltfReaderOptions& options);
const CesiumGltfReader::GltfReaderOptions& options,
const AssetFetcher& assetFetcher);
};
} // namespace Cesium3DTilesContent
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,12 @@
#include <CesiumGltf/Model.h>
#include <CesiumUtility/ErrorList.h>

#include <glm/common.hpp>
#include <glm/gtx/quaternion.hpp>

#include <optional>
#include <string>
#include <vector>

namespace Cesium3DTilesContent {
/**
Expand Down
57 changes: 49 additions & 8 deletions Cesium3DTilesContent/include/Cesium3DTilesContent/GltfConverters.h
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
#include "Library.h"

#include <Cesium3DTilesContent/GltfConverterResult.h>
#include <CesiumAsync/Future.h>
#include <CesiumAsync/IAssetAccessor.h>
#include <CesiumGltfReader/GltfReader.h>

#include <gsl/span>
Expand All @@ -12,6 +14,38 @@
#include <string_view>

namespace Cesium3DTilesContent {

struct ByteResult {
std::vector<std::byte> bytes;
timoore marked this conversation as resolved.
Show resolved Hide resolved
CesiumUtility::ErrorList errorList;
};

/**
* Object that makes a recursive request to fetch an asset, mostly for the
* benefit of i3dm files.
*/
struct CESIUM3DTILESCONTENT_API AssetFetcher {
AssetFetcher(
const CesiumAsync::AsyncSystem& asyncSystem_,
const std::shared_ptr<CesiumAsync::IAssetAccessor>& pAssetAccessor_,
const std::string& baseUrl_,
const glm::dmat4 tileTransform_,
const std::vector<CesiumAsync::IAssetAccessor::THeader>& requestHeaders_)
j9liu marked this conversation as resolved.
Show resolved Hide resolved
: asyncSystem(asyncSystem_),
pAssetAccessor(pAssetAccessor_),
baseUrl(baseUrl_),
tileTransform(tileTransform_),
requestHeaders(requestHeaders_) {}

CesiumAsync::Future<ByteResult> get(const std::string& relativeUrl) const;

const CesiumAsync::AsyncSystem& asyncSystem;
const std::shared_ptr<CesiumAsync::IAssetAccessor> pAssetAccessor;
const std::string baseUrl;
glm::dmat4 tileTransform; // For ENU transforms in i3dm
const std::vector<CesiumAsync::IAssetAccessor::THeader>& requestHeaders;
};

/**
* @brief Creates {@link GltfConverterResult} objects from a
* a binary content.
Expand All @@ -32,9 +66,10 @@ class CESIUM3DTILESCONTENT_API GltfConverters {
* @brief A function pointer that can create a {@link GltfConverterResult} from a
* tile binary content.
*/
using ConverterFunction = GltfConverterResult (*)(
using ConverterFunction = CesiumAsync::Future<GltfConverterResult> (*)(
const gsl::span<const std::byte>& content,
const CesiumGltfReader::GltfReaderOptions& options);
const CesiumGltfReader::GltfReaderOptions& options,
const AssetFetcher& subprocessor);

/**
* @brief Register the given function for the given magic header.
Expand Down Expand Up @@ -117,13 +152,16 @@ class CESIUM3DTILESCONTENT_API GltfConverters {
* the converter.
* @param content The tile binary content that may contains the magic header
* to look up the converter and is used to convert to gltf model.
* @param options The {@link CesiumGltfReader::GltfReaderOptions} for how to read a glTF.
* @param options The {@link CesiumGltfReader::GltfReaderOptions} for how to
* read a glTF.
* @param assetFetcher An object that can perform recursive asset requests.
* @return The {@link GltfConverterResult} that stores the gltf model converted from the binary data.
timoore marked this conversation as resolved.
Show resolved Hide resolved
*/
static GltfConverterResult convert(
static CesiumAsync::Future<GltfConverterResult> convert(
const std::string& filePath,
const gsl::span<const std::byte>& content,
const CesiumGltfReader::GltfReaderOptions& options);
const CesiumGltfReader::GltfReaderOptions& options,
const AssetFetcher& assetFetcher);

/**
* @brief Creates the {@link GltfConverterResult} from the given
Expand All @@ -142,12 +180,15 @@ class CESIUM3DTILESCONTENT_API GltfConverters {
*
* @param content The tile binary content that may contains the magic header
* to look up the converter and is used to convert to gltf model.
* @param options The {@link CesiumGltfReader::GltfReaderOptions} for how to read a glTF.
* @param options The {@link CesiumGltfReader::GltfReaderOptions} for how to
* read a glTF.
* @param assetFetcher An object that can perform recursive asset requests.
* @return The {@link GltfConverterResult} that stores the gltf model converted from the binary data.
timoore marked this conversation as resolved.
Show resolved Hide resolved
*/
static GltfConverterResult convert(
static CesiumAsync::Future<GltfConverterResult> convert(
const gsl::span<const std::byte>& content,
const CesiumGltfReader::GltfReaderOptions& options);
const CesiumGltfReader::GltfReaderOptions& options,
const AssetFetcher& assetFetcher);

private:
static std::string toLowerCase(const std::string_view& str);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#pragma once

#include <Cesium3DTilesContent/GltfConverterResult.h>
#include <CesiumAsync/Future.h>
#include <CesiumGltf/Model.h>
#include <CesiumGltfReader/GltfReader.h>

#include <glm/mat4x4.hpp>
#include <gsl/span>

#include <optional>

namespace Cesium3DTilesContent {
struct AssetFetcher;

struct I3dmToGltfConverter {
static CesiumAsync::Future<GltfConverterResult> convert(
const gsl::span<const std::byte>& instancesBinary,
const CesiumGltfReader::GltfReaderOptions& options,
const AssetFetcher& assetFetcher);
};
} // namespace Cesium3DTilesContent
124 changes: 124 additions & 0 deletions Cesium3DTilesContent/include/Cesium3DTilesContent/LegacyUtilities.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
#pragma once

#include <Cesium3DTilesContent/GltfConverters.h>
#include <CesiumAsync/IAssetAccessor.h>
#include <CesiumAsync/IAssetRequest.h>
#include <CesiumGltf/AccessorView.h>
#include <CesiumGltf/PropertyTransformations.h>
#include <CesiumUtility/ErrorList.h>

#include <glm/fwd.hpp>
#include <rapidjson/document.h>

#include <cstdint>
#include <optional>
#include <vector>

namespace CesiumGltf {
class Model;
class Buffer;
} // namespace CesiumGltf

namespace Cesium3DTilesContent {

namespace LegacyUtilities {
j9liu marked this conversation as resolved.
Show resolved Hide resolved
std::optional<uint32_t> parseOffsetForSemantic(
const rapidjson::Document& document,
const char* semantic,
CesiumUtility::ErrorList& errorList);

typedef bool (rapidjson::Value::*ValuePredicate)() const;

template <typename T> bool isValue(const rapidjson::Value& value);
template <typename T> T getValue(const rapidjson::Value& value);

template <typename T> std::optional<T> getOpt(const rapidjson::Value& value) {
timoore marked this conversation as resolved.
Show resolved Hide resolved
if (isValue<T>(value)) {
return std::make_optional(getValue<T>(value));
}
return {};
}

template <typename T>
std::optional<T>
getValue(const rapidjson::Document& document, const char* semantic) {
const auto valueIt = document.FindMember(semantic);
if (valueIt == document.MemberEnd() || !isValue<T>(valueIt->value)) {
return {};
}
return std::make_optional(getValue<T>(valueIt->value));
}

template <> inline bool isValue<bool>(const rapidjson::Value& value) {
return value.IsBool();
}

template <> inline bool getValue<bool>(const rapidjson::Value& value) {
return value.GetBool();
}

template <> inline bool isValue<uint32_t>(const rapidjson::Value& value) {
return value.IsUint();
}

template <> inline uint32_t getValue<uint32_t>(const rapidjson::Value& value) {
return value.GetUint();
}

bool validateJsonArrayValues(
const rapidjson::Value& arrayValue,
uint32_t expectedLength,
ValuePredicate predicate);

std::optional<glm::dvec3>
parseArrayValueDVec3(const rapidjson::Value& arrayValue);

std::optional<glm::dvec3>
parseArrayValueDVec3(const rapidjson::Document& document, const char* name);

int32_t
createBufferInGltf(CesiumGltf::Model& gltf, std::vector<std::byte> buffer = {});

int32_t createBufferViewInGltf(
CesiumGltf::Model& gltf,
const int32_t bufferId,
const int64_t byteLength,
const int64_t byteStride);

int32_t createAccessorInGltf(
CesiumGltf::Model& gltf,
const int32_t bufferViewId,
const int32_t componentType,
const int64_t count,
const std::string type);

void applyRtcToNodes(CesiumGltf::Model& gltf, const glm::dvec3& rtc);
timoore marked this conversation as resolved.
Show resolved Hide resolved

template <typename GlmType, typename GLTFType>
GlmType toGlm(const GLTFType& gltfVal);

template <typename GlmType, typename ComponentType>
GlmType toGlm(const CesiumGltf::AccessorTypes::VEC3<ComponentType>& gltfVal) {
return GlmType(gltfVal.value[0], gltfVal.value[1], gltfVal.value[2]);
}

template <typename GlmType, typename ComponentType>
GlmType
toGlmQuat(const CesiumGltf::AccessorTypes::VEC4<ComponentType>& gltfVal) {
if constexpr (std::is_same<ComponentType, float>()) {
return GlmType(
gltfVal.value[3],
gltfVal.value[0],
gltfVal.value[1],
gltfVal.value[2]);
} else {
return GlmType(
CesiumGltf::normalize(gltfVal.value[3]),
CesiumGltf::normalize(gltfVal.value[0]),
CesiumGltf::normalize(gltfVal.value[1]),
CesiumGltf::normalize(gltfVal.value[2]));
}
}

} // namespace LegacyUtilities
} // namespace Cesium3DTilesContent
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

#include "GltfConverterResult.h"

#include <CesiumAsync/Future.h>
#include <CesiumGltf/Model.h>
#include <CesiumGltfReader/GltfReader.h>

Expand All @@ -10,9 +11,12 @@
#include <optional>

namespace Cesium3DTilesContent {
struct AssetFetcher;

struct PntsToGltfConverter {
static GltfConverterResult convert(
static CesiumAsync::Future<GltfConverterResult> convert(
const gsl::span<const std::byte>& pntsBinary,
const CesiumGltfReader::GltfReaderOptions& options);
const CesiumGltfReader::GltfReaderOptions& options,
const AssetFetcher& assetFetcher);
};
} // namespace Cesium3DTilesContent
Loading
Loading