OMI glTF Extensions Brainstorm

[robertlong]

Submit your ideas for OMI glTF extensions. These can be support for features from game engines such as physics or additional materials. They could be extensions specific to avatars or equipable items. Anything you'd like to see support for in the glTF ecosystem that you think the OMI community should push forward.

These don't have to be formal proposals. That will be a step later on. The goal of this conversation is to identify extensions that we all want.

Be sure to upvote if you'd like to see an extension developed further and comment if you'd be willing to help champion it.

[robertlong]

Scene Backgrounds / Skyboxes

I think we should support scene backgrounds / skyboxes with equirectangular cubemaps. People already accomplish this by including a sky dome and the associated geometry in their scene. However, in practice, this can result in incorrect draw order, clipping in large scenes, complex geometry being used for the skydome, and more. It'd be much better if we could just specify a texture to use as the scene background.

[robertlong]

Physics

This could be a hard one to specify since physics engines can represent things in pretty different ways, however if we could find a way to specify colliders, it'd mean that character controllers would be able to accurately collide with a scene. I'd ideally like to static, dynamic, and kinematic colliders. Such that we could have collision meshes for the scene, moving platforms, and dynamically movable objects.

[robertlong]

Shadows

glTF has support for defining lights, but not shadows. We should add flags for whether or not an object casts or receives shadows. We also should add a flag for whether a light casts shadows. We could also add properties for lights such as shadow bias, normal bias, radius, resolution, etc.

[robertlong]

Visibility

This is a simple one. You may create an object in a glTF scene and not want to initially show it. A visibility extension would have a single boolean telling the runtime to hide the object.

[aaronfranke]

A few years after this brainstorm, Khronos made such an extension here: KhronosGroup/glTF#2410

[robertlong]

Animation

Animation graphs and timelines can quickly get complex. It'd be nice at least to be able to specify that an animation should auto play and loop on load.

[robertlong]

Light Maps

The Mozilla Hubs team put effort into making this easy already. Maybe we can work with them to formalize their spec?

[robertlong]

Referenced glTF

It'd be great to be able to compose a glTF that references external glTF scenes. Perhaps we could design the spec in such a way where you can pull out pieces of a referenced glTF and reuse them multiple times. Like an asset collection and being able to reuse an asset in a scene multiple times. Another usecase would be to dynamically generate a referenced glTF asset at request time.

[mikeskydev]

I think this will be really important for creating larger scenes from pre-existing assets. I'd love to use an extension like this to represent gallery spaces.

[aaronfranke]

Khronos is working on a format for this here: https://github.com/KhronosGroup/glTF-External-Reference

[robertlong]

Video Textures

You should be able to use a video file or stream as a texture.

[robertlong]

Triggers

This would be an extension for providing basic behaviors to glTF assets. Animation keyframes, collision events, and runtime events could be observed, optionally passed through operators, and then passed to targets. This could start with basic operations on the glTF asset and extensions, but expand much further through the use of external events.

Observers

• Animation Observer
• Collision Observer
• Event Observer

Operators

• Boolean Operators
• State Operators
• Arithmetic Operators

Targets

• Animation Target
• Audio Target
• Transform Target
• Visibility Target
• Event Target
• Material Property Target

[robertlong]

glTF IDs

The glTF spec uses indices as identifiers. Sometimes these indices change between edits to the file. Names are not guaranteed to exist or be unique, so you can't reliably use them as identifiers. This is fine when you know the exact content of your scene because you can refer to a node by index or name. However, it'd be nice to have an identifier that is specified as a unique identifier that is stable between edits. This extension would focus only on adding formally specified unique string identifiers.

[adampaigge]

glTF NFTs

Not sure about how this would be implemented but, a metadata tag to track token contracts (currently working on BSV, but also tracking IPFS hashes) so that files can have referenced ownership based on some key or token in the tag would be useful for the crypto art community to start standardising the way things can be pulled into experiences.

Would also allow experiences to reference a NFT token or contract address to actually fetch the file from IPFS or from a chain directly in the case of bitcoinfiles. (see link for more information on how this implemented in practice)

8b0976a20f21e26b126612b0cc550f3cb665143f5dc5409bc85e9a18d21bf6ff.mp4

[humbletim]

Preserve Scene Hierarchy

on 2021-08-24, community member @dysbulic suggested this on the omi-gltf-extensions discord :

I have long been frustrated by the fact that when a file is exported from Blender to glTF, it flattens the collections hierarchy to a single level.

I have created a simple example with nested objects: https://github.com/dysbulic/stay/blob/react/public/cubes.gltf. The extension field is valid on any glTF object. ¿Can we not use that to mark that nodes are Collections rather than Objects and create the appropriate type when deserializing?

There is a Gitcoin Grants Round coming up Sept. 8ᵗʰ. I'd very much like to identify a GitHub issue to try and rally support for it in the funding process.

This, KhronosGroup/glTF-Blender-IO#1378, is my best guess so far. I've tried inquiring if it will fix the flattening issue, but, frustratingly, gotten no response.

My ultimate interest is a system for medium (weeks long) term rentals where 3D maps partitioned using trees are the sources of truth as to what's available.