-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.html
138 lines (116 loc) · 7.27 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
<!DOCTYPE html><html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Auto Skinning</title>
<style>
*{ font-family: monospace; font-size:16px; }
body{ display:flex; flex-direction:column; align-items:center; }
div{ width:75vw; }
hr{ border-top:2px dotted gray; height:0px; }
h1{ font-size:20px; }
a{ text-decoration:none; color:orangered; }
a:hover{ color: orange; }
li{ margin-bottom: 8px; }
li p{ margin: 2px 0px 4px 0px; color:#505050;}
p.dim{ color:#606060; }
</style>
</head>
<body>
<div>
<h1>Auto Skinning</h1>
<div>
<p class="dim">
This repo will house various experiments on using web techologies to auto skinning 3D meshes. The process
requires heavy number crunching, so this process uses the GPU to execute several compute shaders
the produce the final vertex bone indices & weights. To start, its using raw WebGL over a Three.JS GL Context
to execute all the compute shaders. Most of the steps use GPGPU & Data Textures for computations as there
is a need for random read access which can only be done using texture lookups. The rest uses TransformFeedback
and GL buffers for computations & data storage.
</p>
<p class="dim">
Ultimately, the purpose of this repo is for prototyping one of the building blocks for creating a web application
similar to Adobe's Mixamo. If an app is able to use the user's GPU for the heavy lifting, then there is no need to
have a backend service to perform that process. Couple this with IK Animation retargeting thats already prototyped,
its quite possible to build a mixamo clone that runs everything on the front end.
</p>
<p>
<b>Source Code</b> : <a href="https://github.com/sketchpunklabs/autoskinning">https://github.com/sketchpunklabs/autoskinning</a>
</p>
</div>
<hr>
<table width="100%" cellspacing="5"><tr><td width="50%" valign="top">
<b>WebGL ( ThreeJS ) Prototypes</b>
<ul>
<li><a href="./prototypes/007_refactor.html">Refactor : First complete GPU Prototype</a>
<p>Recoding all the prototypes into a semi-messy API that executes the entire process onto
a mesh. Various visual debugging is available plus the mesh is then animated as a
final test if skinning data is usable.</p>
</li>
<li><a href="./prototypes/006_vertex_data.html">Compute vertex data</a>
<p>With voxel to bone distances known, this prototype then applies it to vertices. For each vertex,
we find which voxel its resides in then adds its distance to the voxel's mid point as
shortest path from the vertex to a bone.</p>
</li>
<li><a href="./prototypes/005_voxel_crawl.html">Compute voxel crawl</a>
<p>With the Voxel-Bone intersection as the starting point, the crawl will then step iterate
threw out the voxel chunk turning on neighboring voxels that are connected on a path to
a bone. Each voxel gets assigned the shortest distance to the bone.</p>
</li>
<li><a href="./prototypes/004_raycast_bones.html">Raycast for procedural skeleton generation</a>
<p>This is a side prototype for procedurally setting up a skeleton on a mesh using
2D points. These points will then be turned into raycasts that will find the
entry & exit positions when it intersects a mesh. It then computes the center
of the two points as the final position for a bone.
</p>
</li>
<li><a href="./prototypes/003_vox_bone_intersect_gltf.html">Compute Voxel-Bone Intersection with GLTF Mesh</a>
<p>Same as previous, but this time using a 3D model of a sworld</p>
</li>
<li><a href="./prototypes/003_vox_bone_intersect.html">Compute Voxel-Bone Intersection</a>
<p>This prototype turns skeleton data into a data texture that then a compute shader uses
for voxel intersection testing on the solid voxel shape from the previous prototype.
This step is to find all the voxels that containing any of the bones.
</p>
</li>
<li><a href="./prototypes/002_vox_fill_2.html">Compute voxel fill</a>
<p>Previous prototype creates a voxel shell, for auto skinning this shell needs to be filled in.
So this prototype goes threw & tries to fill in all the empty space inside the voxel shape.
</p>
</li>
<li><a href="./prototypes/001_gpgpu_vox_norm.html">Compute voxelize a mesh & debug normals</a>
<p>Same as before, this time storing the average normal of all the triangles inside a voxel.
The data is then visualized for debugging. Normal data is needed for the next step in
auto skinning.
</p>
</li>
<li><a href="./prototypes/001_gpgpu_vox.html">Compute voxelize a mesh</a>
<p>This prototype turnes a mesh into a data texture that then can be used by a compute
shader that will then create voxel data. The resulting voxel data is then visualized for debugging.
</p>
</li>
<li><a href="./prototypes/000_gpgpu.html">GPGPU & DataTextures</a>
<p>Implementing a simple API to execute shaders as compute that saves results into a data texture.
Shader & Texture objects created using raw WebGL code that ties into ThreeJS's GL Context</p>
</li>
</ul>
</td><td width="50%" valign="top">
<b>WebGPU Prototypes</b>
<ol>
<li>Coming when I have time</li>
</ol>
</td></tr></table>
<hr>
<b>Resources</b>
<ol>
<li><a href="https://x.com/_naam/status/1436730675319672832?s=20" target="_blank">@_naam twitter post that inspired this work</a></li>
<li><a href="http://www.delasa.net/data/sca2013_voxelization.pdf" target="_blank">White Paper : Geodesic Voxel Binding for Production Character Meshes</a></li>
<li><a href="https://bitbucket.org/sketchpunk/sketchpunk.bitbucket.io/src/master/src/threejs/autoskinning/index.html" target="_blank">CPU Prototype based on white paper</a></li>
<li><a href="http://blog.wolfire.com/2009/11/volumetric-heat-diffusion-skinning/" target="_blank">Wolfire Volumetric Head Diffusion Skinning</a></li>
<li><a href="http://blog.wolfire.com/2009/11/Triangle-mesh-voxelization" target="_blank">Wolfire Triangle-Mesh Voxelization</a></li>
<li><a href="https://bronsonzgeb.com/index.php/2021/06/26/volumetric-heat-diffusion-for-automatic-mesh-skinning/" target="_blank">Bronsonzgeb - Automatic Mesh Skinning in Unity ( Incomplete )</a></li>
<li><a href="https://webglfundamentals.org/webgl/lessons/webgl-gpgpu.html" target="_blank">WebGL - GPGPU Tutorial</a></li>
</ol>
</div>
</body>
</html>