The new 3D engine of TeiaViewer: mesh storage, scene BVH, picking, rendering (WebGPU, with a WebGL2 fallback) and the file loading contract. Headless (no React), built on three.js.
Status: under construction, published to the feed with the other packages.
@stereograph/teiaviewer depends on it and is being moved onto it (v20): its containers, BoxTree
and render path replaced by the engine's, while its scene graph (SceneObject), object state and
commands stay in the viewer. The engine depends on no other package of this repository.
import { PerspectiveCamera } from 'three';
import { EngineSettings, MeshLoader, Renderer, Scene } from '@stereograph/teiaviewer-engine';
import { MgeomReader } from './MgeomReader'; // a FileReader of the application
// The scene's settings, its renderer's too: a config of any values, e.g. loaded from a storage.
const settings = new EngineSettings({ postProcessing: { ambientOcclusion: { enabled: false } } });
const scene = new Scene(settings);
const camera = new PerspectiveCamera(50, 1, 0.1, 1000);
// WebGPU, else WebGL2 (settings.renderer.backend), on the application's canvas.
const renderer = await Renderer.create({ canvas, scene }); // its scene's settings
renderer.setCamera(camera);
renderer.setSize(canvas.clientWidth, canvas.clientHeight);
const loader = new MeshLoader(scene);
loader.addFileReader(new MgeomReader());
loader.signals.updated.connect((progress) => console.log(progress));
const progress = await loader.load([{ url }]); // read in workers; where the load ended
await scene.boxTree.whenApplied(); // the BoxTree has the objects: bounds, picking
// Frames are drawn on demand: ask for one when the camera or the data changes.
renderer.requestFrame();
// The object seen under a ray (e.g. through the pointer), selected through its proxy.
const [hit] = scene.raycaster.intersect(ray, camera, []);
if (hit && 'mesh' in hit) {
hit.mesh.selected = true;
renderer.requestFrame();
}
// Settings apply live, validated, a frame drawn.
settings.meshes.xRayOpacity.value = 0.3;
The viewer (@stereograph/teiaviewer) is built on the engine: its demo (apps/demo,
npm run dev) shows it, with its settings in the developer tools.
| Folder | Main classes | Role |
|---|---|---|
mesh/ |
MeshController, MeshProxy, MeshObjectTable, InstanceTable, PartRaycaster, GeometryStore, GeometryContainer, GeometryBuffersConverter |
Objects (InputMesh in, a MeshProxy out): geometries copied once into large Containers, one instance per part, their parts' world boxes in a BVH; changed through their proxies, read per frame. Three.js geometries converted into GeometryBuffers (GeometryBuffersConverter, sharp edges found by SharpEdgeDetector) |
spatial/ |
BoxTree, buildSubtree, PartFlags, LeafIds, BoxTreeHelper |
The BVH of world boxes (shared with the culling workers), its leaf ids (a part's or a sprite's) and flags (shown, ignoring the cutting planes, a sprite, layer drawn in), its nodes drawn for debugging (BoxTreeHelper, scene.boxTreeHelper): the lowest layer, on three.js and utils/ only |
sprite/ |
SpriteController, SpriteProxy, SpriteTable |
Sprites (InputSprite in, a SpriteProxy out): images facing the camera, one draw per layer of its shown sprites, their leaves in the scene's BoxTree |
texture/ |
TextureArray |
Images in the layers of a texture array, shared by key (the sprites', later the meshes') |
scene/ |
Scene, Raycaster, Lighting, CuttingPlanesController, CuttingPlanesHelper |
The scene composed (meshes, sprites, their BoxTree, cutting planes, origin, picking, lighting, the three.js scene drawn), ray queries |
render/ |
Renderer, WebGL2 and WebGPU backends |
Drawing (webgl/ and webgpu/, each split in mesh/ and sprite/), culling, layers, post-processing |
io/ |
FileReader, MeshLoader, ObjectImportController, fetchFile |
File loading in workers: formats given by the application |
Every value of the engine is a setting, in one EngineSettings (settings/), by theme: scene
(picking, uploads, BoxTree), meshes (selection, x-ray, edges, storage), sprites, lighting,
renderer (pixel ratio, background, GPU buffers), postProcessing (antialiasing,
ambient occlusion), culling and loading, a class each (a Settings: its title, and its
icon, a SettingsIcon, for an interface to show it). A setting is an observable value
(settings/base/: NumberController, BooleanController, ColorController, SelectController),
validated (bounds, options), described for an interface (description). The scene's (new Scene(settings)), its renderer's too: Renderer.create({ scene }) draws one scene, its settings
scene.settings; they follow it and leave it to its owner when disposed. What an interface shows
applies live, a frame drawn; the technical settings are hidden, most read at the creation of their
object: they apply at the next page load. new EngineSettings(config) and applyConfig(config)
take any of the values, theme by theme (EngineConfig, as the viewer's config);
saveToLocalStorage(key) keeps the values changed from their default in the local storage (e.g.
when the page closes), loadFromLocalStorage(key) sets them back (before the scene and the renderer
are made).
An object is a MeshProxy (MeshController.addMeshes returns them), a row of the
MeshObjectTable, with one or more parts (e.g. one per material; the viewer's sub-meshes sharing a
uuid): the proxy holds their instance indices, in the order of its input, and its controller. No
state in the proxy (its getters and setters read and write the controller), no JavaScript object
per part: each part is an instance of the InstanceTable, its world matrix,
16 bytes of properties (colour, opacity, roughness, metalness, emissive, edge colour, layer, flags
visible, selected, x-ray, ignoring the cutting planes, faces hidden and edges hidden) and the index of its geometry's slot, in
the format the GPU reads. A removed object's proxy is invalidated (isValid): its setters do
nothing and its getters throw, even once its row went to a later object.
The engine keeps no uuid and knows no duplicate: each input added is a new object. The
application names its objects (InputMesh.uuid, name, tag, parentUuid, read by its
ObjectImportController, never stored) and links them to its own through the proxy's owner
(read back from a hit): e.g. an object with a 3D and a 2D representation is two objects of the
engine, one hidden (the viewer's BimMesh). The engine is a low-level layer: the viewer builds
the comfortable API on it (uuids, its objects, undo). A part draws
its geometry's primitive (GeometryBuffers.primitive): triangles (with their sharp edges), or
lines (e.g. a 2D drawing's), the segments of a Container stored with the sharp edges. The
table is the only state of the objects: what follows from it (the layer an instance is drawn in,
its pass) is computed where it is used, on the CPU and the GPU. Whether an object is shown is an
input of the application, never written by the culling, whose result is a list of the parts to
draw. The BoxTree's leaves hold a copy of it and of the layer drawn in, which the culling workers
read in place with the boxes (a hidden subtree skipped at its node, the occlusion of a node tested
in its highest layer); the copy follows the table a message later.
The MeshController is the only writer of its objects. They are added and removed many at once
(addMeshes, removeMeshes: one subtree inserted, or the rest of the tree rebuilt); each is
moved (applyMatrix4 composes a transform, setMatrix places it at a world matrix, every part
or one), shown, hidden, moved to another layer, (de)selected, x-rayed, or drawn without its
faces (facesVisible: edges only, a wireframe) or without its sharp edges (edgesVisible)
through its proxy (mesh.visible = false), which calls the controller: it updates its instances, without
allocating, and queues the changes of its parts' leaves for its BoxTree's worker, the changes of
a task merged into one command per type (see Scene BVH). A loop over many proxies is as fast as a
batch: 100k objects shown or hidden in ~30 ms, applied by the worker.
It tells nobody, as three.js's needsUpdate: a write only marks what changed, and whoever needs
it reads it when it uses it, comparing with what it stored. The renderers do once per frame:
MeshController.version (a counter of the changes of what is drawn or culled: culled again),
the InstanceTable's allocationVersion (objects came or went: draws planned again) and changed
ranges (uploaded), and the geometries' changed ranges (their table entries written). Nothing is
mirrored: a renderer reads a part at its instance index (its geometry's slot,
MeshController.getPartSlot), the GPU an instance's geometry by its slot index in the table. A million changes between two frames cost a
million counter increments, and one reading. The
application knows its own changes (its commands): the engine has no signal for them. The Scene
composes the meshes (scene.meshController), the sprites (scene.spriteController), their
BoxTree (scene.boxTree), the frame origin (clear() forgets it), the Raycaster and the
three.js scene the Renderer draws (scene.threeScene: the meshes' and the sprites' groups,
filled by the backend, and the BoxTree helper); it tells the tree's changes
(scene.signals.changed, a frame asked for by the backend). The scene does not know what draws it: one backend at most draws a scene (one GPU
context, several views sharing it), which the backends check among themselves.
The GeometryStore of the MeshController copies the geometries once per id into
GeometryContainers that never grow: the new geometries of an addition larger than a default
Container (1M indices, 10 MB, see settings.meshes.container*), e.g. a group being loaded, go
together to Containers of their exact size (settings.meshes.containerMaxMB at most, each array
under the 128 MiB WebGPU binds); the others go where there is room, a default Container made when
none has. A Container is freed with its last geometry, the renderers freeing their copies; one empty
Container of the default size at most is kept for the next geometries. The geometries are shared by
their parts (counted, freed with the last one), the same data for both backends: a new backend
copies no geometry. Indices are 32-bit, rebased on the Container. A geometry's BVH
(three-mesh-bvh) is built on its first raycast.
A three.js BufferGeometry becomes GeometryBuffers with the GeometryBuffersConverter:
convertGeometry reads its positions, normals (to snorm8) and indices and finds its sharp edges
(SharpEdgeDetector, 30°, as three.js EdgesGeometry), convertLine gives a line's segments.
Each conversion is a new geometry for the engine (an id of its own): parts share a geometry by
sharing the converted buffers. The glTF reader of the viewer uses it, as an application does
for the meshes it makes (e.g. the zones of the Zone plugin).
MeshProxy.createNormalsHelper(size) (MeshController.createNormalsHelper) gives a
MeshNormalsHelper, a debug drawing of the object's normals read from its Containers: a segment
per vertex (the snorm8 normals), or per triangle from its centre for a geometry stored without
normals (lit flat by the shader); its lines left out. It is drawn as it was made: update it once
the object moves.
The BoxTree of the scene (scene.boxTree) holds the world boxes of the mesh parts and of the
sprites, a leaf each, in a growable SharedArrayBuffer (1M nodes at first, then steps of 1M; its
views have a fixed length, replaced when it grows). The id space of the leaves is split in two
(LeafIds): below 2^30 a part's instance index, from it a sprite's index plus 2^30, the tree's
index of the leaves by id a dense array per half. Both carry the same flags (shown, layer drawn
in); each controller writes its own leaves (the sprites are markers, placed now and then: their
changes cull the tree again with the parts). The main thread does no work on it but read it
(BoxTreeReader: picking, bounds), as the culling workers do (which leave the sprites out for
now):
MeshProxy.active / SpriteProxy.active (MeshController.setActive,
SpriteController.setActive) take an object's leaves out of the tree, its geometry, image and
state kept, and put them back: an object the application keeps aside (e.g. deleted, its deletion
undoable) is neither bounded, culled in nor picked. Its moves and flags wait for its return.BoxTree talks with its BoxTreeWorker, the only writer of the
buffer, through its BoxTreeStructure (insertions, removals, refits, rebuilds). The
MeshController sends it the changes (leaves added, moved, removed, their flags, the
tree emptied), gathered and posted at once at the end of the task, applied there in order; the
moves, removals and flags made in a row are merged into one command, so a loop over objects one
by one is refitted once (each insertion stays a subtree of its own). The moves and the flags,
independent, merge past each other: a loop moving and hiding objects one by one sends two
commands, not two per object. An insertion, a removal or a reset stops the merge: they keep
their order. A transfer costs per buffer (400k small buffers: ~45 s in Node), so the arrays of
the small commands are copied into two arrays of the message, transferred alone, the worker
getting views on them: a message transfers a few buffers whatever its commands (100k insertions
one by one: ~0.2 s).
The tree follows them a message later: boxTree.whenApplied() resolves once it has them, and
boxTree.signals.changed tells each change of the buffer (the WebGL backend culls again). The
readers walk the tree while it changes: a write in progress may hide a leaf for that instant.MeshController.addMeshes sends the parts of the objects added as one subtree
(buildSubtreeMorton over their boxes, a linear BVH, a pure function on typed arrays, then
addSubtree), much faster than one insertion per part. A leaf or a subtree is inserted beside
the node where it adds the least surface area, its ancestors refitted and rotated (a grandchild
swapped with its uncle when it lowers the area), as on a removal: leaves added one by one in
order (along a line) keep a tree as good as a rebuilt one, where it became a chain (20,000
leaves: 73 s, against 0.5 s). removeMeshes frees many parts by a rebuild of the rest.settings.scene.boxTreeRebuildDelayMs (a load in progress keeps adding), the worker measures the
tree: the surface area of its internal nodes over that of its leaves. The first time, or when this
ratio went over the one of its last rebuild times settings.scene.boxTreeRebuildCostRatio, it
sends the leaves' boxes to a builder worker, which remakes the tree by binned SAH
(buildSubtree); the BoxTreeWorker goes on applying the changes meanwhile, and writes the new
tree beside the old one, the root switched in one write, if the leaves did not change since (else
it tries again once idle). boxTree.signals.rebuilt tells each rebuild, measured.Raycaster.intersect(ray, camera) gives the visible parts and shown sprites a ray hits, sorted
as seen: by layer drawn (a higher layer is drawn over the lower ones whatever their depth), then
distance. A sprite is hit within its rectangle facing the camera. Every raycast level takes an
options object (RaycastOptions; SceneRaycastOptions for the scene). onlyFirst (default
true): only the hit seen, without raycasting (nor building the BVH of) the parts that cannot
give it: the BoxTree is walked front to back (BoxTreeRayQuery, nearest box first through a
min-heap), layer by layer from the highest, its flags skipping the subtrees without a shown object
of the layer, until the next box starts farther than the best hit. Its flags follow the objects a
message later: while the tree has changes to apply, the hit seen comes from all the boxes the ray
crosses, sorted (exact either way). A part's nearest triangle comes from three-mesh-bvh's
raycastFirst. viewportHeight (the scene's): the height in pixels of the view the ray goes
through; a part drawing lines is then hit by its nearest segment the ray passes within
settings.scene.lineThresholdPixels of on screen (without it, only a segment crossed). The
Raycaster turns it into a LineThreshold with the camera (LineThreshold.fromCamera: base + perDistance × distance along the ray), the option of a part's raycast (PartRaycaster, which
has no camera); the segments are tested one by one in the part's local frame, only the boxes of
the parts drawing lines widened for it. A hit names its object by proxy (hit.mesh and
the rank of its part, or hit.sprite; 'sprite' in hit tells which), its owner the
application's object. The quads of the cutting planes are hit too while their helper is shown,
after every object, as they are drawn behind them (hit.cuttingPlane, its CuttingPlaneProxy;
'cuttingPlane' in hit tells it). filter: the objects hit (SceneRaycastObject: a MeshProxy,
a SpriteProxy or a CuttingPlaneProxy, told by isMeshProxy, isSpriteProxy,
isCuttingPlaneProxy; e.g. by their owner), checked before an object's raycast: the ray goes
through the others without raycasting them, and onlyFirst keeps its early stop. transparent
(default true): with false, the ray goes through the parts drawn transparent (opacity below
settings.meshes.opaqueAbove, x-ray included), skipped the same way.
The Raycaster caches nothing: each query raycasts, its hits the caller's. The application caches
its own (e.g. the hits of one pointer event, shared by its tools).
requestFrame). The caller asks after its
changes (camera, objects, size, scene), e.g. at the end of a command; the engine asks
for the frames of its own asynchronous work, which the caller does not see (a culling result,
the occlusion settling, a buffer grown), and of a setting changed.RenderCamera): Renderer.setSize fits it to the
canvas's aspect (a perspective camera's aspect; an orthographic camera's left and right around
their centre, its vertical extent kept). Orthographic, the edges are moved nearer along the view
axis (not their view ray), the lighting views along the camera's axis, and the transparency weighs
the depth linearly (its near plane may be at 0 or behind it); the culling and the cutting planes
are the same (projected with the view-projection).Renderer.create, kept: a canvas keeps the backend it first drew with.
The setting settings.renderer.backend picks it: auto (the default) draws with WebGPU when
the browser can (adapter and device granted, asked for before the canvas is touched), else
WebGL2; webgpu falls back to WebGL2 too, so that a saved setting never leaves the page without
a drawing. At a change of the setting, the renderer reloads the page: the application saves its
settings then (e.g. at beforeunload).EXT_clip_control).settings.culling.minPixels) and occlusion against a
depth pyramid of the previous frames; in workers on WebGL2, in compute passes on WebGPU.MeshLayers. Each one above the base is drawn in its own target and
composed over the lower ones: a layer hides the lower layers, never the upper ones. A selected
object (MeshProxy.selected) is drawn in the selection colours, in the top layer with
settings.meshes.selectionOverlay.Scene.toolScene (a gizmo, a cursor) are
drawn last, after Scene.threeScene, over everything whatever their depth, without the
post-processing: on the canvas in WebGL2; in WebGPU in their own target, composed over the final
image (three would draw them to the canvas through a target of its own, in place of the image).settings.meshes.xRayOpacity for an x-ray object (MeshProxy.xRay, its edges'
too). Opaque from settings.meshes.opaqueAbove.render/common/WeightedTransparency.ts): the transparent meshes (faces, lines, x-ray edges) are
on a three.js layer of their own, drawn apart once their layer's opaque meshes are, into two
targets sharing that layer's depth (tested, not written): the colours times their alpha and
weight summed with their revealage (RGBA16F), the weights summed (R16F); then composed over the
layer, their weighted mean covering 1 - revealage of it. One geometry pass, any draw order
gives the same image; the weight falls with the log depth between the camera's planes (whatever
the units; linear for an orthographic camera) and rises with the opacity; the segments' is raised, so that they show over the
faces they lie on; the sums are clamped to finite half floats when composed. Approximate:
several nearly opaque surfaces mix rather than the nearest covering the others. 10 bytes per
pixel, made on the first frame that needs them and kept until a resize: one pair of targets
shared by the layers in WebGL (its depth switched), one per layer that drew transparent meshes
in WebGPU (three caches a target's passes without its depth), about 150 MB at most in 4K.settings.culling.minPixels, occlusion, opacity), drawn after the faces of their pass,
whatever the edges' settings. Same segment buffers and emit passes as the edges, in passes of
their own.settings.culling.minEdgePixels), after the lines of their pass: opaque, weighted for
x-ray objects.scene.lighting on settings.lighting, read each frame): Lambert diffuse and GGX
specular for a head light (from the camera) and a sun (a fixed direction), an environment map for
the ambient light and the reflections (three.js's PMREM, prefiltered by the backend when another
or updated image is set; three.js's RoomEnvironment by default), plus the emission, tone mapped
in the faces' shader (Khronos PBR Neutral: the colours under its highlights kept), not on the
final image: what is not lit (background, sprites, edges, lines) keeps its colour. The roughness
is raised where the normal changes within a pixel (specular antialiasing). A selected part is lit
as not metallic; the lines and edges are unlit. Each light has a ratio from 0 to 1
(environmentRatio, headLightRatio, sunRatio), its intensity its ratio over the sum of the
three: the intensities sum to 1, which keeps the colours (a face lit by every light shows nearly
its own colour).Renderer draws
settings.postProcessing.antialiasing.accumulatedFrames more frames by itself (8 by default),
each from the camera moved by a fraction of a pixel (Halton 2, 3; CameraJitter, around the
draw only: the culling never sees it) and antialiased, shown averaged with the previous ones in
a float target: close to supersampling (edges, highlights, thin lines). Any frame asked for (a
change, the backend's own work) starts over; none while the animation loop runs. Same in WebGL2
and WebGPU (WebGLFrameAccumulation, WebGPUFrameAccumulation).primitive: 'lines' or
the sharp edges of a part of triangles, are drawn 1 pixel wide (GL_LINES, line-list), unless it
has a LineStyle (MaterialProps.line, MeshProxy.setLineWidth / setLineDash): then each
segment is a ribbon of 2 triangles facing the camera, width CSS pixels wide on screen at any
distance (times the renderer's pixel ratio in the drawing buffer), dashed when it has a dash and a
gap (in CSS pixels too, measured on screen from the segment's start, or from its end when the
start is behind the camera, the other end cut at the near plane), in the passes of its lines or of
its edges (layers, x-ray, selection, transparency, cutting planes; the edges in their edge colour,
hidden with the edges). The styles are stored once (LineStyleTable, 4 floats: width, dash, gap,
unused), a part keeping a 16-bit index in two free bytes of its properties. The width has no size
in the scene: the boxes stay those of the geometry (a ribbon may be cut a few pixels early at the
screen's border); a line is picked within its threshold in pixels (lineThresholdPixels) or half
its width, the larger. WebGL2: 6 vertices per segment without index (WebGLBatchWideLines), the
ends of the segment read by gl_VertexID in textures of the positions and segment indices of the
geometries drawn as ribbons only, each copied at its first draw (WebGLLineTextures: 16 bytes per
vertex, 4 per segment index). WebGPU: the culling lists a styled part in its wide-line pass
(WIDE_LINE_PASS_OFFSET, beside the line passes) or its wide-edge pass (WIDE_EDGE_PASS_OFFSET,
by edge pass), the emit writes its segment indices as for the 1 pixel segments (each range on an
even word), and the draw is not indexed, 3 vertices per segment index, the vertex shader
(WebGPUWideLineVertex) reading both ends through the emitted indices or the Container's; the
styles in a storage buffer (WebGPUInstanceBuffers.lineStyles). The segments without style keep
their path: no cost.scene.cuttingPlanesController (CuttingPlanesController, settings.scene.maxCuttingPlanes planes, 8)
cuts the scene: what lies behind one of its planes is neither drawn nor picked. A plane is a three.js Plane in scene
coordinates (those of the objects' matrices and of the BoxTree), its normal towards the side kept
(kept where normal · p + constant >= 0, as the frustum's planes). A point is cut by one plane
(union): what is kept is convex, whether one plane, several, or a section box (six planes facing
in). addPlane adds one (copied) and gives its CuttingPlaneProxy, as the meshes' and the
sprites' (its owner, setPlane to move it, visible in the helper, removePlane); clear
removes them all; as the objects, their version is read by whoever uses them, and the caller
asks for a frame. An object ignores them with
MeshProxy.ignoreCuttingPlanes (an InstanceTable flag and a leaf flag,
PART_IGNORES_CUTTING_PLANES) or SpriteProxy.ignoreCuttingPlanes (also in
InputSprite, a SpriteTable flag and the same leaf flag). A plane without a normal (zero length)
is refused, as more planes than the maximum.
ViewCuttingPlanes; WebGLCuttingPlanes / WebGPUCuttingPlanes: uniforms shared by the
meshes and the sprites, nothing compiled), the fragment shaders discard what is behind one:
faces, lines and edges (at their view position before they are moved nearer: cut where their
faces are), sprites. The meshes' discard is compiled only while the scene has planes
(CUTTING_PLANES in WebGL2, the materials made again in WebGPU, the passes kept): without, no
cost and the early depth test kept. The sprites' always is: their alpha test discards already.
The cut faces show the inside of the solids (both faces drawn): no caps.CullView,
the WebGPU cull pass), a box entirely behind one dropped (counted outside), except a node with
an object ignoring them under it; a change culls again, without occlusion until a depth drawn
with the new planes (the one held shows what they now cut).Raycaster takes the part of the ray they keep (getRayInterval, one interval:
what is kept is convex) and searches only it: the BoxTree's boxes outside not entered
(BoxTreeRayQuery, the whole ray under a node with an object ignoring them), each part's
nearest triangle between its ends (PartRaycastOptions.near / far, three-mesh-bvh's
raycastFirst): a triangle cut no longer hides the one kept behind it. A sprite is hit between
them too: on its part kept, as it is drawn.scene.cuttingPlanesHelper (CuttingPlanesHelper, in the three.js scene, shown at
first) draws each plane as the viewer: a transparent quad coloured by the main axis of its
normal (x red, y green, z blue), its border in that colour, the axis's letter in a corner;
following every change of the scene bounds, centred on them and as wide as their sphere. Not
cut; picked while shown, behind every object (see Picking). A plane is hidden alone with
proxy.visible = false, still cutting: its displayVersion changes, not its version (the cut
and the culling's depth kept). The quads are transparent and depth tested without writing
depth: an object in front hides them, while they tint an object behind. Their borders follow
the same depth behaviour (nothing is written for the occlusion or the AO).A sprite is an image facing the camera, of a size in world units, at a world position (three.js
Sprite with size attenuation), e.g. a marker or a label: scene.spriteController.addSprites takes
InputSprites and returns their SpriteProxys (with an owner, as the meshes'). The
SpriteController is their only writer, added and removed many at once, each changed through its
proxy, as the MeshController of the meshes: a sprite is an index of the SpriteTable (three
vec4: position, size, image, layer, shown, the region of its image, opacity), read in place by the
GPU by index, as the InstanceTable (a texture in WebGL2, a storage buffer in WebGPU), the ranges
changed uploaded once per frame. Its image is a layer of a TextureArray
(settings.sprites.imageSize pixels square, mipmapped): drawn once per key from the source of its
first sprite (a bitmap, an image, a canvas: the application draws icons and labels), scaled into
its layer with its aspect kept (its size in pixels kept too), filling its sprite's rectangle (a
sprite of the image's aspect shows it undistorted), shared, freed with its last sprite. The engine
holds no application logic: a click callback, a selectable flag stay in the application.
They are drawn in their MeshLayer, with the meshes of that layer: per layer with a sprite shown,
two draws of a quad instanced over the lists of that layer's shown sprites, opaque and transparent
(their indices, SpriteDrawList, as the meshes' draw lists), in a group of
spriteController.group (in the scene's three.js scene); the lists are made again when a sprite
is shown, hidden, changes layer or becomes opaque or transparent. Above the base, a layer's target
draws the meshes' group then the sprites' (LayerOverlay, not cleared between: depth-tested
together). Their images are alpha-tested. In its layer, an opaque sprite comes after the opaque
faces (it writes the depth, so the ambient occlusion and the culling see it); a sprite less opaque
than settings.meshes.opaqueAbove (SpriteProxy.opacity) is weighted blended with the
transparent parts, as they are. No CPU work per frame: a hidden sprite, or one of
another layer, costs nothing to a layer's draw; the GPU clips the sprites out of view. A sprite always seen, as the
viewer's without depth test, is a sprite in a higher layer.
A sprite's leaf in the scene's BoxTree has its box whatever the camera (its centre, plus half its
diagonal on each axis) and the flags of a part: Raycaster.intersect picks it with the parts (see
Picking), a message after its changes, as theirs.
A TextureArray holds images in the layers of a texture array, one image per layer: unlike an
atlas (images side by side in one texture), an image's mipmaps and repeat stay in its layer, its uv
span 0..1, and the array grows by layers without moving the images. Every layer has the same size:
an image of any size is drawn scaled into it, its aspect kept, centred (getRegion), its own size
kept (getImageSize). Images are shared by key, counted, freed with their last user. The layers
are 2,048 on most GPUs, 256 at least: the renderers lower the array's limit to their device's
(limitLayers). A TextureArrayTexture keeps its three.js texture uploaded, the same for both
backends. The sprites use one; the meshes' textures can use others.
The engine has no file format of its own. The application gives its readers: a FileReader
reads the files of one extension into InputMeshes and the file's offset (the translation that
centered the model around 0);
MeshLoader.addFileReader registers it. The reading always happens in workers, hidden from the
application: the reader's own module is the worker script (createWorker returns
new Worker(new URL('./MyReader.ts', import.meta.url), { type: 'module' }), and the module ends
with MyReader.serve()), so the application writes no worker script.
MeshLoader.load(sources) sends the files to the readers' workers, one file per worker (the cores
but one by default), the next file as soon as a worker is free (the others wait in the load). Each
worker sends its objects in batches (settings.loading.batchMeshes objects or sliceMs of reading
at most), each geometry once per file, with the file's offset. The MeshLoader handles each batch
as it comes: it moves its matrices by its offset minus the scene's origin (Scene.origin, the first
offset received; applyFileOffsets off, the matrices are used as given), leaves out the objects
whose matrix is not affine, and queues the others, those of all the files together: each a new
object of the scene, whatever its uuid (the duplicates are the import controller's to leave out,
e.g. an object already loaded). Before these checks, the
ObjectImportController given to load with the file (to subclass; one per call, so that files
handled differently are loaded with different ones) updates each object (updateObject, e.g. its
uuid or materials) and tells whether to keep it (shouldKeepObject, the object left out as
rejected otherwise); it is told each kept object as it is read (onObjectRead, with or without
geometry: an object without, e.g. an empty glTF node, is never added, only the application knows
it), then each object added to the scene with its proxy (onMeshAdded, in the same task: a
visibility or x-ray set there is drawn from the first frame). As soon as an object takes their new
geometries past settings.loading.targetContainerSizeMB, or the queue to groupMeshes objects, the
queue is added at once (MeshController.addMeshes, the browser drawing in between): its new
geometries into a Container of their exact size, its parts into the BoxTree as one subtree. An
object larger than a group alone is added as it comes, in a Container of its own. The received
geometry buffers are kept until copied into a Container: the objects decoded after name them by id
(InputMeshPart.geometry), so no geometry stays twice in memory. A load while one runs adds its
files to it (loads are cumulative, one at a time). MeshLoader.signals tell where the load is
(started, updated, ended), a LoadProgress: where it is from 0 to 100 (percent, each file
weighing the same, a quarter for its download and the rest for its objects read), bytes downloaded,
objects read, time, the objects left out with their reason, the files' errors (the objects they gave
are kept); load resolves with it once the load is over. MeshLoader.stop stops the load running,
the scene kept as it is: the objects waiting are dropped, their files reported as stopped.
From the repository root:
npm run dev: the viewer's demo app, on the engine.npx vitest run --project @stereograph/teiaviewer-engine: the engine's tests.npm run lint, npm run test:run: before a change is done (see the root CLAUDE.md).