Lessons Learned: When the Drawable Leaks Into Your Render Pipeline

This week, while rendering scenes in Vision Pro using the Untold Engine, I realized that scenes were being rendered with the incorrect color space. Well, initially, I thought it was a color space issue — but something was telling me that this was more than just a color space problem.

After analyzing my render graph and verifying the color targets I was using in the lighting pass and tone mapping pass, I realized that I had made a crucial mistake in the engine.

See, my lighting pass was doing all calculations in linear space, which is correct. However, the internal render targets were being created using the drawable's pixel format. Doing so meant that every platform could change the precision, dynamic range, and even encoding behavior of my internal buffers.

In other words, my lighting results were being stored in formats dictated by the drawable’s target format. That is wrong. The renderer should own its internal formats — not the presentation layer.

Because the drawable format differs per platform (for example, .bgra8Unorm_srgb on Vision Pro), my internal render targets were sometimes:

  • 8-bit
  • sRGB-encoded
  • Not HDR-capable

Even though my lighting calculations were done in linear space, the storage format altered how those results were preserved and interpreted.

So yes — the math was linear, but the buffers holding the results were not consistent across platforms.

That is where the mismatch came from.

To fix this, I explicitly set the color target used in the lighting pass to rgba16Float. By doing this, I ensured:

  • Stable precision
  • HDR-capable storage
  • Linear behavior
  • Platform-independent results

Now, my lighting calculations are identical regardless of the platform, because the internal render targets are explicitly defined by the engine — not by the drawable.

The Second Issue: Tone Mapping Is Not Output Encoding

The other issue was more subtle and made me realize that I still have a lot more to learn about tone mapping.

My pipeline originally followed this path:

  • Lighting Pass
  • Post Processing
  • Tone Mapping
  • Write to Drawable

The problem with this flow was that I assumed that after tone mapping, the image was ready for the screen.

But that is not true.

Different platforms expect different things:

  • Different pixel formats (RGBA vs BGRA)
  • Different encoding (linear vs sRGB)
  • Different gamuts (sRGB vs Display-P3)
  • Different dynamic range behavior (SDR vs EDR)

My pipeline above implicitly assumed that the tone-mapped result already matched whatever the drawable expected.

But tone mapping does not mean “ready for any screen.”

Tone mapping only compresses HDR → display-referred brightness range. It does not:

  • Encode to sRGB automatically
  • Convert color gamut
  • Match the drawable’s storage format
  • Handle EDR behavior

So when I wrote directly to the drawable after tone mapping, I was essentially letting the platform decide how the final color should be interpreted.

And since platforms differ, my final image differed.

What Was I Missing?

I needed to separate responsibilities more clearly.

I needed a pass that owned the creative look — fully defined and controlled by the engine:

  • Exposure
  • White balance
  • Contrast
  • Tone mapping curve

This defines how the image should look artistically.

And I needed a separate pass that is platform-aware — an Output Transform pass — that defines how the display expects pixels to be formatted:

  • Encode to sRGB or not
  • Convert to P3 or not
  • Clamp or preserve HDR
  • BGRA vs RGBA channel order
  • EDR behavior

In my original pipeline, I had collapsed Look + Output Transform into one step. I wasn’t explicitly controlling the final encoding, so the platform’s defaults influenced the final image.

With the extra passes and modifications I made, the Look pass now defines the artistic look of the image. The Output Transform defines how that look is encoded for a specific display.

Previously, I was conflating the two — which allowed the platform’s drawable format to influence the final result.

Here is the image after the fix.

After fix image

Now, the renderer owns the working color space and internal formats, and the drawable only affects the final presentation step.

Thanks for reading.

Lessons Learned: Vision Pro, Large Scenes, and Threading

This week I came across an unexpected issue. Loading a large scene on the Vision Pro would result in a run-time error. But loading the same scene on macOS did not. Both macOS and visionOS use essentially the same loading and rendering path. So what could be causing the issue?

To make things worse, the run-time errors were cryptic and pointed to different functions each time the application crashed. Sometimes it looked like a type issue. Other times it looked like a Foundation error. Nothing clearly indicated what the real problem was.

However, during debugging, I started to see a pattern. Most of the time, the error pointed to scene or component data access. That’s when I began wondering:

What if, during loading, the system is trying to access data that is not yet stable?

In other words, what if one part of the engine is writing to components while another part is reading from them?

Then another thought came to mind. What if this issue is also present on macOS, but because macOS does not use a dedicated render thread in the same way as Vision Pro, the race condition is simply not exposed?

After a few more debugging sessions, I realized I may have been onto something.

The Real Issue

On Vision Pro, rendering runs on a dedicated render thread. When we load a large scene, the Untold Engine performs loading work on a separate thread so that we don’t block execution.

That means we had two things happening at the same time:

  • The render thread traversing the scene graph, iterating component storage, performing culling, and building draw calls.
  • The loading thread creating entities, attaching components, recursively tagging entities for static batching, rebuilding batch data, and updating spatial structures such as the octree.

In other words, the render thread was reading from scene/component data while the loading thread was writing to that same data.

This read/write overlap caused race conditions and eventually corrupted state.

Why This Did Not Happen on macOS

The reason this did not happen on macOS is mostly due to timing and threading differences.

On macOS:

  • The renderer and update loop are more tightly coupled.
  • The mutation window during loading is smaller.
  • The render traversal is less likely to intersect with scene mutation at the exact wrong moment.

On Vision Pro:

  • Rendering runs independently on a dedicated thread.
  • Frame submission follows its own cadence.
  • The renderer can traverse the scene while it is still being mutated.

Large scenes amplify this issue because static batching and recursive hierarchy processing take longer, increasing the window where the world is in a partially mutated state.

The Solution

The solution was to add a gating mechanism to prevent any read/write collision while loading was taking place.

The idea is simple:

  • When a major scene mutation phase begins (for example, during large scene loading or static batch generation), increment a shared counter.
  • When the mutation phase finishes, decrement it.
  • The render thread checks this counter before traversing the scene.
  • If a mutation is in progress, the render thread continues to submit frames but avoids traversing scene or component data that may still be unstable.

It’s important to note that I do not block the render thread on visionOS. I let it continue running, but I prevent it from accessing critical scene data while the loading phase is still mutating that data.

After this fix was in place, loading large scenes with the Untold Engine on Vision Pro no longer caused run-time crashes.

Final Thoughts

In the end, the issue was about concurrency.

  • Rendering reads from the world.
  • Loading mutates the world.

Without proper synchronization, those two operations cannot safely overlap.

Vision Pro didn’t introduce a new bug into the Untold Engine. It exposed a hidden assumption in my threading model.

And that’s a good thing. Thanks for reading.

Lessons Learned While Adding Geometry Streaming

This week I worked on adding Geometry Streaming to the engine and fixed a flickering issue that had been quietly annoying me for a while.

Both tasks ended up being more related than I initially expected.

BTW, here is version 0.10.0 of the engine with the Geometry Streaming Support.

Geometry Streaming Wasn’t the Hard Part — Integration Was

Getting Geometry Streaming working on its own wasn’t too bad. The goal was simple enough: render large scenes without having to load the entire scene into VRAM during initialization. Instead, meshes should be loaded and unloaded on demand, without stalling rendering.

The part that caused friction was not streaming itself, but getting it to behave correctly alongside two existing systems:

  • the LOD system
  • the static batching system

Each of these systems already worked well in isolation. The instability showed up once they had to coexist.

I initially overcomplicated the problem, mostly because I was treating these systems as if they were peers operating at the same level. They’re not.

The Assumption That Broke Everything

The thing that finally made it click was realizing that these systems don’t negotiate with each other — they react to upstream state.

Once I stopped thinking of them as equals and instead thought of them as layers in a pipeline, the engine immediately became more predictable.

A stable frame ended up looking like this:

  • Geometry streaming updates asset residency
  • LOD selection picks the best available representation
  • Static batching groups the selected meshes
  • The renderer submits batches to the GPU

Once I enforced this flow in the update loop, a surprising number of bugs simply disappeared.

The key insight here was that ordering matters more than clever logic.
These systems don’t need to know about each other — they just need to run in the right sequence and respond to state changes upstream.

The Kind of Bugs That Only Show Up Once Things Are “Mostly Working”

Getting the ordering right was half the battle. The other half was dealing with the kind of bugs that only appear once the architecture is almost correct.

For example:

  • I wasn’t clearing the octree properly, which caused the engine to look for entities that no longer existed.
  • One particularly frustrating bug refused to render a specific LOD whenever two or more entities were visible at the same time.

That second one took an entire day to track down.

It turned out the space uniform was getting overwritten during the unload/load phase of the streaming system. Nothing fancy — just a subtle overwrite happening at exactly the wrong time.

That kind of bug is annoying, but it’s also a signal that the system boundaries are finally being exercised in realistic ways.

The Flickering Issue That Didn’t Behave Like a Flicker

The flickering issue was a different kind of problem.

It only showed up in Edit mode, not reliably in Game mode. And it wasn’t the usual continuous flicker you expect when something is wrong. Instead, it would flicker once, stabilize, then flicker again a few seconds later — or sometimes not at all during a debug session.

That made it especially hard to reason about.

At first, I assumed it was a synchronization issue between render passes. I tried adding fences, forcing stricter ordering — none of that helped.

The clue ended up being that the flicker correlated with moments when nothing should have been changing visually.

The Real Cause: State Falling Out of Sync

Eventually, I traced the issue back to the culling system.

In some frames, the culling pass was returning zero visible entities — not because nothing was visible, but because the visibleEntityIds buffer was getting overwritten.

The fix wasn’t to add more synchronization, but to acknowledge reality: the culling system was already using triple buffering, and visibleEntityIds needed to follow the same pattern.

Once I made visibleEntityIds triple-buffered as well, the flickering disappeared completely.

The takeaway here wasn’t “use triple buffering,” but:

Any system that consumes frame-dependent data must respect the buffering strategy of the system producing it.

Final Thoughts

None of the issues this week were caused by exotic bugs or broken math. They all came from small assumptions about ordering, ownership, and state lifetime.

Once those assumptions were corrected, the engine became noticeably more stable — not just faster, but easier to reason about.

That’s usually a good sign that the architecture is moving in the right direction.

Thanks for reading.

Untold Engine Updates: LOD, Static Batching and More !!!

Hey guys,

It’s me again with a new update on the Untold Engine — this time focused on user experience and performance.

You might find this a bit odd coming from an engineer, but user experience matters a lot to me. Sometimes, I even see it as more important than performance itself. I know, that sounds backwards. But honestly, I don’t care how fast a tool is if the user experience is bad. If it’s frustrating to use, then to me, it’s not a good product.

So let’s start with the user-experience improvements I’ve been working on.

BTW, you can read more about version 0.9.0 of the engine here.

Quick USDZ Preview

I was never happy with the fact that every time I wanted to render a model with the Untold Engine, I had to create a full project first. That felt unnecessary and slow.

So I added a Quick Preview feature.

You can now preview a .usdz file directly without creating or importing it into a project. Just click the Quick Preview button, select your .usdz file, and you’re good to go.

Improved Importing Workflow

Next up: importing.

The old importing workflow was confusing at times and a bit error-prone. It was too easy to accidentally import a model into the wrong category, which is never a good experience.

Now, when you click Import, you’re explicitly asked what you want to import. This makes the process clearer and significantly reduces the chances of loading assets into the wrong place.

Scenegraph Parenting Support

At some point, I realized I really wanted to create parent–child relationships between entities directly from the editor — but the Scenegraph didn’t support that at all.

So I added it.

You can now parent entities directly in the Scenegraph by dragging one entity onto another.
To unparent an entity, just right-click it in the Scenegraph and select Unparent.

That said, I think I can make the hierarchy more visually obvious. That might be the next thing I tackle so parent–child relationships are easier to spot at a glance.

Viewport Child Selection

This one was a complete oversight on my end.

If an entity had multiple child meshes and you tried to select one of those meshes in the viewport using a right-click, the parent entity would get selected instead. That’s… not great.

This was a terrible user experience, so I made it a priority to fix.

You can now select child entities directly in the viewport using Shift + Right Mouse Click, which makes working with hierarchical scenes much more intuitive.

Now, let’s talk about performance improvements.

LOD System

The Untold Engine now supports an LOD system.

You can assign an LOD Component to an entity, provide multiple versions of a mesh, and the engine will automatically select the appropriate LOD based on distance. This is especially useful when you want to maintain a steady 60 FPS without rendering fully detailed meshes when they’re not needed.

Static Batching System

The engine now also supports Static Batching.

This is extremely useful for scenes with a large number of static objects. By batching these meshes together, the engine can significantly reduce the number of draw calls it needs to execute.

In one test scene, draw calls dropped from over 2,000 to just 34. That’s a massive improvement and makes a huge difference in frame stability.

That’s all for now.

If you want to follow the development of the engine and stay up to date with future updates, make sure to check out the project on GitHub.

Thanks for reading.

Untold Engine Updates: Multi-Platform support and Camera Behaviors

Hi guys,

Me again with some updates on the status of the Untold Engine. As always, I’ve been working diligently on the engine. Over the past two weeks, I focused on implementing multi-platform support, fixing several async issues with scene loading, and starting work on a couple of camera behaviors that I needed for benchmarking and game testing. So let me walk you through what’s new Untold Engine Studio

Multi-Platform Xcode Project Support

With the Untold Engine, you’re no longer limited to building a game for a single platform. You can now create an Xcode project with multi-platform support, either through the CLI or directly from the Untold Engine Studio.

This makes game development a lot smoother: you code once, and your game runs on macOS, iOS, iOS + AR, and Vision Pro. The only platform still missing is tvOS, but that will be added soon.

Fixed Async Issues

There was an issue when loading large scenes in async mode that could cause a runtime crash. This was tricky to debug, but I eventually tracked it down, and the crash is no longer present.

If you’ve been following the progress of the engine, you’ll know that one of my core rules is that the engine should never crash. If you do encounter any crashes while loading a scene, please let me know by opening a GitHub issue.

Camera Behaviors

Finally, I started implementing the following camera behaviors:

  • Camera path follow using waypoints
  • Camera follow with a dead zone

It’s interesting that I still haven’t implemented first-person or third-person camera behaviors, yet I decided to work on these first. The reason is simple: I’ve been benchmarking the engine, and I needed a camera that could follow a predefined path using waypoints. The goal is to measure frame rate as the camera traverses a heavy scene. Since this behavior didn’t exist, I implemented it and made it part of the engine, as I do see it being useful for game development in general.

The second behavior is one you often see in soccer (fútbol) games. The camera only follows the entity if it moves beyond the boundaries of a box; otherwise, the camera remains stationary. I implemented this because I’m currently developing a soccer video game using the new Untold Engine and Editor.

There’s a clear purpose behind developing this game, and the effort has already paid dividends. I’ve run into several bugs and API confusions that I would not have discovered otherwise. Even though I’m not in the business of making games, I do see game development as a core part of building a game engine. I mean, how else would I really test my own tools?

That’s it for now. I’m going to get back to coding. Stay tuned for upcoming features.

Thanks for reading. And don't forget to download the Untold Engine Studio