The Road to Alexandria (IG: UPX)

[emperor_obewan]

Total Eureka moment about 20 minutes ago that could only be divine inspiration. I figured out where I’m being horribly inefficient in my proof of concept and I can probably gain a good 10-20 frames per second.

The problem is that I very wisely made each virtual screen have its own renderpass. That’s all very well and good, it makes sense to offload the processing of compositing sprites and other buffer twiddling onto the GPU.

But what this implies is: I have 2 x 1.0m x 0.5m screens at minimum 48 ppi each (thats 1920x960), and then a third at 1024x1024, and then when the stats window is running at 96 ppi, a final window of 3840x1920, all running their own render pass, every frame. Regardless of whether the screen had changed. As long as they have at least one “sprite” in their sprite buffer. And a titlebar or an icon to let someone move the window? That’s a sprite.

So basically, I’ve been drawing the equivalent of at least 5 full high resolution monitors, while doing lighting calculations in some cases, every fucking frame. No wonder the performance stats showed that the render time was periodically spiking ten times every second!!

Now that I know this, if I drop every second frame on those monitors, even if I just leave the texture image alone and don’t bother updating it until it comes time to render, I’ll be saving precious milliseconds. And that should let me get my app to that sweet spot of 120fps to avoid the dreaded ASW.

[emperor_obewan]

Yesterday I got my procedural floor and skybox done. Floor is a series of 6400 textured quads, each using a 1600x1600 texture with a full mip chain. I’m currently using ASTC compression on it, but I intend to change that to ETC2 after receiving some advice from Bard that I have to verify which says ASTC can give you a 2-3x additive hit on texture access times, even though it improves image quality.

Skybox is a series of 6 1024x1024 ASTC compressed images, and I’m probably going to stick to ASTC here unless I see ETC2 provide a significant boost in performance. Typical settings or factoids I have been advised by Bard regarding texture usage in games (which also apply to other VR apps):

• A typical game’s VRAM usage can go from 1.5GB (Superhot) to 2GB (Resident Evil) and above (eg Microsoft Flight SImulator) concurrently loaded on the GPU. This includes meshes, textures, descriptor sets etc.
• It is normal for a skybox to be 1K x 1K; on Quest it can go as high as 2048x2048; not recommended to push it higher (duh)
• Typical quality settings for a skybox using ASTC compression: 0.6 to 0.8, I’ve chosen 0.75 to be on the safe side. I had originally done 0.4 for my initial test, which significantly reduces quality.

I personally can’t tell the difference in quality between 0.4 and 0.75 for the skyboxes I’m using, but whateva. I still get some judder when doing spins of the scene, yet the calculations of frame rate show my app is still performing at 60-65 fps even with all the texturing. Here’s the typical performance graph (self generated in app) for a spin:

Notice the rendering spikes near the end; that was the couple of spins of the full scene I did. It’s not immediately obvious from the pic, but I did get one or two spikes where total frame time was 21ms. That’s concerning, I’ll have to see if ETC2 fixes it. I think the main power drain now that the render passes are optimized to only-when-necessary is the 6000+ poly textured floor being rendered with an ASTC texture.

I also need to fix the pointer. Marching cubes was a fun nod to the algorithm, but rendering 25 textured cubes is 25 draw calls where I could be using a single low poly cylinder. And it needs to not spaz out when there’s nothing for the ray to hit.

The image I posted in NFTW is one skybox, but I’ve programmed the ability to load multiple ones specifying by name. I probably need to code it to edit the vertex buffer so I can load a new skybox without dropping another mesh into the ECS. Maybe. It’s not exactly as though VRAM is at a premium; apparently my skybox typifies 0.6-0.8% of a typical game’s texture usage, and people seem to keep saying that texel density doesn’t really impact your FPS, so until I see otherwise I’ll be a bit more relaxed with my scrimping on memory.

Today I’m going to implement strafing and hopefully a wrist menu finally. And I need to fix that damned keyboard because I’m tired of watching the thing fall through the floor and into space.

[emperor_obewan]

The work continues. I don’t get any days off for good behavior

Today I’ve been working on my select box implementation. The first step is done - I have the function to initialize the child mesh complete, as well as a very basic draw buffer function. I implemented a ScrollBar class where the developer sets the bar_frac field of the ScrollBar and then calls the draw() function on the struct, which updates a texture buffer. Developer (that’d be me) then has to call the somewhat unsightly scrollbar.buffer.as_mut().unwrap().upload_image() to upload the changed texture for the scrollbar to the GPU, and of course call the draw_buffer function of the select box itself, which then flags the code that later uploads the window texture to the GPU. I’m divided on whether to highlight the item being moused over to increase immersiveness, the app is already pretty damned immersive as it is even though its still at the end of the proof of concept phase and morphing into being able to do useful things real soon.

Why am I implementing all this windowing system code from scratch? I could just use a shader to draw a hud or even use Hotham’s built in support for the egui windowing system. But, then I wouldn’t be able to pick windows up and travel with them, move them about, rearrange them, lean back and view them as if I’m in the chair of the Razorback from The Expanse and besides I feel like the system I am developing, with callbacks which are completely independent of the windows themselves, gives me the flexibility I need to be able to eventually write any program with minimal setup.

I’ve also been refactoring my initialization code:

Moving all of those pesky one or two liners to initialize my state variable into the implementation for the struct or mod itself. Right now my real_main() is at the somewhat manageable 170 lines of code, but I want to move at least 70 of those lines out into other modules so that startup for future programs using my windowing system isn’t so unwieldy.

I mean, my main tick function has 16 different systems, at least probably half of which I’ve written myself. And my state variable, which is now indelibly intertwined with my windowing system, has 48 fields in it. Some of which I probably need to eventually cut.

I need to store the pointer meshes and textures for the controllers, I need to cache the results of a raycast from the left and right controllers every frame, I need to store the hand trackers and hand tracking data for if hand tracking ever becomes viable, I need to store my custom render pipelines, of which there are three! I store a pointer to the text notification system which lets me draw debug text onto a HUD for myself or for users.I need to store the mesh manager and the (sprite/texture) atlas manager, the fonts which point to a wrapper around a font to draw text, the total number of ticks since the app started, the time the app started, and it just keeps multiplying!

But it is still cool when I fire up the app and it doesn’t crash or do something weird, and I’m in a 3D textured world where my Quest 2 is drawing at least 6.5 thousand polygons every frame and still most of the time getting 70 FPS! I plan on doing a new video soon and also creating a proper branded channel where I can share my dev logs and try to drum up financial support for my project. Just as soon as I can find the time to pull myself away from the emacs window, LOL

[emperor_obewan]

So I finally got my scrollable multiple select box working, with you beaut gradient filled scrollbar. Now I have to round it out before moving on to other things:

• It needs to have the scrollbar and level knob color sent in by the creator of the child mesh. Maybe make them optional, and have defaults, but meh… too much work for that.
• I need to write rounded_gradient_rectangle, a combination of gradient_rectangle and rounded rectangle, that implements drawing rounded rectangles with color gradients, so the knob can be fancy as well. Technically in my slider class, which is separate, I specified the knob could be a texture itself, but that’s kinda overkill.
• I need to fix the algorithm used for determining the top and bottom lines to show. Currently for small numbers of items, at the edges it prints from the top only the last 2 or 3 lines. I seem to recall the kivy source code had an algorithm for calculating that, but for images.

I spent the last couple of hours tracking down a very frustrating bug. I was wondering why the vertical gradient on the scroll bar kept repeating 6-7 times over, and put HUD notifications in to try to figure it out, only to discover it was the gradient function itself: I had missed replacing width by height in my code when I wrote it over a month ago, and apparently hadn’t noticed the problem because the width and height of boxes I was testing with was similar enough (and the gradient colors similar enough) for me not to notice. And the gradients were around the wrong way! c2 to c1 rather than c1 to c2. I need to be more observant when doing UI testing.

I’m still trying to hunt down why Vulkan periodically crashes my app when I wait for device idle and then try to swap out the texture for a mesh in the texture sampler in preparation for its deletion. It only happens about 50% of the time, and I don’t understand why it happens both with WriteTextureDescriptor and vkDestroyImageView. The last time I saw it happen, it was claiming the texture was still in use when it wasn’t, or shouldn’t have been. Also, I need to fix the raycast logic in the callbacks system so that meshes where the raycast is occluded don’t receive and fire a callback request themselves when a touch exists on a child mesh in front of them. All little tiny niggling problems which once I solve them will make the library that much more robust!

The idea with the select box is to next use it to implement the switching in and out of different environments, and to write a specialization of select box for file system selection. I had a bit of fun with the selection names like “Funky Banana” and “Monkey Pee, Monkey Poo!” so I didn’t go fully insane debugging the damned thing.

[emperor_obewan]

Been slowly dealing with the whole “oh shit my colliders went bye bye” situation. Moving stuff around inside the simulation, sometimes parented objects, or even non parented ones which have colliders, I find their colliders going out of whack, Assuming the object’s GlobalTransform is set correctly, that’s no problem. But the logic for making sure the GlobalTransform is correctly set, or calculating what it should be on the next frame, can be tricky.

Normally you’d just let the physics simulation take care of it. But because I’m writing a whole windowing system and the colliders are sensors only and the parented windows don’t get their colliders updated due to the logistics of the library I’m using, I find things getting out of sync if I do weird things (which I’m sure users are going to do if I accidentally let them).

At start up, Global Transforms are not set properly for parented objects, and even though I’m calculating the transformation with respect to parent, something is going wrong. That’s why I hit on a temporary solution of iterating all the semi transparent meshes in the scene to update their colliders from their global transform. But I don’t think this is a permanent solution. I need to make sure my calculation of the updated global transform is correct so I don’t have to iterate over what could be 10-20 translucent windows and update their position so the physics system can raycast to them properlyu on the next frame. Thankfully, setting or getting a transform from the ECS isn’t too expensive. But its still precious milliseconds. I don’t want to needlessly set all the transforms regardless of how simple that solution is.

My test scenario is simple: I have a horned figure which l created earlier in this thread that appropriately looks like the devil – the one @oloy thought looked like someone who had eaten to many allergenic nuts. If I drag him around somewhere (with all the parented meshes attached and moving with him), my ability to raycast to the parented objects goes bye bye and the pointer just goes straight through the window. But if I update all meshes every frame, or update the meshes based on the input context, the problem goes away.

It seems like a trivial problem. But it is remarkably hard to debug. For now I’m using a HUD to do that. I’m tempted to just put it off with the problem of texture re-use until later, because fuck it. I have more important things to worry about like using my newly minted select box implementation to update the environment map / skybox.

I currently have a month or so to get something out on the market before I start having to skip meals to get by. So I gotta get through the remaining trivial shit quickly. I already updated my buymeacoffee to reflect the new business name, and I’m tempted to do a video in the next couple of days. I’m curious if anyone can give me some pro-tips on how and where to advertise new content, or how to ensure the right people see your video. I want to get this right from the beginning.

[Risky]

Sorry if I missed it somehow but what engine and programming language are you using mainly?

[emperor_obewan]

The programming language is Rust and the game / VR development library I’m using is not one of the larger ones (Unity, Godot, bevy etc) but Hotham. It has some problems in its current incarnation (occasional problems due to the audio library it uses, among other things – check out the Github) but the guy who wrote it is a local from this country, its pretty much an indie developer who is working on his own game.

It uses OpenXR and Vulkan (via the low level ash::vk).

[Risky]

Thanks, will check it out btw are you a software developer regularly and do game dev as a hobby?

[emperor_obewan]

No, at the moment it is a full time startup from late June this year, under one of the programs in our local country to help new startups that have a solid business plan. It’s taken significantly longer than I anticipated to get to a point where I can start developing what I want to make, because I’m doing this on my own on a single Oculus Quest 2 headset and an underpowered laptop from 2020, and I had to learn all the Vulkan rendering stuff as well as a new language (Rust as well as GLSL) all from scratch. Hotham only has windows via egui as a flat overlay on top of the main scene, which wasn’t going to cut it for what I needed to do.

Also I’ve had to learn on my own how to convert textures from png/jpg into ktx2 with different compression formats, learn about texel density, mipmapping and all the various concepts needed to quickly render scenes on the GPU.

[Risky]

That’s awesome and takes dedication! Which is why this journal is highly inspiring, keep it up.

I’m at a place where I want to venture out a bit from software programming and see if game dev (client side) or ai might be my next step. That’s why I asked. I’m also a smidge interested in learning Rust. We’ll see…

[SaintSpring]

bro, I read your journal, it inspires me a lot. This is the least of what I can say about your it.

[emperor_obewan]

Thanks for the support guys! It helps when I’m tearing my hair out trying to figure out why my ray cast isn’t working (and it ends up being invisible colliders I didn’t dispose of, lol).

Here’s the next installment in showing the development badly

Hotham 3D windowing system test -- Nov 20, 2023

Google’s ScrewedTube won’t let me post links because I haven’t given it my mark, or shown it my pass, pard, so you’ll have to forgive the half links in the video description

[emperor_obewan]

Had a big win this afternoon with my app. I imported the entire Kp and Ap index data set since 1932 into my program, then took the last 13 years of data since Bartel solar rotation 2407. Then I made my program drop a mesh into the virtual world based on that data. 1.4 million vertices, 455K triangles, 17.28m per side of 8cm rectangular cuboids with height keyed to the Kp index.

Then I navigated through it to see how my shader handled it. I was getting around 50 fps at least which was shocking for so many vertices. It was an entire city of data and I was strafing right through the middle of it. Minimal moire patterns,fairly responsive, and if I am thinking right, I can split the primitives into 3 or 4 separate meshes without any major loss of speed. So I should be able to color the Kp 0-3 green, Kp 0-6 yellow, Kp 6-9 red as it customary, and then see a heat map of when in each solar rotation the peaks occurred.

This is the proof of concept taken further, and if this works tomorrow I will start fine tuning it for other data sets so I can finally start doing cool shit like 3 dimensional histograms or bezier curves through the points. This was such a big win I stopped work before 8pm, which is unusual for me. But being able to do this so successfully was unexpected, I was fully expecting the program to crash as soon as I pressed the button. Stoked that this works!

[emperor_obewan]

I had a bit of a break from coding today. Even though I’ve been running the subs consistently, I just didn’t have the stomach today to work on the code. I did a little on fixing things mipmap wise and then just gave up. I had a bit of a discussion with an old friend I knew 20+ years ago and that was cool but ultimately that was cut short.

My birthday is coming up in a few days and that has been depressing me a little. I’ll be mid forties and just feeling a bit depressed that I don’t have my code done yet. But we’ll see where things go.

[Niten_Ichi-ryu]

@emperor_obewan

My birthday is coming up in a few days and that has been depressing me a little. I’ll be mid forties and just feeling a bit depressed that I don’t have my code done yet. But we’ll see where things go.

You are greater than you think! Don’t be too hard on yourself. Just keep at it. You’re alive and kicking bro! That in itself is a gift.

[emperor_obewan]

Today was supposed to be a day off, but after I got my obligations outside of the house dealt with I ended up writing more code anyway.

One thing I’m going to need to do in writing my program with a limited sized vertex/position/index buffer, is write code to manipulate the buffers in order to re-use the slots I’m done with. That is an interesting task, thanks to the way the engine is designed. Every Mesh has an arena_id::Id, and as it turns out the library has no way to delete such an id. So the meshes remain in the arena.

However, a Mesh is composed of a vector of primitives, and each primitive then stores its index buffer offset, the number of indices, the vertex buffer offset… but not the vertex count! So I have to store the vertex count for these special created meshes that I’ll be re-using in the hecs::World as a component, and then when it comes time to destroy the existing graph and create a new one, I copy all primitives backwards by the combined number of slots for the primitives I’m deleting, decrease the recorded buffer length, and then update those primitives with new vertex and index buffer offsets for each associated mesh.

Similarly, floating text drawn in the air or on the ground is composed of a series of quad meshes. Rather than delete them (because I can’t ), the idea is going to be to mark each quad mesh that I’m drawing with an indicator of its orientation (xy, xz or yz) and make each mesh I’m done with invisible, then push its particulars into a buffer. Then when drawing text in mid air not connected to an existing texture buffer, I make a dedicated function to re-use any meshes that are marked as safe to re-use.

With these things in mind, the only other thing I need to do is be careful about the ordering I create my meshes in, so that there is a minimum number of memory copies to replace or re-use stuff. I’m thinking I reorganize my code to move the creation of the axes and other shit to the start of the graphing logic, so that the gigantoid meshes for the bars or other plotting primitives are at the end of the vertex/index buffers, removing the need for potentially several overlapping memory copys.

Its a complicated ass way to go about things, but at least it keeps things efficient. Who really wants to keep adding millions of points to a vertex buffer? Besides, in the engine, the default maximum vertex buffer length is 2 million, which means I’d be really pushing it to draw two of these graphs (unless I edit the engine source itself and update those numbers… and maybe update the index buffer to have a larger size than the vertex buffer while I’m at it!)

Sub updates

I had to go walkabout today to get a document signed and witnessed. That involved me walking to three or four different destinations. While I was walking, I noticed the same interesting energetic phenomenon I’ve noticed the last couple of times I’m out in public. I’m not sure if its the Hero strength aspects, the new Emperor resilience, or both, but there’s been a strong sense of solidity and noticing my muscles. Using them more. Also a different level of interaction with people.

After the walk, I went out again to see a friend. My friend had some associates with him who were not of the same caliber as him; one of them was clearly on something. They started bad mouthing him for trying to do something with his life (while the two of us were trying to have an ordinary conversation about running our own businesses), and the situation was going downhill quickly. My friend had control of the situation, but intuitively I knew when was the right moment to leave. My posture during the interaction remained relaxed and strong too. Hero and Emperor haven’t changed my predisposition towards being a non-violent individual, but it has made me calmer in troubling situations.

Also I think I’m noticing the resilience/ability to push through especially through physical pain/muscular tension more.

[emperor_obewan]

I got the re-use of quad meshes working and started refactoring the code to be more abstract. Currently each graph is drawn with the same sized bar each time (a cuboid of 8cm on a side with y axis scaled by a factor dependent on what’s being graphed). The refactor is so the minimum and maximum values on each axis can eventually be changed, as well as the scale of the graph.

So far, my axis is already getting kinda complex:

struct Axis {
    pub axis_title: String,
    pub direction: Vec3,
    pub number_of_ticks: u32,
    pub plot_instance: u32,
    pub min_value: f32,
    pub max_value: f32,
    pub tick_text_factor: f32,
    pub scale_factor: f32,
    pub tick_size: f32,
    pub texture_string: String,
}

Ie, an axis can be textured with an arbitrary ktx2 texture, it has a minimum and maximum value, each tick is tick_size apart, and is scaled down to a human viewable size of scale_factor (compared to 1.0 == 1 metre), and the text for the ticks has a factor by which it varies from a 72 point font which determines the size of the character meshes drawn on the axis. Each axis has a title which will be drawn on the ground underneath the tick values. And of course the direction is the direction in which that axis increases… for Z, that’s (0,0,-1).

That’s just what I’ve thought of so far. Each time I draw an axis, I need to check whether I’ve already created one, and if I have I need to manipulate the vertex points of the cylinder based on knowledge that the number of segments in the cylinder isn’t going to change.

There’s all sorts of little complexities like this that I need to figure out in preparation for making the incoming data set arbitrary. I will eventually need to change the plotting algorithm too so the meshes can be created as a series of cuboids, a smooth surface, or for the slightly more crazy-making, some arbitrary mesh instance (like a sphere). Of course, the more complex the marker is at a point, the more vertices need to be drawn and the slower things get.

I’ve been tempted today by the desire to just say f’ it give the code a miss for the rest of the day, and up till now I’ve rejected that desire. Its late afternoon here though, I may just pack it in in the evening (and then wake up at 3am to start this whole crazy all over again)

[emperor_obewan]

I’ve been working on the conversion of the axis drawing code all day. The aggravating problem I have had has been to do with text alignment. When drawing text normally, the characters turn out as expected: I advance by the horizontal advance each time, I ignore the horizontal side bearing which is miniscule anyway for the text I’m drawing, I respect the pixel bounding box of each char in deciding whether its run off the screen, and all is good. However, when I’m generating text character by character, to create partially transparent meshes of each character to save texture space, characters like l and t are getting misaligned. Neither the l nor the t have any horizontal side bearing component, which I would have expected. Now I’m wondering whether the misalignment is due to the x and y values being returned by the draw call, or something else entirely.

Aside from that, the refactoring has improved things somewhat. My plot dataset function now extents from line 289 to 636, while my draw axis function goes from 146 to 287. 140 lines for draw axis, almost 350 for the plot function. Over 100 lines of that is for differentiating between the 5 different types of data you can plot and adjusting the scale factor, and recording maximum and minimum values and their location in space.

I say I just ignore the text alignment problems for now, annoying as they are, and focus on making the data input and configuration of the graph a bit more flexible. That probably means me pulling another all nighter ugh, whoever said writing a data analysis app like this would be easy? Still, the more I get done before December starts, the better off I’ll be.

[emperor_obewan]

Solved a couple of problems today. Things are slowly coming along with the CSV/other database import code.

• Implemented the file select box. Still need to add .. folder to the top of the selections since scan_dir doesn’t pick it up, but no biggy. Selecting a file is working.
• In the meantime had to more bullet proof my mesh destruction code to set a flag on the mesh to indicate if it’s dead. If its dead, the callbacks system is instructed to drop any further callbacks for the dead mesh on the floor. This was brought on by some kind of weird stack overflow error upon return from the mesh destruction code. Apparently the callback system was still happily iterating through the callbacks despite the mesh being dead and buried.
• Also had to fix a bug in my code to flow text. Evidently, I hadn’t considered the case for flowing a single line of text, which causes my return buffer of start and end characters to be zero length. Ouch.
• Wrote a basic draw buffer function, found some nice blue tones to file out the import window in, added the button sprite to the window to select the CSV file, and figured out an efficient layout for the boolean and other simple aspects of the import form.

And that’s where I’ve finished the day. I’d like to keep coding since I never did get that all nighter in last night, but right now KB has me buzzing like a bee and can’t sit still. The heat in the body is kind off obscene, so I’m thinking some time outside to cool off is a good idea.

EDIT: Okay, 1.40am here now. I finally got the checkbox code written and added each checkbox as a sprite with a callback function and notification function to my import window, fixed a buttload of errors and got the damned thing to compile. Only now I don’t really feel like testing the thing this late … honestly, at this hour I’d rather smoke a fatty and just go to bed. But… the pull of the code… it burns! LOL.

Tomorrow the fun and mayhem I have to deal with is going to be naming fields on CSV files which have no column names (yeah, that’s a thing) and being able to save and load settings for importing a particular file or type of file on the headset’s volatile storage. I hope to be able to import an arbitrary data-set by the end of tomorrow. SQL and formats like HDF5 can come later.

[emperor_obewan]

Wow. I thought I had hit a pretty intractable block yesterday, but I seem to have found a way around it.

After solving the problem with dead meshes going zombie again (turned out after the callback function closed the mesh, the parent code was calling draw_buffer after that, before I even returned to the callbacks_system), I ran into a problem with polars.

Apparently polars 0.34 and 0.35 began using functionality which is not stable in Rust 1.67.1. The Github issues page had me thinking 1.65 or 1.66 was the MSRV, but in fact it had been updated to 1.71 just over a month ago in October. It wasn’t obvious though when compiling originally what version of Rust was necessary, as it was basically just a footnote inside the README.md which was not prominently displayed or easy to find.

I originally thought cargo msrv would give me a correct answer on dependencies, foolish me! I wish it were that simple. I ended up having to track back through Blame and File History on Github to identify when documentation of the MSRV in their tiny little footnote had changed (which is not the same as when it changed in the code), and then figure out by implication a suitably recent Semver to test.

So a solution to this problem was quicker than I thought (I figured I’d be roadblocked for at least two days while I figured this out). This is a plus as it means I can get on with updating my draw_buffer function to reflect any problems importing a file, as well as updating the content of the first ten lines of data in the file to the panel.