GMM Dec/23 part 1 - Saving images from interactive toyBrot

15/Dec/2023

For my Godot Mini last month, I decided to make an interactive version of toyBrot and put it on the web using Godot. It was really fun and worked quite well, even with the limited time I allowed myself. But there’s only so much you can do in two days and some of the basic functionality of toyBrot was still missing

Well, it just so happens that with Foreteller taking so much of November, I ended up doing the mini right at the end of the month and… oh look, December just rolled over! If that isn’t the perfect excuse to go back and add those missing features, I guess we’re never finding one

Even though this wasn’t THAT much work, though, it does involve a lot of moving parts and talking about them ended up taking a lot of time. So, in order to not just kind of rush through and make this a post that boils down to “X, Y and Z exist, kthxbai, like, share and subscribe” I’ve decided to split this report in two. I need to solve two different problems and have a solution which works for two different environments, might as well have two posts

So here’s the plan:

This post will talk about me implementing saving toyBrot’s output as an image file and implementing the file access support for desktop platforms

Next week’s post will talk about implementing a different feature where the current state of toyBrot can be saved and loaded and what needs to change for all this file access nonsense to work in the web, which presents additional challenges

Now that we’re all on the same page, let’s get nerding!

A mini what now?

As part of my ongoing studies with Godot and in the interest of keeping contact with the engine and avoid falling too far apart as I juggle all of my other projects, I’ve decided to try and make sure I do one “mini project” using Godot each month.

The rules are:

It HAS to use Godot in some way
I HAVE to finish it in one or two days
I NEED to have something to show for it at the end
It SHOULD give me the excuse to try something new with the engine, as a treat

So while this may rule out bigger projects, the goal here is make sure I’m active, even if most of my attention is trapped in a different thing. Managing the scope for these is a good exercise in itself and finding new bits of the engine to poke is always a satisfying experience

As for what each mini project actually IS, whatever strikes my fancy at the time will do. Though this means I’m likely to try and look for things that will build towards larger projects, it could just be a free-form minigame, or demo or what have you. Fun is definitely a goal here

What is the nature of a toyBrot?

If you’re a first timer, toyBrot is a long running project that revolves around generating a fractal through raymarching and studying and comparing different parallelisation options. It has its own running series in the blog. If you want to know more about the specifics of that, you can check this post about the conversion to raymarching that also compares many different implementations

So, a full build of the regular toyBrot project has a LOT of dependencies but, apart from, say, a C++17 capable compiler and CMake if you don’t want to rewrite the build, I guess, ALL OF THEM are optional. If CMake finds CUDA in your machine, it builds CUDA. If it finds SYCL, it builds SYCL, if it finds ISPC, it builds for them juicy SIMD extensions. But even if toyBrot finds NOTHING; if you have CMake a C++ compiler and nothing else, it builds for the C++ std::threads and std::async implementations, which you can then run and it’ll tell you how long they took. It does the work, trust me

But without verifying that work, you may, say, build with AMD’s AOCC and not know why it is that you are getting absolutely unreal results. And toyBrot doesn’t really have any sort of automated checks implemented for it, but it DOES offer two ways for manual confirmation.

The first is that, if it finds SDL2 in your machine, then every implementation it happens to build will have an optional graphical output window that will display the generated fractal. If you’re running on a graphical environment you can just look at the image and see if it’s what you’d expect (which is bad news for those unreal numbers AOCC puts out and I THINK intel icc as well. Defaults for those compilers are a tad too aggressive for toyBrot)

This doesn’t work if you’re batching your work or if you need to, say, run on a remote cluster while you figure out how it reacts to, say, AVX2 vs SSE4 for your ISPC output. So in comes libPNG. Just like SDL, if this is found, it’s added as an additional dependency for ALL of the generated targets and, then, adds the option to saving their output to a png image file. Personally I also use this for saving the images to use as desktop wallpapers, the background of this blog, etc…

# Every non-web toyBrot project has this on its CMakeLists.txt

if(TB_SDL_FOUND)
    target_link_libraries(${PROJECT_NAME} PRIVATE SDL2::SDL2)
    target_compile_definitions(${PROJECT_NAME} PRIVATE "TOYBROT_ENABLE_GUI")
    target_sources(${PROJECT_NAME} PRIVATE
                               "${TB_COMMON_SRC_DIR}/FracGenWindow.cpp"
                               "${TB_COMMON_SRC_DIR}/FracGenWindow.hpp"       )

endif()

if(TB_PNG_FOUND)
    target_link_libraries(${PROJECT_NAME} PRIVATE PNG::PNG)
    target_compile_definitions(${PROJECT_NAME} PRIVATE "TOYBROT_ENABLE_PNG")
    target_sources(${PROJECT_NAME} PRIVATE
                               "${TB_COMMON_SRC_DIR}/pngWriter.cpp"
                               "${TB_COMMON_SRC_DIR}/pngWriter.hpp"     )

endif()

The interactive implementation of toyBrot using Godot and fragment shaders already has the GUI built-in, that project was all about GUI even, but it doesn’t do any saving, it’s missing this one basic feature. Well, can’t have that now, can we?

One problem, two solutions

So, from my previous experience with deploying regular toyBrot to the web with emscripten I already knew that I was going to need separate solutions for web and desktop. There was a chance that there would be some hooking up underneath so that the desktop solution COULD work in the web mostly automagically, but I wasn’t holding up hope. It’s a tricky problem with some fundamental constraints. So I was, from the get go, assuming this would need to branch out for each side separately (an assumption which turned out to be correct).

Whenever you have more than one problem to deal with, it’s tricky to decide whether you should start by the easy or the hard one. Starting from the easy one can give you a leg up in that at least you have something going but it can, then, be disheartening to, say… spend two weeks refactoring Cartomancer because of Unicode woes while nothing visibly new gets implemented. But front-loading that hard part, while facilitating the ending later on can mean you’re immediately drained of the enthusiasm for the entire solution

In this case, I ended up opting for the easy one first. Partly because it was much easier to test but also because, knowing there was more to come, I could already start laying as much of the groundwork and isolate the web nonsense as much as possible. This meant that, for my initial task, I had two steps:

Capture an image of the fractal
Open up a dialog to save it

Godot has these parts more or less sorted out for you, so there’s a lot already there so we can build on. First, there is a FileDialog node which does what it says on the tin. It opens a Dialog that lets you choose a file or folder and then you can read from and/or save to it.

Just pop in a cute little window

The only problem I found here was that the documentation of the FileDialog itself seemed a bit lacking in how you actually integrate it. Surprisingly enough, it was Zenva who came to the rescue again. Not through an actual video course but through a written tutorial I ran into. Between what was there and in the docs themselves I managed to get things working fairly straightforwardly.

The interesting thing here was that, due to knowing this wasn’t going to be used for web, I already could make some decisions regarding the implementation. The FileDialog itself is created and configured through script, not really being part of the scene file itself. With it being a simple element to configure, this wasn’t a big deal. This also meant that every function which interacted with the file system was getting two execution paths depending on whether the program was running in the web

func _ready():
    #Other init
    if (OS.get_name() == "Web"):
        pass
    else:
        file_dialog = FileDialog.new()
        add_child(file_dialog)
        file_dialog.dialog_hide_on_ok = true
        file_dialog.set_access(FileDialog.ACCESS_FILESYSTEM)
        file_dialog.size = Vector2(500, 400)
        file_dialog.connect("file_selected", _on_file_selected)
        file_dialog.set_initial_position(Window.WINDOW_INITIAL_POSITION_CENTER_MAIN_WINDOW_SCREEN)

You also need to be careful to always set the dialog properly before bringing it up.

What’s the initial folder, what are you looking for, what will you do with it… Because the dialog itself is a constant element that you just show and hide. You CAN instead create and destroy it if you want, I guess, but I’m not sure that would be better from a performance point of view, and could end up adding some additional hassle with connecting signals and whatnot. Finally, you could just have different dialogs for each thing but THAT to me feels like some easy to avoid waste

func _save_image():
    if(OS.get_name() == "Web"):
        pass
    else:
        file_dialog.set_filters(["*.png; PNG images"])
        file_dialog.set_file_mode(FileDialog.FILE_MODE_SAVE_FILE)
        file_dialog.set_current_dir(OS.get_system_dir(OS.SYSTEM_DIR_PICTURES))
        dialog_is_img = true
        file_dialog.popup()

Showing off your best side

The second part of that problem, and one that will be reused in the web is actually getting an image to save. Godot has some really handy built-in functionality for this sort of “screenshot” feature. Everything in Godot is drawn to a Viewport. Normally this will be the draw space for your main window and you can just ask for the texture data for whatever that viewport is drawing, from which you can then get an Image object, if you ask nicely

Once you have that Image, it has a function that is literally save_png(path). It doesn’t really get much more straightforward than that. All you need is to ask that Dialog for what that path should be and you’re basically sorted!

But for me this presented two problems I wanted to overcome. The first is that just “taking a screenshot” means that your UI is all there, which is not good if you want to save this as a cool image all on its own and aren’t just demoing a feature. The second is that one of the cool things that you can do in regular toyBrot is, once you find a view you like, ask it to render a high resolution version of that image to save. So I also wanted to have a high res option. Both of these problems were pointing toward the same solution: I needed a separate viewport

A window into a world of adventure

One of the many types of Containers in Godot is a SubViewportContainer. The way viewports work, from a distance is quite simple. Viewports are the actual surfaces where your application gets drawn. If you just start placing items in your scene, they draw to your main viewport, because it is the only one there is, this one is always implicit. But what actually happens is your nodes start searching up the Node Tree looking for the first viewport parent they find and then draw to THAT

So, that Container can hold one SubViewport and allow you to position and size it according to container rules, with layouts and anchors and whatnot. This container is only supposed to have a single child, one SubViewPort which then gets drawn on where the container is. It’s like any other UI element, but instead of “draw this picture of a button here”, you’re defining a texture and, say, your Camera3D will draw what it sees to THAT

This enables you to easily mix and match 2D and 3D and manage your own rendering layers. Interactive toyBrot is a 2D scene, but the window draws a viewport that contains the 3D scene with the camera and the raymarched volume and, instead of taking an image from the main viewport, the one that has everything, I can instead ask for an image of THIS one. I don’t even need to “hide the UI” or anything like that

Additionally, this means that the viewport I’m drawing in is no longer tied to the size of the actual application window. So I can, say, resize it to some arbitrarily large size before taking my image and then just shrink it back to match later. This trick allows me to take a screenshot from a frame rendered at a much bigger size without needing the application to rendering in 4K all the time or anything of the sort

There are mostly two things to watch out for here. The first one is that it’s not enough to just “set resolution, take screenshot, reset resolution”. After increasing the size of your viewport you need to explicitly wait until it renders a new frame so you can capture it. This one thing will cause the larger image to flash for a moment on the screen. I COULD work around this by hiding this process and, in a more polished application this would be a nice touch. But in yet another rare show of restraint, I deemed this work to be out of scope. It’s no biggie and nothing breaks because of it

This back and forth does also mean I need to take care to match the size of the SubViewport to that of the main one manually, especially since in the desktop version the main window is resizeable. So this means there’s some additional care taken in connecting signals to make sure this we don’t get weird behaviour

func _on_root_resized():
    $FractalContainer/FractalViewport.size = size

func save_screenshot(screenshot_size : Vector2i):
    $FractalContainer/FractalViewport.set_size(screenshot_size)
    await RenderingServer.frame_post_draw
    img_to_save = $FractalContainer/FractalViewport.get_texture().get_image()
    $FractalContainer/FractalViewport.set_size(get_tree().root.get_viewport().size)
    #This is the function from before, which sets and pops up the dialog
    _save_image()

func _on_file_selected(path):
    if(dialog_is_img):
        img_to_save.save_png(path)
    else:
        pass

A picture is worth a couple dozen lines of code

At this point this feature is just working! After adding a button to the UI, I can click it and it’ll save your image! With the amount of help Godot gives here, it really doesn’t take a lot to get this working

You can check it out right now!

But I DID say web was going to need a different solution, right? And if you do check it out… that’s not the Godot FileDialog at all… Also there are even MORE new buttons there

As I mentioned before, this post was starting to get a little on the long side so, instead of just zooming by everything I’ve decided to take my time with this and split the post in two. This means we already know what next week’s going to be all about!

For the next part I’m going to talk about implementing the “state save and load” feature as well as how it was massaging all of this to work in the web. Some of it was easy, some of it was tricky, some of it was uncooperative browsers

I’ll see you folks next week and hope you have fun until then

The Great Refactoring