Vulkan GLSL CMake Build System

Over the past couple of months I’ve gotten more familiar with Vulkan and everything that comes with it. During this process one thing that irked me was how to properly integrate the shader compilation into my projects. Over time I’ve used several different processes, from doing it manually using the CLI, to using a python script that compiles them for me. But after I while I settled on a solution that integrates into the CMake build system, and honestly just works perfectly and I haven’t looked back since.

In this article I want to go over how I designed this system and explain how and why it works, and hopefully leave you with inspiration for how you want to set it up in your project. Or copy the results of the script at the end of the article.

The main motivation for creating this was the lackluster results I had using the python script I had before. This script would just find all shader files in a directory and invoke the CLI for glslc and place the spir-v files next to it in the same directory. This had the following issues:

  1. If I was working on my shaders, I needed to remember to run this script for the shaders to update.
  2. If there were any errors during shader compilation, it was easy to miss.
  3. It had to recompile all the shaders, every time. While not an issue on smaller projects, this can get cumbersome as projects grow.

The plan

We’re going to design a CMakeLists.txt script that will perform the following:

  1. Find the glslc program on your machine
  2. Find all the shaders in your project
  3. Create a bin folder where all the spir-v files will be compiled to
  4. Compile each shader using glslc
  5. And create a custom target with in the build system

Directory structure

Project directory structure

The shader folder will contain all the shaders that are used in the project. glslc expects file extensions, like .frag and .vert to be able to compile. I’m using the .glsl extension for files that I want to include into my other shaders. This can be useful for sharing common structures and reusing constant values.

We will compile all the shaders into the shaders/bin directory. Here we append the .spv extension onto the files, so we still maintain a relationship between the files and can easily identify the binary back to the original shader file.

And I have created the shaders/CMakeLists.txt file. This will be the script where we are going to perform all the compilation steps.

Finding GLSLC

To compile the shaders we are going to make use of glslc. This is a wrapper around glslangValidator, which is the official shader compiler by the Khronos Group. glslc is part of Google's shaderc project, and adds additional options and customizations and integrates better with build systems (like we are doing now). This is part of the LunarG SDK, and if you haven't already you should install it.

To find glslc in CMake we are going to use the find_program command. As implied this command is used to find a program on the machine. On Windows this will make use of your Path variable and find the program through there. So make sure the Vulkan SDK bin folder is in your Path. You can easily test this by typing glslc in a terminal and getting this response:

glslc: error: no input files

So with that in mind we can write our first line of code:

find_program(GLSLC glslc HINTS 
    /usr/bin 
    /usr/local/bin 
    $ENV{VULKAN_SDK}/Bin/ 
    $ENV{VULKAN_SDK}/Bin32/)

if (NOT GLSLC)
    message(FATAL_ERROR "glslc not found at ${GLSLC}")
endif ()
  • GLSLC will be the name of the variable where we store the path to the glslc program
  • glslc is the name of the program we are trying to find
  • HINTS tells CMake to look for the program in additional directories
    • We want to do an extra check in the Linux bin directories
    • And we check the Vulkan SDK directory directly and look in the bin folders

Next up we do a check to see whether we actually found the program and if not, we log a message and exit.

Finding the shaders

Next we want to find all the shader files that should be compiled. We can assume that all the shader files in the shader directory is everything that needs to be processed.

set(SHADER_DIR "${CMAKE_CURRENT_SOURCE_DIR}")

file(GLOB_RECURSE SHADERS CONFIGURE_DEPENDS
        ${SHADER_DIR}/*.frag
        ${SHADER_DIR}/*.vert
        ${SHADER_DIR}/*.tesc
        ${SHADER_DIR}/*.tese
        ${SHADER_DIR}/*.geom
        ${SHADER_DIR}/*.comp
)

file(GLOB_RECURSE GLSL_SHADERS CONFIGURE_DEPENDS ${SHADER_DIR}/*.glsl)

First we create a variable called SHADER_DIR that is set to the current source directory, which is just the directory the CMakeLists.txt file is in. Then we use the file command to find files in specific directories and store them in a list variable called SHADERS.

  • GLOB_RECURSE recursively finds all files that match the specified patterns. So in this case we look for all the different shader file extensions in the shader directory.
  • SHADERS is the list in which we stored all the shader files that are found.
  • CONFIGURE_DEPENDS tells CMake to re-run this command automatically if any files matching the specified patterns are added or removed. Without it, CMake would only check for new files when the script is first run, not on future builds.

After that we do the same again, but for all the .glsl files. This is important because .glsl files can be included in other shader files, with this separate list we can set a dependency later that will recompile all shader files if a .glsl file was changed. Keep in mind that we also won’t be compiling our .glsl files, since their only purpose is to be included in other shader files.

Creating a bin folder

We want all our compiled spir-v files to be stored in a bin folder, but we need to make sure this exists first.

set(SPIRV_BIN "${CMAKE_CURRENT_SOURCE_DIR}/bin")
file(MAKE_DIRECTORY ${SPIRV_BIN})

First we create the SPIRV_BIN variable that saves the directory of the bin folder. Then we use the file command, with the MAKE_DIRECTORY sub-command to create a directory at the specified path.

Compiling the shaders

Next, we’re going to write a function that will compile a single shader, so that we can easily use it on each of the shaders we have gathered.

function(compile_shader shader)

endfunction()

foreach (SHADER ${SHADERS})
    compile_shader(${SHADER})
endforeach()

Here we have simple function definition and we invoke that function on all our shaders

function(compile_shader shader)
    get_filename_component(FILE_NAME ${shader} NAME)
    set(SPIRV_OUTPUT "${SPIRV_BIN}/${FILE_NAME}.spv")

endfunction()

Currently, the shader variable is the complete path of the shader file. We use the get_filename_component command to save the name of the shader file in the FILE_NAME variable. We use the NAME mode to get the file name without the directory, but with extension. We can then use that file name to construct the final path of our spir-v path. A path is constructed inside the bin folder, with the .spv extension appended, and saved inside the SPIRV_OUTPUT variable.

function(compile_shader shader)
    get_filename_component(FILE_NAME ${shader} NAME)
    set(SPIRV_OUTPUT "${SPIRV_BIN}/${FILE_NAME}.spv")

    add_custom_command(
            OUTPUT ${SPIRV_OUTPUT}
            COMMAND "${GLSLC}" ${shader} -o ${SPIRV_OUTPUT}
            COMMENT "Compiling ${shader} to SPIR-V"
            DEPENDS ${shader} ${GLSL_SHADERS}
            VERBATIM
    )
    
endfunction()

Now we’re finally getting to the meat of it. We’re going to add the add_custom_command command to our script. This is the part that will actually invoke the glslc program on our shader file and compile it into spir-v binaries. Let's go over it:

  1. OUTPUT here we set the path to where the resulting file should be written to. We set up the SPIRV_OUTPUT variable for this. This informs CMake whether it has to re-run this command if the .spv file was deleted.
  2. COMMAND here we describe the command that should be executed.
    1. The GLSLC variable contains where the glslc program is located.
    2. shader contains the directory of the shader file to be compiled.
    3. -o tells glslc to write the output to a file, and we can use SPIRV_OUTPUT again to tell glslc where to write to.
  3. COMMENT describes the message that will appear in the terminal when you are building the CMake project. Useful for informing you have what files are being compiled.
  4. DEPENDS sets a dependency on another file, in this case the input shader. Should any changes be made to this shader, CMake will recompile the file so it stays updated. Additionally, we also pass the GLSL_SHADERS here, meaning that if any .glsl shader files are changed all files are recompiled. This is a simple way of resolving the inclusion problem. Because if a .glsl file is changed, all files including that should be recompiled, so this is a brute force way of making sure all files stay up-to-date. A separate solution can be check the contents of the shader file for #include statements and set-up dependencies like that.
  5. VERBATIM ensures the arguments in the command are passed exactly as written, with proper escaping to avoid issues with special characters. It’s good practice to use this with this command.

With that we have a functioning CMake script that will compile all our shaders. But I want to add a few small changes to make sure it works properly in a project.

First we want to save all the compiled shader files into a list, and we’ll do that by appending the SPIRV_OUTPUT to a list variable called SPIRV_SHADERS, that is available in the parent scope.

function(compile_shader shader)
    # ... 
    
    list(APPEND SPIRV_SHADERS ${SPIRV_OUTPUT})
    set(SPIRV_SHADERS ${SPIRV_SHADERS} PARENT_SCOPE)
endfunction()

Then at the end of script we add a custom target for our shader compilation.

add_custom_target(Shaders ALL DEPENDS ${SPIRV_SHADERS})

We create a custom built target called Shaders that depends on all our compiled spir-v binaries. The ALL keyword makes sure Shaders is part of the default build process.

Then in our main CMakeLists.txt we can add the following.

add_subdirectory(shaders)
add_dependencies(Game Shaders)

We add the shaders sub-directory and then depend on it on our Game executable.

Conclusion

At the start, I described the issues with rudimentary scripts for compiling glsl to spir-v and needed something more robust. By writing a CMake script we were able to make shader compilation part of our CMake build process. This has the advantage that the shaders will automatically be compiled when we build the project. We set up dependencies between the spir-v binaries and the shader files, so that if any shader file is updated it will automatically recompile the shader, and it will even handle including .glsl shader files. Finally, if CMake receives an error code from glslc it will fail compilation, meaning the project won’t run if the shaders won’t successfully compile, and it will inform you with an error message.

I’ve been using this build script for a few weeks now, and haven’t encountered any issues yet. It’s a perfect addition to my build system, and hopefully it can help out yours too. Below you can find the complete script.

find_program(GLSLC glslc HINTS 
    /usr/bin 
    /usr/local/bin 
    $ENV{VULKAN_SDK}/Bin/ 
    $ENV{VULKAN_SDK}/Bin32/)

if (NOT GLSLC)
    message(FATAL_ERROR "glslc not found at ${GLSLC}")
endif ()

set(SHADER_DIR "${CMAKE_CURRENT_SOURCE_DIR}")

file(GLOB_RECURSE SHADERS CONFIGURE_DEPENDS
        ${SHADER_DIR}/*.frag
        ${SHADER_DIR}/*.vert
        ${SHADER_DIR}/*.tesc
        ${SHADER_DIR}/*.tese
        ${SHADER_DIR}/*.geom
        ${SHADER_DIR}/*.comp
)

file(GLOB_RECURSE GLSL_SHADERS CONFIGURE_DEPENDS
        ${SHADER_DIR}/*.glsl
)

set(SPIRV_BIN "${CMAKE_CURRENT_SOURCE_DIR}/bin")
file(MAKE_DIRECTORY ${SPIRV_BIN})

function(compile_shader shader)
    get_filename_component(FILE_NAME ${shader} NAME)
    set(SPIRV_OUTPUT "${SPIRV_BIN}/${FILE_NAME}.spv")

    add_custom_command(
            OUTPUT ${SPIRV_OUTPUT}
            COMMAND "${GLSLC}" ${shader} -o ${SPIRV_OUTPUT}
            COMMENT "Compiling ${shader} to SPIR-V"
            DEPENDS ${shader} ${GLSL_SHADERS}
            VERBATIM
    )

    list(APPEND SPIRV_SHADERS ${SPIRV_OUTPUT})
    set(SPIRV_SHADERS ${SPIRV_SHADERS} PARENT_SCOPE)
endfunction()

foreach (SHADER ${SHADERS})
    compile_shader(${SHADER})
endforeach ()

add_custom_target(Shaders ALL DEPENDS ${SPIRV_SHADERS})