About VK_EXT_sample_locations

More than a year ago, I had worked on the implementation of VK_EXT_sample_locations extension for anv, the Intel Vulkan driver of mesa3D, as part of my work for Igalia. The implementation had been reviewed (see acknowledgments) at the time, but as the conformance tests that were available back then had to be improved and that was work in progress, the feature was stalled, and I had forgotten about it… Until some weeks ago, when I realized that it has been merged into mesa! 🎉

As this feature is now available to the users, I’ve decided to write a short blog post to explain what this extension is about. You can read the extension’s specification if you are interested in learning how to use it and other details.

VK_EXT_sample_locations

When multisampling (MSAA) is enabled, the sample points used to calculate the pixel colors have some “standard” locations in the pixel coordinate system that are hard coded in the driver. With VK_EXT_sample_locations it becomes possible to set them from Vulkan programs to create custom sampling patterns on the pixels and improve anti-aliasing.

From the spec:

This extension allows an application to modify the locations of samples within a pixel used in rasterization. Additionally, it allows applications to specify different sample locations for each pixel in a group of adjacent pixels, which can increase antialiasing quality (particularly if a custom resolve shader is used that takes advantage of these different locations).

With this extension, the user can query the GPUs multisampling capabilities (maximum number of samples that can be sampled at rasterization, if a grid of pixels is supported and more) and then set N custom (x, y) locations where x, y ∈ [0, 1] on a pixel or grid and N is one of the supported number of samples (eg: 1, 2, 4, 8, 16).

It is also possible to modify the sample locations used to sample a texture during the initial layout transition of attachments if this is supported by the hardware (see also limitations below).

Programmable sample locations example

Let’s see an example where we set the sample locations of 4 samples. We render two quads (red and blue) that share a common edge 3 times. The first time we set all the locations to the left, the second to the right and the third time we use the default locations that are uniformly distributed on the pixel.

If you zoom the output image you will see that when we use the default sample locations that are more uniformly distributed the common edge is purple (which is the average of blue and red), when locations are set to the left the left color is sampled and the common edge is red, and when the locations are set to the right blue is sampled and the common edge is blue:

The output is something like that:

 
This program can help you see the effect of changing the sample locations to some extreme positions. A trick you could try to check how another pattern of locations affects your rendering is to render triangles of different colors that are smaller than a pixel in your sample locations positions. Display them one by one to check that each location is set as expected (if you see the triangle’s color, location was sampled) and then display combinations of them to check how the average pixel color is calculated.

Limitations

Some Intel GPUs have the following hardware limitations (not so important IMO):

• Grid size is usually one pixel: you can set a pattern on a pixel but not on a grid of pixels. The grid size can be queried and be returned in one of the fields of VkSampleLocationsInfoEXT.
• Applications can always create sampling patterns for rasterization but cannot change them when the initial layout transition of an attachment takes place.

Make sure you always query the GPU capabilities before setting the locations or validation layers are enabled!

Acknowledgments

More people had contributed to this extension.
Two patches came from:

• Jason Ekstrand
• Lionel Landwerlin

and the reviews came from:

• Jason Ekstrand
• Lionel Landwerlin
• Sagar Ghuge

(many thanks to all of them!)

Links:

Some documentation and papers I’ve read to understand the role of sample locations in rasterization, how this extension should be used and which could be some convenient sample location patterns for applications:

[1]: VK_EXT_sample_locations extension specification:
Contains the structs and functions that can be used to set the sample locations and instructions.

[2]: Intel Programming Reference Manual:
Any Intel GPUs hardware restrictions can be found here.

[3]: A parallel algorithm for polygon rasterization, by Juan Pineda:
Modern GPUs don’t use the scanline based algorithms we use in software rasterization but parallel algorithms based on the so called “Pineda rasterization”. Terms like “coverage” (a percentage that shows how much a pixel is “covered”) are explained in this paper which is a good introductory reading although it doesn’t cover the details of MSAA.

[4]: A trip through the Graphics Pipeline 2011, part 6, by Fabian “ryg” Giesen:
Papers mentioned under “What we need around here is more hierarchy” section are about rasterization with MSAA in modern GPUs. They helped me understand how the multiple samples are used to calculate the average color in the driver and might be useful to people planning to modify the locations:

[5]: Supersampling patterns (Wikipedia):
Some known sample locations patterns that are used in supersampling and could be set with this extension.

That was a brief overview of the extension and things you can do with it, for more details you can check the specs above.