This is the second post of the OpenGL and Vulkan interoperability series, where I explain some EXT_external_objects and EXT_external_objects_fd use cases with examples taken by the Piglit tests I’ve written to test the extensions as part of my work for Igalia‘s graphics team.
We are going to see a very simple case of Vulkan/GL interoperability where an image is allocated using Vulkan and filled using OpenGL. This case is implemented in Piglit’s vk-image-overwrite test for images of different formats.
The vk-image-overwrite test:
- Allocates an image using Vulkan.
- Creates an OpenGL texture from the Vulkan allocated image memory.
- Fills the texture with blue color using OpenGL.
- Compares the texture values in a shader with blue: when the input color is blue the fragment is painted green when not it is painted red.
- Checks if all pixels are green and if yes the test passes, if at least one non-green pixel is found it fails
- The rendering/check is repeated for many images of different formats and tiling modes.
Step 1: Image allocation (Vulkan)
In tests/spec/ext_external_objects/vk_image_overwrite.c (see links below to checkout the code from Gitlab), we allocate each image in function run_subtest:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
if (!vk_set_image_props(piglit_width, piglit_height, d, num_samples, num_levels, vk_gl_format[case_num].vkformat, vk_gl_format[case_num].tiling, 0)) { piglit_report_subtest_result(PIGLIT_SKIP, "%s: Unsupported image format.", vk_gl_format[case_num].name); return PIGLIT_SKIP; } if (!vk_create_ext_image(&vk_core, &vk_img_props, &vk_img_obj)) { piglit_report_subtest_result(PIGLIT_FAIL, "%s: Failed to create external Vulkan image.", vk_gl_format[case_num].name); return PIGLIT_FAIL; } |
Functions prefixed as vk_ can be found in tests/spec/ext_external_objects/vk.[hc] and are Vulkan helper functions. As the name implies vk_set_image_properties just set the image properties. The important function is vk_create_ext_image that allocates an external Vulkan image (which is an image whose memory can be accessed by OpenGL):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
bool vk_create_ext_image(struct vk_ctx *ctx, struct vk_image_props *props, struct vk_image_obj *img) { VkExternalMemoryImageCreateInfo ext_img_info; VkImageCreateInfo img_info; memset(&ext_img_info, 0, sizeof ext_img_info); ext_img_info.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_IMAGE_CREATE_INFO; ext_img_info.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT_KHR; memset(&img_info, 0, sizeof img_info); img_info.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO; img_info.pNext = &ext_img_info; [...] |
In the snippet above I’ve used an extra Vulkan structure VkExternalMemoryImageCreateInfo when I filled VkImageCreateInfo and then I allocated the image calling the helper function alloc_memory (see vk.c) here:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
static VkDeviceMemory alloc_memory(struct vk_ctx *ctx, const VkMemoryRequirements *mem_reqs, VkMemoryPropertyFlagBits prop_flags) { VkExportMemoryAllocateInfo exp_mem_info; VkMemoryAllocateInfo mem_alloc_info; VkDeviceMemory mem; memset(&exp_mem_info, 0, sizeof exp_mem_info); exp_mem_info.sType = VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO; exp_mem_info.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT; memset(&mem_alloc_info, 0, sizeof mem_alloc_info); mem_alloc_info.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO; mem_alloc_info.pNext = &exp_mem_info; mem_alloc_info.allocationSize = mem_reqs->size; mem_alloc_info.memoryTypeIndex = get_memory_type_idx(ctx->pdev, mem_reqs, prop_flags); if (mem_alloc_info.memoryTypeIndex == UINT32_MAX) { fprintf(stderr, "No suitable memory type index found.\n"); return VK_NULL_HANDLE; } if (vkAllocateMemory(ctx->dev, &mem_alloc_info, 0, &mem) != VK_SUCCESS) return VK_NULL_HANDLE; return mem; } |
Here again a secondary VkExportMemoryAllocateInfo struct was necessary in order to set the VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT to the VkMemoryAllocateInfo struct.
Step 2: Creating an OpenGL texture from the Vulkan allocated image (Interoperability)
First of all, I had to import the Vulkan memory in OpenGL, in other words create a GL memory object from a Vulkan memory object. For that I used the function gl_create_mem_obj_from_vk_mem from tests/spec/ext_external_objects/interop.[hc] (which is the file that contains all the interoperability helper functions):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
bool gl_create_mem_obj_from_vk_mem(struct vk_ctx *ctx, struct vk_mem_obj *vk_mem_obj, GLuint *gl_mem_obj) { VkMemoryGetFdInfoKHR fd_info; int fd; PFN_vkGetMemoryFdKHR _vkGetMemoryFdKHR = (PFN_vkGetMemoryFdKHR)vkGetDeviceProcAddr(ctx->dev, "vkGetMemoryFdKHR"); if (!_vkGetMemoryFdKHR) { fprintf(stderr, "vkGetMemoryFdKHR not found\n"); return false; } memset(&fd_info, 0, sizeof fd_info); fd_info.sType = VK_STRUCTURE_TYPE_MEMORY_GET_FD_INFO_KHR; fd_info.memory = vk_mem_obj->mem; fd_info.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT; if (_vkGetMemoryFdKHR(ctx->dev, &fd_info, &fd) != VK_SUCCESS) { fprintf(stderr, "Failed to get the Vulkan memory FD"); return false; } glCreateMemoryObjectsEXT(1, gl_mem_obj); glImportMemoryFdEXT(*gl_mem_obj, vk_mem_obj->mem_sz, GL_HANDLE_TYPE_OPAQUE_FD_EXT, fd); if (!glIsMemoryObjectEXT(*gl_mem_obj)) return false; return glGetError() == GL_NO_ERROR; } |
In this snippet, I took the Linux file descriptor that corresponds to the Vulkan memory and its size from Vulkan, and I’ve created an OpenGL object using the EXT_memory_object_fd import function glImportMemoryFdEXT to assign the object id to the memory. Then I checked that this newly created object is a valid OpenGL memory object.
After that, I had to “tell” OpenGL that this object will correspond to a GL texture. Creating a texture from external memory is quite similar with creating a texture from some internal storage:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
bool gl_gen_tex_from_mem_obj(const struct vk_image_props *props, GLenum tex_storage_format, GLuint mem_obj, uint32_t offset, GLuint *tex) { GLint filter; GLuint target = gl_get_target(props); const struct sized_internalformat *format = get_sized_internalformat(tex_storage_format); GLint tiling = props->tiling == VK_IMAGE_TILING_LINEAR ? GL_LINEAR_TILING_EXT : GL_OPTIMAL_TILING_EXT; glGenTextures(1, tex); glBindTexture(target, *tex); glTexParameteri(target, GL_TEXTURE_TILING_EXT, tiling); switch (target) { case GL_TEXTURE_1D: assert(props->depth == 1); glTexStorageMem1DEXT(target, props->num_levels, tex_storage_format, props->w, mem_obj, offset); break; case GL_TEXTURE_2D: assert(props->depth == 1); glTexStorageMem2DEXT(target, props->num_levels, tex_storage_format, props->w, props->h, mem_obj, offset); break; case GL_TEXTURE_3D: glTexStorageMem3DEXT(target, props->num_levels, tex_storage_format, props->w, props->h, props->depth, mem_obj, offset); break; default: fprintf(stderr, "Invalid GL texture target\n"); return false; } } |
The GL call glTexStorageMem*DEXT that is introduced in EXT_memory_object creates a texture like glTexStorage would if the object was “internal”.
Something important when creating textures from external memory objects is to set the appropriate tiling mode calling glTexParameteri before calling glTexStorageMem*DEXT. This is because the external texture will become immutable at creation and changing the tiling mode in an immutable object is forbidden! The tiling mode should match the one used in Vulkan (default is optimal).
After this point, the texture was ready to be used by OpenGL.
Step 3: Rendering to texture (OpenGL)
I rendered to texture as I would do using an ordinary 2D texture render target. For simplicity I rendered all the pixels blue.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
static bool gl_draw_texture(enum fragment_type fs_type, uint32_t w, uint32_t h) { glBindTexture(gl_target, gl_tex); glBindFramebuffer(GL_FRAMEBUFFER, gl_fbo); glBindRenderbuffer(GL_RENDERBUFFER, gl_rbo); glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH24_STENCIL8, w, h); glBindRenderbuffer(GL_RENDERBUFFER, 0); glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_STENCIL_ATTACHMENT, GL_RENDERBUFFER, gl_rbo); glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, gl_target, gl_tex, 0); if (!check_bound_fbo_status()) return false; glClearColor(1.0, 1.0, 0.0, 1.0); glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); glEnable(GL_DEPTH_TEST); glBindFramebuffer(GL_FRAMEBUFFER, 0); glDisable(GL_DEPTH_TEST); glClearColor(0.0, 0.0, 1.0, 1.0); glClear(GL_COLOR_BUFFER_BIT); return glGetError() == GL_NO_ERROR; } |
In the snippet above I used the Vulkan-memory texture as the color attachment of a previously generated FBO. Then I cleared the framebuffer color to blue.
Step 4: Checking that the pixels have the correct color
Next thing I did was to check if the pixels are actually blue using a pixel shader. The shader compares each pixel’s color with an expected color passed as a uniform to the shader if the colors match the fragment is painted green else red. The uniform was necessary only because I wanted to check different format images (int, float, uint) and so I couldn’t hardcode the blue. 🙂
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
#define MAKE_FS(SAMPLER, VEC4) \ "#version 130\n" \ "in vec2 tex_coords;\n" \ "uniform " #SAMPLER " tex; \n" \ "uniform " #VEC4 " expected_color;\n" \ "out vec4 color;\n" \ "void main() \n" \ "{\n" \ " " #VEC4 " sampled_color = texture(tex, tex_coords);\n" \ " " #VEC4 " res = " #VEC4 " (abs(expected_color - sampled_color));\n" \ " res.x += res.y + res.z + res.w;\n" \ " color = mix(vec4(1.0, 0.0, 0.0, 1.0), vec4(0.0, 1.0, 0.0, 1.0), step(0, -float(res.x)));\n" \ "}\n" static const char *fs[] = { MAKE_FS(sampler2D, vec4), MAKE_FS(isampler2D, ivec4), MAKE_FS(usampler2D, uvec4), }; |
(I used the MAKE_FS macro to allow different formats and I used different expected colors depending on the number of bits and if they were int or float and signed or unsigned (see run_subtest of vk_image_overwrite.c for more).
Step 5: Checking the result with some piglit helper functions
After having rendered a green or green/red quad in step 4, I used some simple piglit helper functions to check if all pixels are green. That was necessary because the purpose of the test was to validate that we can overwrite a Vulkan allocated image with OpenGL. I used one of the piglit_probe_* functions that can probe the color of one or more pixels for that. When the color was green the test passed when it was red it was failing. And that was all. Below I am going to explain some other parts of the test because they might be useful to understand parts of the follow-up examples.
Other details of the piglit test
All interoperability tests of the series are inside the tests/spec/ext_external_objects directory. The Vulkan helper functions used above (of the form vk_) can be found in tests/spec/ext_external_objects/vk.[hc] and the interoperability helper functions in tests/spec/ext_external_objects/interop.[hc]
Piglit provides some callbacks that are very similar in concept to the freeglut callbacks. In most tests, I only fill piglit_init that initializes the OpenGL context, and piglit_display that is similar to GlutDisplayFunc callback and contains the operations that are related to the rendering.
Functions vk_init and gl_init initialize Vulkan and OpenGL data structures we need for all the subtests of the test respectively. piglit_require_extension is set to enable the other potentially required extensions.
To check if the OpenGL and Vulkan devices are compatible, we used the extended GetUnsignedBytei_v and GetUnsignedBytev to query the IDs of the device and the driver and see if they match each other as it is specified in EXT_external_objects. (I think that code was written by Juan A. Suarez Romero who also helped with the tests, credits!)
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
bool gl_check_vk_compatibility(const struct vk_ctx *ctx) { GLubyte deviceUUID[GL_UUID_SIZE_EXT]; GLubyte driverUUID[GL_UUID_SIZE_EXT]; /* FIXME: we select the first device so make sure you've * exported VK_ICD_FILENAMES */ glGetUnsignedBytei_vEXT(GL_DEVICE_UUID_EXT, 0, deviceUUID); glGetUnsignedBytevEXT(GL_DRIVER_UUID_EXT, driverUUID); if ((strncmp((const char *)deviceUUID, (const char *)ctx->deviceUUID, GL_UUID_SIZE_EXT) != 0) || (strncmp((const char* )driverUUID, (const char* )ctx->driverUUID, GL_UUID_SIZE_EXT) != 0)) { fprintf(stderr, "Mismatch in device/driver UUID\n"); return false; } return glGetError() == GL_NO_ERROR; } |
Note that for simplicity here we assumed that the user has only one Vulkan device. In later versions I detect the first device that supports graphics pipelines.
Links:
[1]: EXT_external_objects extension: https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_external_objects.txt
[2]: EXT_external_objects_fd extension: https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_external_objects_fd.txt
[3]: Piglit Code: https://gitlab.freedesktop.org/mesa/piglit
[4]: Samuel Iglesias Gonsálvez’s posts on Piglit: https://blogs.igalia.com/siglesias/2015/06/12/piglit-v-how-to-contribute-to-piglit-and-table-of-contents/
Previous posts in these series:
[1]: [OpenGL and Vulkan Interoperability on Linux] Part 1: Introduction https://eleni.mutantstargoat.com/hikiko/gl-vk-interop-intro/
See you next time!
Another great article. I wish I had this the last time I was setting interoperability code.
Because of immutability, tiling needs to be set before glTexStorageMem*DEXT is called. What about properties like GL_TEXTURE_WRAP_S?
Properties like GL_TEXTURE_WRAP_S are not changed by this extension. This particular property is related to sampling (how OpenGL will sample the texture), so you can use what you would use if the texture was a native OpenGL texture.
Any comments regarding using dedicated versus non dedicated memory in the context of interoperability between OpenGLES and Vulkan?
In some GPUs for example some AMD GPUs (that use radv, radeonsi) the external Vulkan memory is dedicated instead of suballocated. If you check vk.c of Piglit when we allocate image memory we can tell if dedicated memory is required (see alloc_image_memory) by checking the memory requirements. The dedicated memory support in the framework comes from a patch from Bas Nieuwenhuizen who works for AMD.
If glIsMemoryObjectEXT returns false, we would expect that glGetError() != GL_NO_ERROR right after glIsMemoryObjectEXT , right?
I meant to say right after glImportMemoryFdEXT
I use the GL_NO_ERROR check in every function, it’s not to check a particular command but to check that the whole block is error free. glIsMemoryObjectEXT returns false if the uid doesn’t correspond to a memory object so we will have already tested it when glGetError runs. The checks are unrelated to each other.
The idea is: we try to import the object. Then we check if it’s a valid memory object. At the end of the function (and each function) we check that there were no OpenGL errors (any type of errors).