This is the 4th post on OpenGL and Vulkan Interoperability on Linux. The first one was an introduction to EXT_external_objects and EXT_external_objects_fd extensions, the second was describing a simple interoperability use case where a Vulkan allocated textured is filled by OpenGL, and the third was about a slightly more complex use case where a Vulkan texture was filled by Vulkan and displayed by OpenGL. In this 4th and last post about shared textures, we are going to see a use case where a Vulkan texture is filled by Vulkan, then gets overwritten by OpenGL, then is read back from Vulkan and then displayed again using OpenGL. This more complex use case has also been written for Piglit using the small Vulkan framework I’ve written to test the external objects extensions. The source code can be found inside the tests/spec/ext_external_objects directory of the mesa/piglit master branch.
The test vk-image-display-overwrite:
- Allocates an external image using Vulkan and renders the stripped pattern we’ve seen in Part 3 again using Vulkan.
- Creates an OpenGL texture from the external image’s memory
- Renders another stripped pattern on the external image using OpenGL
- Maps the external Vulkan memory from Vulkan and takes a pointer to the new texture data.
- Validates that these data are the ones coming from the OpenGL rendering.
 |
 |
Step 1: Vulkan and OpenGL structs initialization
This step is very similar with the initialization we’ve seen in the previous test. Again we have a vk_init and a gl_init function that initialize the Vulkan and OpenGL structs that are required for each rendering. They are both called in piglit_init where we also import the external resources:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
if (!vk_init(w, h, d, num_samples, num_levels, num_layers, color_format, depth_format, color_usage, depth_usage, color_tiling, depth_tiling, color_in_layout, depth_in_layout, color_end_layout, depth_end_layout)) { fprintf(stderr, "Failed to initialize Vulkan, skipping the test.\n"); piglit_report_result(PIGLIT_SKIP); } if (!gl_create_mem_obj_from_vk_mem(&vk_core, &vk_color_att.obj.mobj, &gl_mem_obj)) { fprintf(stderr, "Failed to create GL memory object from Vulkan memory.\n"); piglit_report_result(PIGLIT_FAIL); } if (!gl_gen_tex_from_mem_obj(&vk_color_att.props, gl_tex_storage_format, gl_mem_obj, 0, &gl_tex)) { fprintf(stderr, "Failed to create texture from GL memory object.\n"); piglit_report_result(PIGLIT_FAIL); } if (!gl_create_semaphores_from_vk(&vk_core, &vk_sem, &gl_sem)) { fprintf(stderr, "Failed to import semaphores from Vulkan.\n"); piglit_report_result(PIGLIT_FAIL); } if (!gl_init()) { fprintf(stderr, "Failed to initialize structs for GL rendering.\n"); piglit_report_result(PIGLIT_FAIL); } |
vk_init in this test is slightly different from vk_init of vk-image-display's that we’ve seen in Part 3: after creating and initializing the structs we are going to need for rendering, we need to also create a Vulkan buffer:
|
1 2 3 4 |
if (!vk_create_buffer(&vk_core, w * h * 4 * sizeof(float), VK_BUFFER_USAGE_TRANSFER_DST_BIT, 0, &vk_bo)) { fprintf(stderr, "Failed to create buffer.\n"); goto fail; } |
This buffer is only necessary to allow us reading back the Vulkan image pixels. As the external image will be stored in device memory we can’t always map it from a user’s program. In some GPUs like Intel where the device memory is RAM a direct image mapping is allowed. But when the device memory is actual GPU memory, we usually have to copy each image to a buffer to read its data back.
Step 2: Vulkan renderpass
In piglit_display (similar to FreeGLUT display callback), we render again using Vulkan (like in Part 3):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
GLuint layout = gl_get_layout_from_vk(color_in_layout); if (vk_sem_has_wait) { glSignalSemaphoreEXT(gl_sem.gl_frame_ready, 0, 0, 1, &gl_tex, &layout); glFlush(); } struct vk_image_att images[] = { vk_color_att, vk_depth_att }; vk_draw(&vk_core, 0, &vk_rnd, vk_fb_color, 4, &vk_sem, vk_sem_has_wait, vk_sem_has_signal, images, ARRAY_SIZE(images), 0, 0, w, h); layout = gl_get_layout_from_vk(color_end_layout); if (vk_sem_has_signal) { glWaitSemaphoreEXT(gl_sem.vk_frame_done, 0, 0, 1, &gl_tex, &layout); } |
We use semaphores for the GL server to block while Vulkan is rendering the stripped image we’ve rendered in the previous example:
Step 3: OpenGL overwriting
Let us see how OpenGL reuses and overwrites the image:
First of all, remember that we have imported the object at initialization (see piglit_init:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
if (!gl_create_mem_obj_from_vk_mem(&vk_core, &vk_color_att.obj.mobj, &gl_mem_obj)) { fprintf(stderr, "Failed to create GL memory object from Vulkan memory.\n"); piglit_report_result(PIGLIT_FAIL); } if (!gl_gen_tex_from_mem_obj(&vk_color_att.props, gl_tex_storage_format, gl_mem_obj, 0, &gl_tex)) { fprintf(stderr, "Failed to create texture from GL memory object.\n"); piglit_report_result(PIGLIT_FAIL); } |
These functions are explained in more detail in Part 1 of the series. After calling them gl_tex is a valid GLuint OpenGL texture that we can use as render target for a new rendering operation.
To do so, in gl_init we need to create a framebuffer object (FBO) whose color attachment is gl_tex:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
glGenFramebuffers(1, &gl_fbo); glGenRenderbuffers(1, &gl_rbo); glBindTexture(gl_target, gl_tex); glBindFramebuffer(GL_FRAMEBUFFER, gl_fbo); glBindRenderbuffer(GL_RENDERBUFFER, gl_rbo); glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH24_STENCIL8, w, h); glBindRenderbuffer(GL_RENDERBUFFER, 0); glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_STENCIL_ATTACHMENT, GL_RENDERBUFFER, gl_rbo); glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, gl_target, gl_tex, 0); if (!check_bound_fbo_status()) return false; |
Then, in piglit_display where the rendering takes place we can just bind the FBO and render the second stripped pattern where the middle band is gray instead of yellow to it:
|
1 2 3 4 5 6 7 |
/* OpenGL overwrites the image */ glBindTexture(gl_target, gl_tex); glBindFramebuffer(GL_FRAMEBUFFER, gl_fbo); glUseProgram(gl_prog_overwrite); piglit_draw_rect(-1, -1, 2, 2); glBindFramebuffer(GL_FRAMEBUFFER, 0); glFinish(); |
The shader program used in this case gl_prog_overwrite replaces the pattern with these two shaders:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
static const char vs_overwrite[] = "#version 130\n" "in vec4 piglit_vertex;\n" "in vec2 piglit_texcoord;\n" "out vec2 tex_coords;\n" "void main()\n" "{\n" " gl_Position = piglit_vertex;\n" " tex_coords = piglit_texcoord;\n" "}\n"; static const char fs_overwrite[] = "#version 130\n" "in vec2 tex_coords;\n" "uniform sampler2D tex; \n" "out vec4 color;\n" "const vec4 colors[] = vec4[] (\n" " vec4(1.0, 0.0, 0.0, 1.0),\n" " vec4(0.0, 1.0, 0.0, 1.0),\n" " vec4(0.0, 0.0, 1.0, 1.0),\n" " vec4(0.5, 0.5, 0.5, 1.0),\n" " vec4(1.0, 0.0, 1.0, 1.0),\n" " vec4(0.0, 1.0, 1.0, 1.0));\n" "void main()\n" "{\n" " int band = int(gl_FragCoord.x * 6.0 / 160.0);\n" " color = colors[band];\n" "}\n"; |
Note that it’s very important to call glFinish after rendering because we must not have any pending OpenGL commands or partially written memory when we are going to use the texture from Vulkan!
Step 4: Reading back the pixels using Vulkan
Next step is to validate that when we re-read the shared memory from Vulkan it is updated and contains the stripped pattern written by OpenGL.
In some GPUs where the Vulkan “device memory” is in reality RAM (for example the Intel integrated GPUs) it is possible to call vkMapImageMemory and take a pointer to the pixels, as the device memory is allocated using the VK_MEMORY_PROPERTY_HOST_COHERENT_BIT. This is not possible with GPUs where the device memory is located on the GPU and so to read back the pixels we first need to copy the Vulkan image to a Vulkan buffer:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
vk_copy_image_to_buffer(&vk_core, &vk_color_att, &vk_bo, w, h); if (vkMapMemory(vk_core.dev, vk_bo.mobj.mem, 0, vk_bo.mobj.mem_sz, 0, &pixels) != VK_SUCCESS) { fprintf(stderr, "Failed to map Vulkan image memory.\n"); piglit_report_result(PIGLIT_FAIL); } /* Because we can't render using Vulkan on piglit, we use the * pixels we've just read from Vulkan memory as texture data * in a new OpenGL texture */ glBindTexture(gl_target, gl_disp_tex); glTexImage2D(gl_target, 0, gl_tex_storage_format, w, h, 0, GL_RGBA, GL_FLOAT, pixels); glFinish(); vkUnmapMemory(vk_core.dev, vk_bo.mobj.mem); |
In the code snippet above, we copy the image to a buffer, then we map the buffer memory and take a pointer to the pixels. From this point we can do anything with them to validate them: display them with Vulkan, dump them to a file or anything.
I used them as pixel data in a new OpenGL texture created by glTexImage2D. The reason for that is that Piglit is an OpenGL testing framework and so I can only display OpenGL images. Moreover, I needed to use the piglit functions that probe pixel colors to validate their values as I’ll explain in the next step.
Step 5: Validating the pixel values using OpenGL
So, as I said, to validate that the new image contains the second stripped pattern with the gray band I’ve rendered it and then called piglit_probe_pixel_rgba for a pixel in the middle of each band. This function is calling glReadPixels internally.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
/* OpenGL renders the Vulkan image pixels we've just read * from memory */ glUseProgram(gl_prog); glBindTexture(gl_target, gl_disp_tex); piglit_draw_rect_tex(-1, -1, 2, 2, 0, 0, 1, 1); const float y = (float)piglit_height / 2.0; for (i = 0; i < 6; i++) { float x = i * (float)piglit_width / 6.0 + (float)piglit_width / 12.0; if (!piglit_probe_pixel_rgba(x, y, colors[i])) return PIGLIT_FAIL; } piglit_present_results(); |
And with this final piglit_present_results call, the second stripped pattern should be displayed:
And that was it. We’ve filled an image from Vulkan, we’ve overwritten it with OpenGL, we’ve read back the new content from Vulkan and we used it in a new OpenGL texture that we mapped in a quad to validate the change took place.
Links:
- [1] EXT_external_objects specification
- [2] EXT_external_objects_fd specification
- [3] Previous posts on interoperability:
- [OpenGL and Vulkan Interoperability on Linux] Part 1: Introduction
- [OpenGL and Vulkan Interoperability on Linux] Part 2: Using OpenGL to draw on Vulkan textures.
- [OpenGL and Vulkan Interoperability on Linux] Part 3: Using OpenGL to display Vulkan allocated textures.
- [OpenGL and Vulkan Interoperability on Linux]: The XDC 2020 presentation
What’s next:
The upcoming posts will be about exchanging pixel and vertex buffers between OpenGL and Vulkan. Stay tuned!
