From ecd7f6149adff83cd7751a381b41e4b387f1cf7a Mon Sep 17 00:00:00 2001 From: Eleni Maria Stea Date: Thu, 20 May 2021 10:48:46 +0300 Subject: [PATCH] notes --- notes | 253 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 253 insertions(+) create mode 100644 notes diff --git a/notes b/notes new file mode 100644 index 0000000..4de0a2b --- /dev/null +++ b/notes @@ -0,0 +1,253 @@ +Hi, I am Eleni Maria Stea and I am going to talk to you about a few +ideas I explored and experiments I've ran concerning the adoption of +ANGLE in WebKit/WPE. + +Before I begin, let's se what ANGLE is and what we want to do with it. + +So,ANGLE is an EGL/GLES implementation. + +GLES is the API we use to render graphics with the GPU +and EGL is the API we use to create the GLES context + +Both are implemented on top of other APIs. +In case of EGL these APIs can be the native system EGL, GLX, gbm or others. +In case of GLES they can be the native system driver GLES, OpenGL, Vulkan +or others. + +This is because the primary purpose of ANGLE is to provide an EGL/GLES +implementation to systems lacking it. + +The users can set the backend they prefer using some extra EGL +extensions provided by ANGLE in eglext_angle.h header file. + +On WebKit we'd like to use ANGLE in WebGL two that is implemented in +GLES two. The main reason is the performance. + +But there is a problem with our current design: +As ANGLE renders on a GLES 2 texture created by an ANGLE context +and the WebKit graphics pipeline renders on OpenGL textures that are +later assembled in one by the WebKit compositor, we currently need to +copy the WebGL texture data from an ANGLE texture to an OpenGL one. + +So all the experiments I am going to present are to replace this copy +with something better. + +Let's see a list of them. + +First I've written a program that uses ANGLE to render an image on an X11 +window and the native system drivers to render another image on another +X11 window and display them. + +That was to create a similar situation to the one we have on WebKit. + +Then, I experimented with shared context and textures. I've created a +shared texture between the native context and the native context of the +ANGLE backend. + +Then I filled it using the ANGLE context and displayed the result using +the native driver context. I had to abandon the shared context idea +for reasons that I'll explain later. + +My next experiment was to use the Linux kerne dma buffers framework to +associate two textures with the same kernel memory. Then filling one +texture was automatically filling the other. This method seemed to do +exactly what we want and so I am going to use it in WebKit as well. + +Last experiment was to try something similar but with multiple processes. +We might need it in the future. + +Let's see them in detail. + +First experiment + +For all my experiments I've written small standalone programs because +WebKit is complex and building it takes ages. So, in order to use ANGLE +in them I needed to build ANGLe with debugging symbols, and to find some +way to use two implementations of the same API in the same program. + +So the first step of the first experiment was to set up ANGLE for debugging +with GDB. I will quickly mention that ANGLE by default redirects the +debugging symbols and uses non-standard paths so I had to modify the +build system arguments and the GDB settings to be able to step into ANGLE +functions. + +You can read about these in my blog post. + +So in this test, I've rendered two images one from ANGLE context and +one from EGL like in this screenshot where the green image is the ANGLE one +and the XOR pattern comes from the native driver. + +In order to have both implementations working together I linked with the +system driver libEGL and libGLES and I dynamically opened ANGLE's libEGL +and libGLES and loaded each EGL and gl function prefixed by angle_. + +One other thing I needed for this experiment to work was to invalidate the +ANGLE context before every draw call. This is an interesting ANGLE problem. + +ANGLE is using some sort of internal context caching. This is to avoid +calling MakeCurrent when it's not absolutely necessary. So every time we +call angle_MakeCurrent ANGLE checks if the context has been changed and if +not it ignores the call. + +When we use two contexts from two drivers in the same process ANGLE +considers it's the only EGL implementation available in the program. +So, in a rendering loop like the one of the slide, ANGLE would always +see angle_MakeCurrent being called with the same context and wouldn't +update it. +A simple solution is to invalidate it by calling angle_MakeCurrent using +a null context at the top of display. This problem wouldn't happen with +threads or multiple processes but it's interesting to know it. + +You can read more about these changes in my blog in the blog post sections +mentioned in the slide. + +Let's see the next experiment. In this one I tried to exchange texture data +using shared textures and shared context. + +So, what's shared context? + +When we create a new context with EGL we can set an existing context +as parameter and use it as "shared". After that all textures created by +the shared context can be directly accessed from the other context without +leaving the GPU. + +For the shared context to work both contexts must be created by the same +process and both context must be created by the same API. + +This means that we can't use shared context directly between EGL and ANGLE, +but we can use it between the EGL/OpenGL native system driver and the +EGL/OpenGL ANGLE backend. + +========== + +To do so in WebKit, we'd need to force the EGL/OpenGL backend in ANGLE +and write an ANGLE extension that can tell ANGLE when the shared +context is native and not ANGLE so that it's used in the backend call. + +========= + +This is how I had designed this extension. In the first picture of the +slide we see how we'd create shared context with native EGL and with +ANGLE. + +In the second image we see how we'd pass a native context to ANGLE. One +of the EGL attributes passed to angle_eglCreateContext should indicate +that the context is native +========== + +And this is what happens internally in ANGLE: +When the extension attribute is set we create a shared context in the +backend. When not, we handle the shared context as ANGLE shared context. + +=========== + +Why we've rejected this method? +Main reason is that we plan to split the graphics operations in +multiple processes on WebKit. + +But fortunately, there was a better approach to follow. + +============================= + +In that approach, I've used a Linux kernel framework for content sharing: +DMA buffers. + +======================== + +This framework can be used to import dma buffers that are used +as backing storage for textures from other drivers. This is handy +because if two textures use the same dma buffer storage, filling +one will fill the other. + +It has some advantages like that it can easily used by EGL as there are +EGL extensions for it. +it's driver independent +it works with multiple processes and + +as long as ANGLE can implement and expose the extensions to import a +dma buffer it's also angle backend independent. + +The only problem is that DMA buffers are used from Linux only while +shared context is universal. But in this particular WebKit backend, +we don't care. + +=================================== + +I have a list of the EGL extensions that can be used to share a buffer +among drivers and links to the two example programs I've written. + +==================================== + +Let's see how we could access a DMA buffer from context A in context B. +In this example texture texA corresponds to contextA and texture texB is +from texture B. + +In the exporter that is context A, we create an EGL image from contextA +and we export some information about it as well as the file descriptor +that corresponds to its underlying kernel dma buffer. + +struct tex_storage_info stores the extra information. + +======================================= + +In the importer that is context B we can then +create another image that accesses the dma buffer using +the file descriptor and the information we extracted before + +and we can use it to create another texture tex B using +where the underline dma buffer is the imported one + +Then filling tex B will fill tex A. + +========================================= + +I've followed these steps with context A being native and context B +being ANGLE and created a similar case to the one we had in WebKit. + +It worked well! So, I've tested also the multiple processes case. + +============================== + +DMA buffers seem to work with multiple processes but require some sort +of Interprocess communication to exchange the file descriptor and the +information about the buffer we've seen before. + +I've found a client server example that uses unix sockets to exchange the +file descriptor and works well. Code is on gitlab. + +================================ + +So, finally let's see if this could be integrated to WebKit. + +The extensions to import dma buffers are fully implemented and exposed in +ANGLE and so in WebGL we could import a dma buffer from a graphics pipeline +texture when we create the WebGL render target. + +The extension to export dma buffers is implemented on Mesa and so it can be +used in main pipeline. In case it's not found we could fallback to some +alternative that is copying or maybe libgbm. + +So this week I am investigating what needs to be done on WebKit. + +- I've build ANGLE and fixed the compile errors of that backend +- I am currently fixing some link errors (unfortunately there's one I +haven't found yet) +- and I've taken a look at the code where I need to make the change. + +========================== + +References... I intend to upload these slides, and so I've appended a list +of links that could be useful in this task at the end. Feel free to +take a look at it + +============================ + +And Closing +I'd like to thank you very much for watching my talk and for +giving me the opportunity to work on this task! + +Goodbye and have a nice day or evening! + +========================== + + -- 1.7.10.4