Name NV_stream_consumer_gltexture_yuv Name Strings EGL_NV_stream_consumer_gltexture_yuv Contributors James Jones Daniel Kartch Nikhil Mahale Daniel Koch Jeff Gilbert Contacts James Jones, NVIDIA (jajones 'at' nvidia 'dot' com) Status Complete Version Version 4 - November 14, 2017 Number EGL Extension #94 Extension Type EGL display extension Dependencies Requires EGL_KHR_stream_consumer_gltexture References EGL_EXT_yuv_surface Interactions with EGL_EXT_yuv_surface This extension makes use of several tokens defined in the EGL_EXT_yuv_surface extension spec. However support for this EGLStream extension does not require the EGLSurface extension, or vice versa. Only the tokens are shared. Overview The EGL_KHR_stream_consumer_gltexture extension allows EGLStream frames to be latched to a GL texture for use in rendering. These frames are assumed to be stored in RGB format and accessed as such by shader programs. If the producer uses a different color space, the stream implementation must perform an implicit conversion. In cases where the producer operates in a native YUV color space, it may be desirable for shaders to directly access the YUV components, without conversion. This extension adds a new variant of the function to bind GL textures as stream consumers which allows attributes to specify the color space. New Types None New Functions EGLBoolean eglStreamConsumerGLTextureExternalAttribsNV( EGLDisplay dpy, EGLStreamKHR stream, const EGLAttrib *attrib_list) New Tokens Accepted as attribute name in by eglStreamConsumerGLTextureExternalAttribsNV: EGL_YUV_PLANE0_TEXTURE_UNIT_NV 0x332C EGL_YUV_PLANE1_TEXTURE_UNIT_NV 0x332D EGL_YUV_PLANE2_TEXTURE_UNIT_NV 0x332E Reused Tokens From EGL_EXT_yuv_surface Accepted as attribute name in by eglStreamConsumerGLTextureExternalAttribsNV: EGL_YUV_NUMBER_OF_PLANES_EXT 0x3311 Accepted as value for EGL_COLOR_BUFFER_TYPE attribute in by eglStreamConsumerGLTextureExternalAttribsNV: EGL_YUV_BUFFER_EXT 0x3300 Replace entire description of eglStreamConsumerGLTextureExternalKHR in section "3.10.2.1 GL Texture External consumer" of EGL_KHR_stream_consumer_gltexture extension. Call EGLBoolean eglStreamConsumerGLTextureExternalAttribsNV( EGLDisplay dpy, EGLStreamKHR stream, const EGLAttrib *attrib_list) to connect one or more texture objects in the OpenGL or OpenGL ES context current to the calling thread as the consumer(s) of . The identity and format of the texture objects used are determined by and the current context state. must either be NULL or point to an array of name/value pairs terminated by EGL_NONE. Valid attribute names are EGL_COLOR_BUFFER_TYPE, EGL_YUV_NUMBER_OF_PLANES_EXT, and EGL_YUV_PLANE_TEXTURE_UNIT_NV. If the value of EGL_COLOR_BUFFER_TYPE is EGL_RGB_BUFFER (the default), then the stream will be connected to a single texture whose contents are available to shaders as RGB values. If the value of EGL_COLOR_BUFFER_TYPE is EGL_YUV_BUFFER_EXT the stream will be connected to some number of planar textures, determined by the value of EGL_YUV_NUMBER_OF_PLANES_EXT, whose contents are available to shaders as YUV values. The mapping between YUV values and texture contents is described in table 3.10.2.1. If EGL_COLOR_BUFFER_TYPE is EGL_YUV_BUFFER_EXT, the default value of EGL_YUV_NUMBER_OF_PLANES_EXT is 2. Otherwise it is 0. PLANE0 PLANE1 PLANE2 # Planes Values Fields Values Fields Values Fields -------------------------------------------------------------- 1 YUV XYZ unused unused 2 Y X UV XY unused 3 Y X U X V X Table 3.10.2.1 YUV Planar Texture Mappings If EGL_COLOR_BUFFER_TYPE is EGL_RGB_BUFFER, the stream is connected to the texture object currently bound to the active texture unit's GL_TEXTURE_EXTERNAL_OES texture target in the current context. If EGL_COLOR_BUFFER_TYPE is EGL_YUV_BUFFER_EXT, attribute values must be specified for EGL_YUV_PLANE_TEXTURE_UNIT_NV for all less than the number of planes. The value of each attribute must either be a valid texture unit index or EGL_NONE. No two of these attributes may specify the same valid texture unit index or reference separate texture units bound to the same texture object. Plane of the stream contents will be connected to the texture object currently bound to the indexed texture unit's GL_TEXTURE_EXTERNAL_OES texture target in the current context, or will be left unused if the index is EGL_NONE. Once connected, the stream will remain associated with the initial texture object(s) even if the texture units are bound to new textures. (Note: Before this can succeed a GL_TEXTURE_EXTERNAL_OES texture must be bound to the appropriate texture units of the GL context current to the calling thread. To create a GL_TEXTURE_EXTERNAL_OES texture and bind it to the current context, call glBindTexture() with set to GL_TEXTURE_EXTERNAL_OES and set to the name of the GL_TEXTURE_EXTERNAL_OES (which may or may not have previously been created). This is described in the GL_NV_EGL_stream_consumer_external extension.) On failure EGL_FALSE is returned and an error is generated. - EGL_BAD_DISPLAY is generated if is not a valid, EGLDisplay. - EGL_BAD_STREAM_KHR is generated if is not a valid EGLStreamKHR created for . - EGL_BAD_STATE_KHR is generated if is not in state EGL_STREAM_STATE_CREATED_KHR. - EGL_BAD_ATTRIBUTE is generated if any attribute name in is not a valid attribute. - EGL_BAD_PARAMETER is generated if the value of EGL_COLOR_BUFFER_TYPE is not EGL_RGB_BUFFER or EGL_YUV_BUFFER_EXT. - EGL_BAD_MATCH is generated if EGL_COLOR_BUFFER_TYPE is EGL_RGB_BUFFER and EGL_YUV_NUMBER_OF_PLANES_EXT is not 0, or if EGL_COLOR_BUFFER_TYPE is EGL_YUV_BUFFER_EXT and EGL_YUV_NUMBER_OF_PLANES_EXT is not 1, 2, or 3. - EGL_BAD_MATCH is generated if any EGL_YUV_PLANE_TEXTURE_UNIT_NV is not specified for any less than EGL_YUV_NUMBER_OF_PLANES_EXT, or if it is specified for any greater than or equal to EGL_YUV_NUMBER_OF_PLANES_EXT. - EGL_BAD_ACCESS is generated if any EGL_YUV_PLANE_TEXTURE_UNIT_NV is set to anything other than a valid texture unit index or EGL_NONE. - EGL_BAD_ACCESS is generated if there is no GL context current to the calling thread. - EGL_BAD_ACCESS is generated unless nonzero texture object names are bound the GL_TEXTURE_EXTERNAL_OES texture target of each of the appropriate texture units of the GL context current to the calling thread. - EGL_BAD_ACCESS is generated if more than one planar surface would be bound to the same texture object. - EGL_BAD_ACCESS is generated if the implementation cannot support the requested planar arrangement. On success the texture(s) are connected to the , is placed in the EGL_STREAM_STATE_CONNECTING_KHR state, and EGL_TRUE is returned. When a producer is later connected, if it cannot support the planar arrangement of the GL texture connection, it will fail with an EGL_BAD_ACCESS error. If any texture is later deleted, connected to a different EGLStream, or connected to an EGLImage, then will be placed into the EGL_STREAM_STATE_DISCONNECTED_KHR state. If the is later destroyed then the textures will be "incomplete" until they are connected to a new EGLStream, connected to a new EGLImage, or deleted. The function EGLBoolean eglStreamConsumerGLTextureExternalKHR( EGLDisplay dpy, EGLStreamKHR stream) is equivalent to eglStreamConsumerGLTextureExternalAttribsNV with list set to NULL. In the remainder of section "3.10.2.1 GL Texture External consumer", replace all singular references to "texture" with "textures" and make appropriate grammatical modifications. Issues 1. This competes with GL_EXT_yuv_target as a means for specifying how YUV values can be directly accessed by a texture shader without conversion to RGB. However, that extension also requires a means to render to YUV surfaces in addition to using them as textures. Should we go with the approach used here or create a GL extension which defines a subset GL_EXT_yuv_target? RESOLVED: The extension as is serves immediate needs. Conflicts and overlap with other extensions will be addressed if and when there is a need to promote to EXT. 2. This also contradicts how previous extensions for EXTERNAL GL textures bind multiplanar surfaces, using separate texture objects rather than a single virtual texture object which requires multiple texture units. This allows the application greater control of the planar arrangement, and the ability to leave planes unbound, which may reduce overhead for the producer. But it makes applications less portabile if the desired arrangement isn't supported. RESOLVED: The extension as is serves immediate needs. Conflicts and overlap with other extensions will be addressed if and when there is a need to promote to EXT. Revision History #4 (November 14, 2017) Mozilla Corporation - Const-qualify attrib_list. #3 (August 19, 2015) NVIDIA Corporation - Added enum values. - Cleaned up and added contact info for publication. #2 (May 6, 2015) NVIDIA Corporation - Consolidated error codes to make GL interaction simpler. #1 (April 15, 2015) NVIDIA Corporation - Initial draft