Name KHR_stream KHR_stream_attrib Name Strings EGL_KHR_stream EGL_KHR_stream_attrib Contributors Marcus Lorentzon Acorn Pooley Robert Palmer Greg Prisament Daniel Kartch Miguel A. Vico Moya Contacts Acorn Pooley, NVIDIA (apooley 'at' nvidia.com) Marcus Lorentzon, ST-Ericsson AB (marcus.xm.lorentzon 'at' stericsson.com) Daniel Kartch, NVIDIA (dkartch 'at' nvidia.com) Notice Copyright (c) 2009-2016 The Khronos Group Inc. Copyright terms at http://www.khronos.org/registry/speccopyright.html Status Complete. Approved by the Khronos Board of Promoters on December 2, 2011. Version Version 27 - May 23, 2016 Number EGL Extension #32 Dependencies EGL_KHR_stream requires EGL 1.2. EGL_KHR_stream_attrib requires EGL_KHR_stream and EGL 1.5. EGL_KHR_stream_attrib interacts with EGL_KHR_stream_consumer_gltexture. This extension is written based on the wording of the EGL 1.2 specification. Overview This extension defines a new object, the EGLStream, that can be used to efficiently transfer a sequence of image frames from one API to another. The EGLStream has mechanisms that can help keep audio data synchronized to video data. Each EGLStream is associated with a "producer" that generates image frames and inserts them into the EGLStream. The producer is responsible for inserting each image frame into the EGLStream at the correct time so that the consumer can display the image frame for the appropriate period of time. Each EGLStream is also associated with a "consumer" that retrieves image frames from the EGLStream. The consumer is responsible for noticing that an image frame is available and displaying it (or otherwise consuming it). The consumer is also responsible for indicating the latency when that is possible (the latency is the time that elapses between the time it is retrieved from the EGLStream until the time it is displayed to the user). Some APIs are stream oriented (examples: OpenMAX IL, OpenMAX AL). These APIs may be connected directly to an EGLStream as a producer or consumer. Once a stream oriented producer is "connected" to an EGLStream and "started" it may insert image frames into the EGLStream automatically with no further interaction from the application. Likewise, once a stream oriented consumer is "connected" to an EGLStream and "started" it may retrieve image frames from the EGLStream automatically with no further interaction from the application. Some APIs are rendering oriented and require interaction with the application during the rendering of each frame (examples: OpenGL, OpenGL ES, OpenVG). These APIs will not automatically insert or retrieve image frames into/from the EGLStream. Instead the application must take explicit action to cause a rendering oriented producer to insert an image frame or to cause a rendering oriented consumer to retrieve an image frame. The EGLStream conceptually operates as a mailbox. When the producer has a new image frame it empties the mailbox (discards the old contents) and inserts the new image frame into the mailbox. The consumer retrieves the image frame from the mailbox and examines it. When the consumer is finished examining the image frame it is either placed back in the mailbox (if the mailbox is empty) or discarded (if the mailbox is not empty). Timing is mainly controlled by the producer. The consumer operated with a fixed latency that it indicates to the producer through the EGL_CONSUMER_LATENCY_USEC_KHR attribute. The consumer is expected to notice when a new image frame is available in the EGLStream, retrieve it, and display it to the user in the time indicated by EGL_CONSUMER_LATENCY_USEC_KHR. The producer controls when the image frame will be displayed by inserting it into the stream at time T - EGL_CONSUMER_LATENCY_USEC_KHR where T is the time that the image frame is intended to appear to the user. This extension does not cover the details of how a producer or a consumer works or is "connected" to an EGLStream. Different kinds of producers and consumers work differently and are described in additional extension specifications. (Examples of producer specifications: EGL_KHR_stream_producer_eglsurface EGL_KHR_stream_producer_aldatalocator OpenMAX_AL_EGLStream_DataLocator Example of consumer extension specification: EGL_KHR_stream_consumer_gltexture ) Glossary EGLStream An EGL object that transfers a sequence of image frames from one API to another (e.g. video frames from OpenMAX AL to OpenGL ES). Image frame A single image in a sequence of images. The sequence may be frames of video data decoded from a video file, images output by a camera sensor, surfaces rendered using OpenGL ES commands, or generated in some other manner. An image frame has a period of time during which it is intended to be displayed on the screen (starting with the "Image Frame Display Time" and ending with the "Image Frame Display Time" of the next image frame in the sequence). Image Frame Insertion Time The point in time when the producer inserts the image frame into the EGLStream. This is the "Image Frame Intended Display Time" minus the "Consumer Latency". Image Frame Intended Display Time The point in time when the user should first see the image frame on the display screen. Image Frame Actual Display Time The point in time when the user actually first sees the image frame on the display screen. Consumer Latency The elapsed time between an image frame's "Image Frame Insertion Time" and its "Image Frame Actual Display Time". The consumer is responsible for predicting this and indicating its value to the EGLStream. The producer is responsible for using this value to calculate the "Image Frame Insertion Time" for each image frame. The application has access to this value through the EGL_CONSUMER_LATENCY_USEC attribute. Producer The entity that inserts image frames into the EGLStream. The producer is responsible for timing: it must insert image frames at a point in time equal to the "Image Frame Intended Display Time" minus the "Consumer Latency". Consumer The entity that retrieves image frames from the EGLStream. When the image frames are to be displayed to the user the consumer is responsible for calculating the "Consumer Latency" and reporting it to the EGLSteam. State (stream state) At any given time an EGLStream is in one of several states. See section "3.10.4.3 EGL_STREAM_STATE_KHR Attribute" in this extension for a description of the states and what transitions occur between them. New Types This is the type of a handle that represents an EGLStream object. typedef void* EGLStreamKHR; This is a 64 bit unsigned integer. typedef khronos_uint64_t EGLuint64KHR; New functions defined by EGL_KHR_stream EGLStreamKHR eglCreateStreamKHR( EGLDisplay dpy, const EGLint *attrib_list); EGLBoolean eglDestroyStreamKHR( EGLDisplay dpy, EGLStreamKHR stream); EGLBoolean eglStreamAttribKHR( EGLDisplay dpy, EGLStreamKHR stream, EGLenum attribute, EGLint value); EGLBoolean eglQueryStreamKHR( EGLDisplay dpy, EGLStreamKHR stream, EGLenum attribute, EGLint *value); EGLBoolean eglQueryStreamu64KHR( EGLDisplay dpy, EGLStreamKHR stream, EGLenum attribute, EGLuint64KHR *value); New functions defined by EGL_KHR_stream_attrib EGLStreamKHR eglCreateStreamAttribKHR( EGLDisplay dpy, const EGLAttrib *attrib_list); EGLBoolean eglSetStreamAttribKHR( EGLDisplay dpy, EGLStreamKHR stream, EGLenum attribute, EGLAttrib value); EGLBoolean eglQueryStreamAttribKHR( EGLDisplay dpy, EGLStreamKHR stream, EGLenum attribute, EGLAttrib *value); EGLBoolean eglStreamConsumerAcquireAttribKHR( EGLDisplay dpy, EGLStreamKHR stream const EGLAttrib *attrib_list); EGLBoolean eglStreamConsumerReleaseAttribKHR( EGLDisplay dpy, EGLStreamKHR stream, const EGLAttrib *attrib_list); New Tokens This value is returned from eglCreateStreamKHR in the case of an error. It is an error to attempt to use this value as a parameter to any EGL or client API function. EGL_NO_STREAM_KHR ((EGLStreamKHR)0) This enum is accepted as an attribute in the parameter of eglCreateStreamKHR and as the parameter of eglStreamAttribKHR, eglSetStreamAttribKHR, eglQueryStreamKHR and eglQueryStreamAttribKHR. EGL_CONSUMER_LATENCY_USEC_KHR 0x3210 These enums are accepted as the parameter of eglQueryStreamu64KHR. EGL_PRODUCER_FRAME_KHR 0x3212 EGL_CONSUMER_FRAME_KHR 0x3213 This enum is accepted as the parameter of eglQueryStreamKHR and eglQueryStreamAttribKHR. EGL_STREAM_STATE_KHR 0x3214 Returned in the parameter of eglQueryStreamKHR or eglQueryStreamAttribKHR when is EGL_STREAM_STATE. EGL_STREAM_STATE_CREATED_KHR 0x3215 EGL_STREAM_STATE_CONNECTING_KHR 0x3216 EGL_STREAM_STATE_EMPTY_KHR 0x3217 EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR 0x3218 EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR 0x3219 EGL_STREAM_STATE_DISCONNECTED_KHR 0x321A These errors may be generated by EGLStream calls. EGL_BAD_STREAM_KHR 0x321B EGL_BAD_STATE_KHR 0x321C Add a new section "2.5 Streams" after section "2.4 Shared State" EGL allows efficient interoperation between APIs through the EGLStream object. An EGLStream represents a sequence of image frames. Each EGLStream is associated with a producer that generates image frames and inserts them into the EGLStream. Each EGLStream is also associated with a consumer that retrieves image frames from the EGLStream. Add a new section "3.10 EGLStreams" after section "3.9 Posting the Color Buffer" 3.10 EGLStreams EGL provides functions to create and destroy EGLStreams, for querying and setting attributes of EGLStreams, and for connecting EGLStreams to producers and consumers. Each EGLStream may be connected to only one producer and one consumer. Once an EGLStream is connected to a consumer, it will be connected to that consumer until the EGLStream is destroyed. Likewise, once an EGLStream is connected to a producer it will be connected to that producer until the EGLStream is destroyed. Further semantics are described for each type of consumer and producer that can be connected. Add subsection 3.10.1 to section "3.10 EGLStreams" 3.10.1 Creating an EGLStream Call EGLStreamKHR eglCreateStreamKHR( EGLDisplay dpy, const EGLint *attrib_list); to create a new EGLStream. specifies the EGLDisplay used for this operation. The function returns a handle to the created EGLStream. The EGLStream cannot be used until it has been connected to a consumer and then to a producer (refer to section "3.10.2 Connecting an EGLStream to a consumer" and section "3.10.3 Connecting an EGLStream to a producer"). It must be connected to a consumer before being connected to a producer. There is no way for the application to query the size, colorformat, or number of buffers used in the EGLStream (although these attributes may be available from the producer's API or the consumer's API depending on what type of producer/consumer is connected to the EGLStream). The parameter contains a list of attributes and values to set for the EGLStream. Attributes not in the list are set to default values. EGLStream attributes are described in section "3.10.4 EGLStream Attributes". If an error occurs eglCreateStreamKHR will return EGL_NO_STREAM_KHR and generate an error. - EGL_BAD_ATTRIBUTE is generated if any of the parameters in attrib_list is not a valid EGLStream attribute. - EGL_BAD_ACCESS is generated if any of the parameters in attrib_list is read only. - EGL_BAD_PARAMETER is generated if any of the values in attrib_list is outside the valid range for the attribute. - EGL_BAD_ALLOC is generated if not enough resources are available to create the EGLStream. - EGL_BAD_DISPLAY is generated if is not a valid, initialized EGLDisplay. If EGL_KHR_stream_attrib is present, add to the end of this section Streams may also be created by calling EGLStreamKHR eglCreateStreamAttribKHR( EGLDisplay dpy, const EGLAttrib *attrib_list); This is equivalent to eglCreateStreamKHR, but allows pointer and handle attributes to be provided on 64-bit systems. Add section 3.10.2 to section "3.10 EGLStreams" 3.10.2 Connecting an EGLStream to a consumer. Before using an EGLStream it must be connected to a consumer. Refer to sections 3.10.2.1 and following for different ways to connect a consumer to an EGLStream. Once an EGLStream is connected to a consumer it will remain connected to the same consumer until the EGLStream is destroyed. If the consumer is destroyed then the EGLStream's state will become EGL_STREAM_STATE_DISCONNECTED_KHR. Any attempt to connect an EGLStream which is not in state EGL_STREAM_STATE_CREATED_KHR will fail and generate an EGL_BAD_STATE_KHR error. When an EGLStream is connected to a consumer its state becomes EGL_STREAM_STATE_CONNECTING_KHR. 3.10.2.1 No way to connect consumer to EGLStream EGL does not currently define any mechanisms to connect a consumer to an EGLStream. These will be added via additional extensions. (Example: See extension specification EGL_KHR_stream_consumer_gltexture) If EGL_KHR_stream_attrib is present, add to the end of this section 3.10.2.2 Acquiring and releasing consumer frames Methods for acquiring frames from a stream and releasing them back to a stream are dependent on the type of consumer. Some consumers support calling EGLBoolean eglStreamConsumerAcquireAttribKHR( EGLDisplay dpy, EGLStreamKHR stream const EGLAttrib *attrib_list); to acquire the next available frame in and EGLBoolean eglStreamConsumerReleaseAttribKHR( EGLDisplay dpy, EGLStreamKHR stream, const EGLAttrib *attrib_list); to release a frame back to the stream. Not all consumers are required to support either or both of these functions. Where supported, the specific behavior is defined by the consumer type, and may be affected by the contents of . must either be NULL or a pointer to a list of name/value pairs terminated by EGL_NONE. Valid attributes are listed in tables 3.10.2.1 and 3.10.2.2. Attribute Type Section ------------------------ ---------- ------- Currently no acquire attributes are defined Table 3.10.2.1 EGLStream Consumer Acquire Attributes Attribute Type Section ------------------------ ---------- ------- Currently no release attributes are defined Table 3.10.2.2 EGLStream Consumer Release Attributes If no new image frame is available in the stream, eglStreamConsumerAcquireAtrribKHR may block, retrieve an old frame, or return an error, as defined by the type of consumer. If one or more image frames are already acquired by the consumer when eglStreamConsumerAcquireAttribKHR is called, the behavior is determined by the type of consumer. If successful, eglStreamConsumerAcquireAttribKHR returns EGL_TRUE and an image frame from will be bound into the address space of the consumer as defined for its type. On failure, the function returns EGL_FALSE and generates an error. Additionally, image objects in the consumer's address space may become invalid, as determined by the consumer type. - EGL_BAD_ACCESS is generated if the consumer of does not support acquiring frames through eglStreamConsumerAcquireAttribKHR. - EGL_BAD_STATE_KHR is no frame is available for acquisition after any timeout determined by the consumer. - EGL_BAD_ATTRIBUTE is generated if an attribute name in is not recognized or is not supported by the consumer. - EGL_BAD_STREAM_KHR is generated if is not a valid EGLStream created for . - EGL_BAD_DISPLAY is generated if is not a valid EGLDisplay. - EGL_NOT_INITIALIZED is generated if is not initialized. Calling eglStreamConsumerReleaseAttribKHR will release a frame held by the consumer back to the stream. If more than one frame is held by the consumer, the frame returned is determined by the consumer type and the contents of . If no frames are currently held, the behavior is determined by the consumer type. Once returned, the consumer may no longer access the contents of the frame, and attempts to do so will result in errors as determined by the consumer type. Upon success, eglStreamConsumerReleaseAttribKHR returns EGL_TRUE. If eglStreamConsumerReleaseAttribKHR fails, EGL_FALSE is returned and an error is generated. - EGL_BAD_ACCESS is generated if the consumer of does not support releasing frames through eglStreamConsumerReleaseAttribKHR. - EGL_BAD_STATE_KHR is generated if is not in state EGL_STREAM_STATE_EMPTY_KHR, EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR or EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR. - EGL_BAD_ATTRIBUTE is generated if an attribute name in is not recognized or is not supported by the consumer. - EGL_BAD_STREAM_KHR is generated if is not a valid EGLStream created for . - EGL_BAD_DISPLAY is generated if is not a valid EGLDisplay. - EGL_NOT_INITIALIZED is generated if is not initialized. If EGL_KHR_stream_consumer_gltexture is present in addition to EGL_KHR_stream_attrib, the eglStreamConsumerAcquireKHR function is equivalent to eglStreamConsumerAcquireAttribKHR with set to NULL, the eglStreamConsumerReleaseKHR function is equivalent to eglStreamConsumerReleaseAttribKHR with set to NULL, and the definitions provided for those functions define their behavior for a GL texture consumer. Add section 3.10.3 to section "3.10 EGLStreams" 3.10.3 Connecting an EGLStream to a producer. Before using an EGLStream it must be connected to a producer. The EGLStream must be connected to a consumer before it may be connected to a producer. The size and colorformat of the images in the EGLStream are determined by the EGL implementation based on the requirements of the producer and the consumer. The EGL implementation may determine these at the time the producer is connected to the EGLStream, at the time that the first image frame is inserted into the EGLStream, or any time in between (this is left up to the implementation). It is the responsibility of the producer to convert the images to a form that the consumer can consume. The producer may negotiate with the consumer as to what formats and sizes the consumer is able to consume, but this negotiation (whether it occurs and how it works) is an implementation detail. If the producer is unable to convert the images to a form that the consumer can consume then the attempt to connect the producer to the EGLStream will fail and generate an EGL_BAD_MATCH error. Refer to sections 3.10.3.1 and following for different ways to connect a producer to an EGLStream. Once an EGLStream is connected to a producer it will remain connected to the same producer until the EGLStream is destroyed. If the producer is destroyed then the EGLStream's state will become EGL_STREAM_STATE_DISCONNECTED_KHR (refer to "3.10.4.3 EGL_STREAM_STATE_KHR Attribute"). Any attempt to connect an EGLStream which is not in state EGL_STREAM_STATE_CONNECTING_KHR will fail and generate an EGL_BAD_STATE_KHR error. When an EGLStream is connected to a producer its state becomes EGL_STREAM_STATE_EMPTY_KHR. At this point the producer may begin inserting image frames and the consumer may begin consuming image frames, so the state may immediately change to EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR and/or EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR. 3.10.3.1 No way to connect producer to EGLStream EGL does not currently define any mechanisms to connect a producer to an EGLStream. These will be added via additional extensions. (For example see extension specifications EGL_KHR_stream_producer_eglsurface EGL_KHR_stream_producer_aldatalocator OpenMAX_AL_EGLStream_DataLocator .) Add section 3.10.4 to section "3.10 EGLStreams" 3.10.4 EGLStream Attributes Each EGLStream contains a set of attributes and values as described in table 3.10.4.4. Each attribute has a type and a value and is either read-only (ro), read/write (rw) or initialize only (io - meaning it may be set in the attrib_list but not changed once the EGLStream is created). Attribute Read/Write Type Section -------------------------- ---------- ------ -------- EGL_STREAM_STATE_KHR ro EGLint 3.10.4.3 EGL_PRODUCER_FRAME_KHR ro EGLuint64KHR 3.10.4.4 EGL_CONSUMER_FRAME_KHR ro EGLuint64KHR 3.10.4.5 EGL_CONSUMER_LATENCY_USEC_KHR rw EGLint 3.10.4.6 Table 3.10.4.4 EGLStream Attributes 3.10.4.1 Setting EGLStream Attributes Call EGLBoolean eglStreamAttribKHR( EGLDisplay dpy, EGLStreamKHR stream, EGLint attribute, EGLint value); to set the value of an attribute for an EGLStream. The is the new value for . Only read/write (rw) attributes with type EGLint may be set with eglStreamAttribKHR (see "Table 3.10.4.4 EGLStream Attributes"). If an error occurs, EGL_FALSE is returned and an error is generated. - EGL_BAD_STATE_KHR is generated if is in EGL_STREAM_STATE_DISCONNECTED_KHR state. - EGL_BAD_ATTRIBUTE is generated if is not a valid EGLStream attribute. - EGL_BAD_ACCESS is generated if is read only. - EGL_BAD_PARAMETER is generated if value is outside the valid range for . - EGL_BAD_STREAM_KHR is generated if is not a valid EGLStream created for . - EGL_BAD_DISPLAY is generated if is not a valid, initialized EGLDisplay. 3.10.4.2 Querying EGLStream Attributes Call EGLBoolean eglQueryStreamKHR( EGLDisplay dpy, EGLStreamKHR stream, EGLint attribute, EGLint *value); to query the value of an EGLStream's attribute with type EGLint and call EGLBoolean eglQueryStreamu64KHR( EGLDisplay dpy, EGLStreamKHR stream, EGLenum attribute, EGLuint64KHR *value); to query the value of an EGLStream's attribute with type EGLuint64KHR. If an error occurs EGL_FALSE is returned and an error is generated. - EGL_BAD_STREAM_KHR is generated if is not a valid EGLStream created for . - EGL_BAD_ATTRIBUTE is generated by eglQueryStreamKHR if is not a valid EGLStream attribute with type EGLint. - EGL_BAD_ATTRIBUTE is generated by eglQueryStreamu64KHR if is not a valid EGLStream attribute with type EGLuint64KHR. 3.10.4.3 EGL_STREAM_STATE_KHR Attribute The EGL_STREAM_STATE_KHR attribute is read only. It indicates the state of the EGLStream. The EGLStream may be in one of the following states: - EGL_STREAM_STATE_CREATED_KHR - The EGLStream has been created but not yet connected to a producer or a consumer. - EGL_STREAM_STATE_CONNECTING_KHR - The EGLStream has been connected to a consumer but not yet connected to a producer. - EGL_STREAM_STATE_EMPTY_KHR - the EGLStream has been connected to a consumer and a producer, but the producer has not yet inserted any image frames. - EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR - the producer has inserted at least one image frame that the consumer has not yet retrieved. - EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR - the producer has inserted at least one image frame, and the consumer has already retrieved the most recently inserted image frame. - EGL_STREAM_STATE_DISCONNECTED_KHR - either the producer or the consumer (or both) are no longer connected to the EGLStream (e.g. because they have been destroyed). Once the EGLStream is in this state it will remain in this state until the EGLStream is destroyed. In this state only eglQueryStreamKHR and eglDestroyStreamKHR are valid operations. Only the following state transitions may occur: -> EGL_STREAM_STATE_CREATED_KHR A new EGLStream is created in this state. EGL_STREAM_STATE_CREATED_KHR -> EGL_STREAM_STATE_CONNECTING_KHR Occurs when a consumer is connected to the EGLStream. EGL_STREAM_STATE_CONNECTING_KHR -> EGL_STREAM_STATE_EMPTY_KHR Occurs when a producer is connected to the EGLStream. EGL_STREAM_STATE_EMPTY_KHR -> EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR Occurs the first time the producer inserts an image frame. EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR -> EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR Occurs when the consumer begins examining a newly inserted image frame. EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR -> EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR Occurs when the producer inserts a new image frame. * -> EGL_STREAM_STATE_DISCONNECTED_KHR Occurs when the producer or consumer is destroyed or is otherwise unable to function normally. 3.10.4.4 EGL_PRODUCER_FRAME_KHR Attribute The EGL_PRODUCER_FRAME_KHR attribute indicates how many image frames have been inserted into the EGLStream by the producer. This is also known as the "frame number" of the most recently inserted frame (where the first frame inserted has a frame number of 1). When EGL_STREAM_STATE_KHR is EGL_STREAM_STATE_CREATED_KHR, EGL_STREAM_STATE_CONNECTING_KHR, or EGL_STREAM_STATE_EMPTY_KHR then this value is 0. This value will wrap back to 0 after about 10 million millennia. 3.10.4.4 EGL_CONSUMER_FRAME_KHR Attribute The EGL_CONSUMER_FRAME_KHR attribute indicates the frame number of the image frame that the consumer most recently retrieved. This is the value that EGL_PRODUCER_FRAME_KHR contained just after this image frame was inserted into the EGLStream. 3.10.4.5 EGL_CONSUMER_LATENCY_USEC_KHR Attribute This attribute indicates the number of microseconds that elapse (on average) from the time that an image frame is inserted into the EGLStream by the producer until the image frame is visible to the user. It is the responsibility of the consumer to set this value. Some types of consumers may simply set this value to zero or an implementation constant value. Other consumers may adjust this value dynamically as conditions change. It is the responsibility of the producer to use this information to insert image frames into the EGLStream at an appropriate time. The producer should insert each image frame into the stream at the time that frame should appear to the user MINUS the EGL_CONSUMER_LATENCY_USEC_KHR value. Some types of producers may ignore this value. The application may modify this value to adjust the timing of the stream (e.g. to make video frames coincide with an audio track under direction from a user). However the value set by the application may be overridden by some consumers that dynamically adjust the value. This will be noted in the description of consumers which do this. If EGL_KHR_stream_attrib is present, add to the end of section "3.10.4.1 Setting EGLStream Attributes" Attributes may also be set by calling EGLBoolean eglSetStreamAttribKHR( EGLDisplay dpy, EGLStreamKHR stream, EGLenum attribute, EGLAttrib value); This is equivalent to eglStreamAttribKHR, but allows attributes with pointer and handle types, in addition to EGLint. If EGL_KHR_stream_attrib is present, add to the end of section "3.10.4.2 Querying EGLStream Attributes" Attributes may also be queried by calling EGLBoolean eglQueryStreamAttribKHR( EGLDisplay dpy, EGLStreamKHR stream, EGLenum attribute, EGLAttrib *value); This is equivalent to eglQueryStreamKHR, but allows attributes with pointer and handle types, in addition to EGLint. Add sections 3.10.5 and 3.10.6 to section "3.10 EGLStreams" 3.10.5 EGLStream operation 3.10.5.1 EGLStream operation in mailbox mode The EGLStream conceptually operates as a mailbox. When the producer has a new image frame it empties the mailbox and inserts the new image frame into the mailbox. If the image frame is intended to be displayed at time T then the producer must insert it into the EGLStream at time T - EGL_CONSUMER_LATENCY_USEC_KHR The consumer retrieves the image frame from the mailbox and examines it. When the consumer is finished examining the image frame it is either placed back in the mailbox (if the mailbox is empty) or discarded (if the mailbox is not empty). This operation implies 2 things: - If the consumer consumes frames slower than the producer inserts frames, then some frames may be lost (never seen by the consumer). - If the consumer consumes frames faster than the producer inserts frames, then the consumer may see some frames more than once. Some details of EGLStream operation are dependent on the type of producer and consumer that are connected to it. Refer to the documentation for the producer and consumer for more details (section 3.10.2.* and 3.10.3.*). 3.10.6 Destroying an EGLStream Call EGLBoolean eglDestroyStreamKHR( EGLDisplay dpy, EGLStreamKHR stream); to mark an EGLStream for deletion. After this call returns the will no longer be a valid stream handle. The resources associated with the EGLStream may not be deleted until the producer and consumer have released their references to the resources (if any). Exactly how this is done is dependent on the type of consumer and producer that is connected to the EGLStream. If an error occurs, EGL_FALSE is returned and an error is generated. - EGL_BAD_STREAM_KHR is generated if is not a valid EGLStream created for . Issues 1. Are EGL_WIDTH and EGL_HEIGHT parameters needed? RESOLVED: No. The width and height of images managed by the stream are determined by the producer. No application access to the size is currently required. 2. Is EGL_BUFFER_SHOW_ALL_KHR required, or should the stream always act as EGL_BUFFER_REPLACE_KHR? RESOLVED: this has been removed. The old EGL_BUFFER_SHOW_ALL_KHR behavior is described in a separate extension: EGL_KHR_stream_fifo 3. What are the exact semantics of the producer? RESOLVED: The exact semantics vary depending on the type of producer. Refer to the extension that defines the type of producer for more information. In general, the producer is responsible for inserting image frames into the EGLStream at the correct time. The correct time depends on how the image frames are being created and on the value of EGL_CONSUMER_LATENCY_USEC_KHR. 4. What are the exact semantics of the consumer? RESOLVED: The exact semantics vary depending on the type of consumer. Refer to the extension that defines the type of consumer for more information. In general, the consumer is responsible for retrieving image frames from the EGLStream when they become available. The consumer is also responsible for setting the EGL_CONSUMER_LATENCY_USEC_KHR when that is possible. 5. When will the EGLStream resources be deleted? RESOLVED: this depends on the type of consumer and producer. Refer to the description of the consumer and producer (e.g. in the extension that describes them). 6. How does A/V sync work? RESOLVED: The producer is responsible for A/V sync, but the consumer needs to help. The consumer indicates the latency (the average time that it takes the consumer to retrieve an image from the EGLStream and place it on the display screen) by setting the EGL_CONSUMER_LATENCY_USEC_KHR. The producer uses knowledge about the audio stream to determine the correct time to display an image frame, and inserts the image frame at that time MINUS the EGL_CONSUMER_LATENCY_USEC_KHR. 7. What if the consumer cannot determine the latency? RESOLVED: If the consumer does not set the EGL_CONSUMER_LATENCY_USEC_KHR attribute then its default value will be used. This default value is implementation defined and may be zero. See the description of the specific type of consumer you are using (e.g. the extension that defines it) for more details related to that consumer. 8. What colorformats are supported by EGLStream RESOLVED: No specific formats are required, but it is expected that this work with the main YUV formats supported by the platform's video HW and the main RGB(A) formats supported by the platform's OpenGL (ES) hardware. It is the responsibility of the producer to negotiate a format that will work with the consumer. If the internal formats supported by the producer do not coincide with the internal formats supported by the consumer then the producer may choose to convert to a format that the consumer understands, or it may choose to fail and generate an error when an attempt is made to connect it to the EGLStream. Exactly which it does for which formats is further discussed in the producer endpoint documentation (refer to the extension that describes the producer endpoint). 9. Is any EGLImage extension required by this extension? RESOLVED: No. This extension may be implemented using some of the same code that is used to implement EGLImages, but there is no dependency on EGLImages. 10. Why describe the "io" attribute type if no attributes use it. RESOLVED: Future extensions will add attributes of "io" type (initialize only - meaning they can be set in the attribute list when creating the EGLStream, but not modified once the EGLStream is created). Rather than requiring each such extension to describe the "io" type (and possibly getting slightly different definitions or types in different extensions) the "io" type is defined here so that other extensions can easily use it. This helps layered extensions to all use the same language. Revision History #27 (May 23, 2016) Daniel Kartch - For compatibility with EGL 1.5 and support of 64-bit platforms, add EGL_KHR_stream_attrib extension with variants of original functions that accept attributes of type EGLAttrib. - Corrected line length violations. #26 (July 12, 2012) Acorn Pooley - Fix error in description of consumer latency. #25 (October 12, 2011) Acorn Pooley - Add issue 10 #24 (October 11, 2011) Acorn Pooley - add error condition to eglDestroyStreamKHR #23 (October 5, 2011) Acorn Pooley - refer to related EGL_KHR_... extension specs rather than EGL_NV_... ones. #22 (September 27, 2011) Acorn Pooley - Fix enum value for EGL_STREAM_STATE_KHR (bug 8064) #21 (September 27, 2011) Acorn Pooley - Assign enum values (bug 8064) #20 (September 23, 2011) Acorn Pooley - Rename EGL_NO_IMAGE_STREAM_KHR to EGL_NO_STREAM_KHR #19 (Aug 3, 2011) Acorn Pooley - fix some error conditions #18 (Aug 2, 2011) Acorn Pooley - Add eglQueryStreamu64KHR - add EGLuint64KHR - make EGL_PRODUCER_FRAME_KHR and EGL_CONSUMER_FRAME_KHR 64 bit. #17 (Aug 2, 2011) Acorn Pooley - fix grammar #16 (July 6, 2011) Acorn Pooley - rename from EGL_KHR_image_stream to EGL_KHR_stream #15 (June 29, 2011) Acorn Pooley - major re-write - remove EGL_SWAP_MODE_KHR and EGL_BUFFER_SHOW_ALL_KHR - add new functions: eglStreamAttribKHR eglQueryStreamKHR - add new attributes: EGL_CONSUMER_LATENCY_USEC_KHR EGL_PRODUCER_FRAME_KHR EGL_CONSUMER_FRAME_KHR EGL_STREAM_STATE_KHR - add concept of EGL_STREAM_STATE_KHR - add new error: EGL_BAD_STATE_KHR - add more thorough overview section - add description of buffering - place the functions in section 3 of the spec (were in section 2) - mention some of the consumer and producer specs that may be needed to make use of this extension. - remove very old issues that no longer make any sense - add new issues and resolutions #14 (June 4, 2010) Greg Prisament - fix minor typo #13 (June 2, 2010) Marcus Lorentzon - add EGL enum values #12 (May 21, 2010) Marcus Lorentzon - add clarifications on swap modes #11 (April 13, 2010) Marcus Lorentzon - fix tyops - make eglDestroyStream return EGLBoolean, not void #10 (March 17, 2010) Marcus Lorentzon - fix typo - remove obsolete text - update issue 2 resolution #9 (December 15, 2009) Marcus Lorentzon - move EGL_IMAGE_USE_* attributes to the endpoint extension - resolved issue 5 #8 (December 6, 2009) Marcus Lorentzon - remove EGL_INIT_COLOR_KHR - relax the definition of the Producer to allow not only video frames to be generated - clean up the language of recently produced, supplied, pending images #7 (October 19, 2009) Acorn Pooley - Update based on comments from Robert and Bruce - remove mention of OpenWF - make EGL_BUFFER_REPLACE_KHR be the default EGL_SWAP_MODE_KHR - add issue 5 - remove EGLAPI and EGLAPIENTRY #6 (September 16, 2009) Acorn Pooley - remove EGL_WIDTH and EGL_HEIGHT parameters - add issue 4 - clarify swap modes - other clarifications and simplifications #5 (July 2, 2009) Acorn Pooley - remove reference to no-longer-existing parameter. - mention dependancy on EGL_KHR_image_uses extension. - add description of EGL_IMAGE_USE_AS_* enums. #4 (June 3, 2009) Acorn Pooley - Fix typos: change old EGLImageStream occurances to EGLStream #3 (April 22, 2009) Marcus Lorentzon - Updated revide comments - Removed external image support #2 (March 30, 2009) Marcus Lorentzon - Replaced image surface with image stream #1 (February 21, 2009) Marcus Lorentzon - Initial draft # vim:ai:ts=4:sts=4:expandtab:textwidth=70