Name KHR_stream_fifo Name Strings EGL_KHR_stream_fifo Contributors Acorn Pooley Contacts Acorn Pooley, NVIDIA (apooley 'at' nvidia.com) Notice Copyright (c) 2011-2013 The Khronos Group Inc. Copyright terms at http://www.khronos.org/registry/speccopyright.html Status Complete. Approved by the Khronos Board of Promoters on December 2, 2011. Version Version 6, October 12, 2011 Number EGL Extension #36 Dependencies Requires EGL 1.2. Requires the EGL_KHR_stream extension. This extension is written based on the wording of the EGL 1.2 specification. The EGL_KHR_stream_producer_eglsurface and EGL_NV_stream_producer_eglsurface extensions affect the wording of this extension. The EGL_KHR_stream_producer_aldatalocator and EGL_NV_stream_producer_aldatalocator extensions affect the wording of this extension. The EGL_KHR_stream_consumer_gltexture and EGL_NV_stream_consumer_gltexture extensions affect the wording of this extension. Overview This extension allows an EGLStream to operate as a fifo rather than as a mailbox. The EGL_KHR_stream extension defines the EGLStream object. The EGLStream object works like a 1 entry mailbox, allowing the consumer to consume the frame that the producer most recently inserted. If the consumer requests image frames faster than the producer creates them then it gets the most recent one over and over until a new one is inserted. If the producer inserts frames faster than the consumer can consume them then the extra frames are discarded. The producer is never stalled. This extension allows an EGLStream to be placed into fifo mode. In fifo mode no images are discarded. If the producer attempts to insert a frame and the fifo is full then the producer will stall until there is room in the fifo. When the consumer retrieves an image frame from the EGLStream it will see the image frame that immediately follows the image frame that it last retrieved (unless no such frame has been inserted yet in which case it retrieves the same image frame that it retrieved last time). Timing of the EGLStream in mailbox mode, as described by the EGL_KHR_stream extension, is the responsibility of the producer (with help from the consumer in the form of the EGL_CONSUMER_LATENCY_USEC_KHR hint). In contrast, timing of an EGLStream in fifo mode is the responsibility of the consumer. Each image frame in the fifo has an associated timestamp set by the producer. The consumer can use this timestamp to determine when the image frame is intended to be displayed to the user. New Types This type represents an absolute time in nanoseconds. typedef khronos_utime_nanoseconds_t EGLTimeKHR New functions EGLBoolean eglQueryStreamTimeKHR( EGLDisplay dpy, EGLStreamKHR stream, EGLenum attribute, EGLTimeKHR *value); New Tokens Accepted as an attribute in the parameter of eglCreateStreamKHR and as the parameter of eglQueryStreamKHR. EGL_STREAM_FIFO_LENGTH_KHR 0x31FC These enums are accepted the parameter of eglQueryStreamTimeKHR. EGL_STREAM_TIME_NOW_KHR 0x31FD EGL_STREAM_TIME_CONSUMER_KHR 0x31FE EGL_STREAM_TIME_PRODUCER_KHR 0x31FF Add 4 new entries to "Table 3.10.4.4 EGLStream Attributes" in the EGL_KHR_stream extension spec: Attribute Read/Write Type Section -------------------------- ---------- ---------- -------- EGL_STREAM_FIFO_LENGTH_KHR io EGLint 3.10.4.xx EGL_STREAM_TIME_NOW_KHR ro EGLTimeKHR 3.10.4.xx EGL_STREAM_TIME_CONSUMER_KHR ro EGLTimeKHR 3.10.4.xx EGL_STREAM_TIME_PRODUCER_KHR ro EGLTimeKHR 3.10.4.xx Add a new paragraph to the end of section "3.10.4.2 Querying EGLStream Attributes" in the EGL_KHR_stream extension. Call EGLBoolean eglQueryStreamTimeKHR( EGLDisplay dpy, EGLStreamKHR stream, EGLenum attribute, EGLTimeKHR *value); to query from for attributes whose type is EGLTimeKHR. If an error occurs EGL_FALSE is returned and an error is generated. - EGL_BAD_STREAM_KHR is generated if is not a valid EGLStream created for . - EGL_BAD_ATTRIBUTE is generated if is not a valid EGLStream attribute with type EGLTimeKHR. Add new sections 3.1.4.xx at the end of section "3.10.4 EGLStream Attributes" in the EGL_KHR_stream extension. 3.1.4.x EGL_STREAM_FIFO_LENGTH_KHR Attribute The EGL_STREAM_FIFO_LENGTH_KHR may be set in the parameter of eglCreateStreamKHR(), but is read-only once the stream is created. It can be queried with eglQueryStreamKHR(). Its default value is 0. Setting it to a value less than 0 generates an EGL_BAD_PARAMETER error. When EGL_STREAM_FIFO_LENGTH_KHR is 0 the EGLStream operates in mailbox mode as described in section "3.10.5.1 EGLStream operation in mailbox mode" When EGL_STREAM_FIFO_LENGTH_KHR is greater than 0 then the EGLStream operates in fifo mode as described in section "3.10.5.2 EGLStream operation in fifo mode". In fifo mode the EGLStream contains up to N image frames, where N is the value of the EGL_STREAM_FIFO_LENGTH_KHR attribute. The value of EGL_STREAM_FIFO_LENGTH_KHR is independent from the number of internal buffers used by the producer. The producer may require some number of internal buffers, but those are in addition to the fifo buffers described by EGL_STREAM_FIFO_LENGTH_KHR. 3.1.4.x+1 EGL_STREAM_TIME_NOW_KHR Attribute This indicates the current time. It is measured as the number of nanoseconds since some arbitrary event (e.g. the last time the system rebooted). 3.1.4.x+2 EGL_STREAM_TIME_CONSUMER_KHR Attribute This indicates the timestamp of the image frame that the consumer most recently consumed (i.e. frame number EGL_CONSUMER_FRAME_KHR). The frame should first be displayed to the user when EGL_STREAM_TIME_NOW_KHR matches this value. In mailbox mode the timestamp for an image frame is always equal to the time that the producer inserted the image frame into the EGLStream, minus the value of EGL_CONSUMER_LATENCY_USEC_KHR. In fifo mode the timestamp for an image frame is set by the producer when it is inserted into the EGLStream. The timestamp uses the same time units as EGL_STREAM_TIME_NOW_KHR. 3.1.4.x+3 EGL_STREAM_TIME_PRODUCER_KHR Attribute This indicates the timestamp of the image frame that the producer most recently inserted into the EGLStream (i.e. frame number EGL_PRODUCER_FRAME_KHR). Modify the first sentence of section "3.10.5.1 EGLStream operation in mailbox mode" in the EGL_KHR_stream extension to: When the EGL_STREAM_FIFO_LENGTH_KHR attribute is 0 then the EGLStream conceptually operates as a mailbox. Add a new section after section "3.10.5.1 EGLStream operation in mailbox mode" in the EGL_KHR_stream extension. 3.10.5.2 EGLStream operation in fifo mode When the EGL_STREAM_FIFO_LENGTH_KHR attribute is greater than 0 then the EGLStream operates in fifo mode. The length of the fifo is the value of the EGL_STREAM_FIFO_LENGTH_KHR attribute. In fifo mode the EGLStream conceptually operates as a fifo. When the consumer wants to consume a new image frame, behavior depends on the state of the EGLStream. If the state is EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR then the fifo is not empty and the image frame to consume is removed from the tail of the fifo. If the state is EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR then the fifo is empty and the consumer consumes the same frame that it most recently consumed. Otherwise there are no image frames available to consume (behavior in this case is described in the documentation for each type of consumer - see section "3.10.2 Connecting an EGLStream to a consumer"). If the fifo is empty when the consumer is finished consuming an image frame then the consumer holds on to the image frame in case it needs to be consumed again later (this happens if the consumer wants to consume another image frame before the producer has inserted a new image frame into the fifo). In this case the state of the EGLStream will be EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR until the producer inserts a new image frame (or until the state becomes EGL_STREAM_STATE_DISCONNECTED_KHR). The producer inserts image frames at the head of the fifo. If the fifo is full (already contains image frames, where is the value of the EGL_STREAM_FIFO_LENGTH_KHR attribute) then producer is stalled until the fifo is no longer full. When the fifo is not empty the EGLStream state is EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR. This operation implies: - Frames are never discarded until the consumer has examined them. - If the consumer consumes frames slower than the producer inserts frames, then the producer will stall. - If the consumer consumes frames faster than the producer inserts frames, then the consumer may see some frames more than once. - The consumer can see each frame exactly once if it always waits until the stream is in the EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR state before retrieving an image from the stream. In mailbox mode the producer is responsible for timing. In fifo mode the consumer is responsible for timing. In fifo mode the producer marks each image frame with a timestamp. The timestamp indicates at what time the image frame should first be visible to the user. Exactly how a producer sets the timestamp is described in the documentation for each type of producer. If the value of an image frame's timestamp is T then the producer must insert that image frame *before* time T - EGL_CONSUMER_LATENCY_USEC_KHR Image frames must be inserted in increasing timestamp order. The consumer is responsible for presenting each image frame to the user at the time indicated by its timestamp. The consumer should indicate its minimum latency to the producer by setting the EGL_CONSUMER_LATENCY_USEC_KHR attribute. If the EGL_KHR_stream_producer_eglsurface or EGL_NV_stream_producer_eglsurface extension is present then add a paragraph to the end of section "3.10.3.1 Stream Surface Producer" from that extension: If 's EGL_STREAM_FIFO_LENGTH_KHR value is nonzero then operates in fifo mode. Each time the EGLSurface is passed to eglSwapBuffers() an image frame is inserted into the fifo. The eglSwapBuffers call sets the timestamp of the image frame to the time that eglSwapBuffers was called PLUS the value of the EGL_CONSUMER_LATENCY_USEC_KHR attribute. If the EGL_KHR_stream_producer_eglsurface or EGL_NV_stream_producer_eglsurface extension is present then add a paragraph to section "3.9.x Posting to a Stream" from that extension, between the 2nd paragraph (which begins "If is the producer of an EGLStream...") and the 3rd paragraph (which begins "When eglSwapBuffers returns the contents..."): If the value of the EGL_STREAM_FIFO_LENGTH_KHR attribute, is greater than zero, and there are already image frames in the EGLStream fifo, then the eglSwapBuffers function blocks (does not return and does not insert the new image frame) until there is room in the EGLStream fifo (i.e. there are less than image frames in the fifo). If the EGL_KHR_stream_producer_aldatalocator or EGL_NV_stream_producer_aldatalocator extension is present then replace the 2nd to last paragraph (the one that starts "The OpenMAX AL object will use the value of...") of section "3.10.3.1 OpenMAX AL Stream Producer" from that extension with the following 2 paragraphs: If 's EGL_STREAM_FIFO_LENGTH_KHR value is zero then the stream operates in mailbox mode. The OpenMAX AL object will use the value of the EGL_CONSUMER_LATENCY_USEC_KHR attribute of to determine when to insert each image frame. If the EGL_CONSUMER_LATENCY_USEC_KHR attribute is modified (by the consumer and/or by the application) then then OpenMAX AL object will adjust its timing within 500 milliseconds of the change. If an image frame is intended to appear to the user at time T (e.g. so that it is synchronized with audio) then the OpenMAX AL object must insert the image frame at time T - EGL_CONSUMER_LATENCY_USEC_KHR and set the image frame's timestamp to T. If the 's EGL_STREAM_FIFO_LENGTH_KHR value is nonzero then operates in fifo mode. If an image frame is intended to appear to the user at time T then the OpenMAX AL object will insert the image frame into the fifo before time T - EGL_CONSUMER_LATENCY_USEC_KHR and set the image frame's timestamp to T. If the EGL_KHR_stream_consumer_gltexture or EGL_NV_stream_consumer_gltexture extension is present then replace the 3rd to last paragraph (the one that starts "If the producer has not inserted any new image frames...") of section "3.10.2.1 GL Texture External consumer" from that extension with the following 2 paragraphs: When 's EGL_STREAM_FIFO_LENGTH_KHR value is zero then the stream operates in mailbox mode. If the producer has not inserted any new image frames since the last call to eglStreamConsumerAcquireNV then eglStreamConsumerAcquireNV will "latch" the same image frame it latched last time eglStreamConsumerAcquireNV was called. If the producer has inserted one new image frame since the last call to eglStreamConsumerAcquireNV then the eglStreamConsumerAcquireNV will "latch" the newly inserted image frame. If the producer has inserted more than one new image frame since the last call to eglStreamConsumerAcquireNV then all but the most recently inserted image frames are discarded and the producer will "latch" the most recently inserted image frame. When 's EGL_STREAM_FIFO_LENGTH_KHR value is nonzero then operates in fifo mode. Each call to eglStreamConsumerAcquireNV "latches" the next image frame in the fifo into the OpenGL texture, removing that image frame from the fifo. If there are no new image frames in the fifo then eglStreamConsumerAcquireNV will "latch" the same image frame it latched last time eglStreamConsumerAcquireNV was called. Issues 1. Is this extension useful? RESOLVED: Yes. Browser vendors and others have expressed interest. 2. Why not include this functionality in the base EGL_KHR_stream extension? RESOLVED: Including it there was confusing. Several developers interested in EGLStream have thought at first that they want to use EGLStreams in fifo mode. Later after thinking about it they realize standard mode (non-fifo or "mailbox" mode) is more useful. Mailbox mode is easier to use and is less confusing for aldatalocator-producer, gltexture-consumer usecase which was the primary usecase for the extension at the time it was devised. Trying to describe both mailbox mode and fifo mode in the same extension made the extension complicated. It was confusing when the timestamps were useful (only in fifo mode). It was confusing how the EGL_CONSUMER_LATENCY_USEC_KHR attribute worked in different modes. these problems the fifo functionality was split into this separate extension. This also allows existing consumer and producer extensions to be defined in terms of mailbox mode, simplifying them and making them easier to understand. Then interactions with fifo mode can be described separately. Also, the fifo mode is more complicated to use and implement than the mailbox mode. It was thought that there might be problems with the fifo mode that could lead to a new extension replacing the fifo mode extension. By keeping the fifo mode functionality segregated into its own extension this would be easier to accomplish. Revision History #6 (October 12, 2011) Acorn Pooley - Clarify fifo mode operation. (Does not change behavior.) #5 (October 11, 2011) Acorn Pooley - Resolve issue 1 - fix typos - add issue 2 #4 (September 27, 2011) Acorn Pooley - Assign enum values (bug 8064) #3 (July 6, 2011) Acorn Pooley - Rename EGL_KHR_image_stream to EGL_KHR_stream #2 (version #2 skipped) #1 (July 1, 2011) Acorn Pooley - Initial draft # vim:ai:ts=4:sts=4:expandtab:textwidth=70