In bulk data, each element is interpreted first by addressing it in some form, then by interpreting the addressed values. Texels often represent a color (or other data) as a multi-dimensional set of values, each representing a channel. The bulk-data image or buffer then describes a number of these texels. Taking the simplest case of an array in the C programming language as an example, a developer might define the following structure to represent a color texel:

typedef struct _MyRGB {
  unsigned char red;
  unsigned char green;
  unsigned char blue;
} MyRGB;

MyRGB *myRGBarray = (MyRGB *) malloc(100 * sizeof(MyRGB));

To determine the location of, for example, the tenth element of myRGBarray, the compiler needs to know the base address of the array and sizeof myRGB. Extracting the red, green and blue components of myRGBarray[9] given its base address is, in a sense, orthogonal to finding the base address of myRGBarray[9].

Note also that sizeof(MyRGB) will often exceed the total size of red, green and blue due to padding; the difference in address between one MyRGB and the next can be described as the pixel stride in bytes.

Figure 2. (Trivial) 1D address offsets for 1-byte elements, start of buffer

Figure 3. 1D address offsets for 2-byte elements, start of buffer

Figure 4. 1D address offsets for R,G,B elements (padding in gray), start of buffer

An alternative representation is a “structure of arrays”, distinct from the “array of structures” myRGBarray:

typedef struct _MyRGBSoA {
  unsigned char *red;
  unsigned char *green;
  unsigned char *blue;
} MyRGBSoA;

MyRGBSoA myRGBSoA;
myRGBSoA.red = (unsigned char *) malloc(100);
myRGBSoA.green = (unsigned char *) malloc(100);
myRGBSoA.blue = (unsigned char *) malloc(100);

In this case, accessing a value requires the sizeof each channel element. The best approach depends on the operations performed: calculations on one whole MyRGB a time likely favor MyRGB, those processing multiple values from a single channel may prefer MyRGBSoA. A “pixel” need not fill an entire byte — nor need pixel stride be a whole number of bytes. For example, a C++ std::vector<bool> can be considered to be a 1-D bulk data structure of individual bits.

The simplest way to represent two-dimensional data is in consecutive rows, each representing a one-dimensional array — as with a 2-D array in C. There may be padding after each row to achieve the required alignment: in some cases each row should begin at the start of a cache line, or rows may be deliberately offset to different cache lines to ensure that vertically-adjacent values can be cached concurrently. The offset from the start of one horizontal row to the next is a line stride or row stride (or just stride for short), and is necessarily at least the width of the row. If each row holds an whole number of pixels, row stride can be described either in bytes or pixels; it is rare not to start each row on a byte boundary. In a simple 2-D representation, the row stride and the offset from the start of the storage can be described as follows:

Figure 5 shows example coordinate byte offsets for a 13×4 buffer, padding row stride to a multiple of four elements.

Figure 5. 2D linear texel offsets from coordinates

Figure 6. 2D R,G,B byte offsets (padding in gray) from coordinates for a 4×4 image

By convention, this “linear” layout is in y-major order (with the x axis changing most quickly). This does not necessarily imply that an API which accesses the 2D data will do so in this orientation, and there are image layouts which have x and y swapped. The direction of the axes (particularly y) in relation to the image orientation also varies between representations.

Each contiguous region described in this way can be considered as a plane of data. In the same way that 1-D data can be described in structure-of-arrays form, two-dimensional data can also be implemented as multiple planes.

Figure 7. 2D R,G,B plane offsets from coordinates for 8×4 texels

In early graphics, planes often contributed individual bits to the pixel. This allowed pixels of less than a byte, reducing bandwidth and storage; combining 1-bit planes allowed, for example, 64-color, 6-bit pixels. Storing six bits per pixel consecutively would make the number of memory bytes holding a pixel vary by alignment; a planar layout needs simpler hardware. This hardware could often process a subset of the planes — scrolling text may require blitting just one plane, and changes to the display controller start position for each plane independently provide cheap parallax effects.

In modern architectures, planes are typically byte-aligned and hold separate channels. One motivation is avoiding padding. Some common hardware can only access powers of two bytes for each pixel: for example, texels of three 1-byte channels may need padding to four bytes for alignment restrictions; separate planes of one byte per texel need no padding. Other benefits are supporting different downsampling of each channel (described below), and more efficient independent processing of each channel: many common operations (such as filtering and image compression) treat channels separately.

Note that, once all the data from any contributing channels have been combined, the interpretation of the resulting values is again orthogonal to addressing the data.

In many applications, the simple “linear” layout of memory has disadvantages. The primary concern is that data which is vertically adjacent in two-dimensional space may be widely separated in memory address. Many computer graphics rendering operations involve accessing the frame buffer in an order not aligned with its x-axis; for example, traversing horizontally across the width of a textured triangle while writing to the frame buffer will result in a texture access pattern which depends on the orientation of the texture relative to the triangle and the triangle relative to the frame buffer. If each texel access processed a different cache line, the resulting performance would be heavily compromised. Modern GPUs process multiple texels in parallel, meaning that many nearby texels in different orientations may need to be accessed quickly. Additionally, texture filtering operations typically read a 2×2 quad of texels, inherently requiring access to texels which would be distant in memory in the linear representation.

One solution to this is to divide the image into smaller rectangular tiles of texels (of tw × th) rather than horizontal rows. The texel ordering within the tiles may be treated as though each tile were an independent, smaller image, and the order of tiles in memory may be as in the linear layout:

This approach can be implemented efficiently in hardware, so long as the tile dimensions are powers of two, by interleaving some bits from the y-coordinate with the bits contributed by the x-coordinate. If twb = log₂(tw) and thb = log₂(th):

pixelOffset = (x & ((1 << twb) - 1)) | ((y & ((1 << thb) - 1)) << twb)
            | ((x & ~((1 << twb) - 1)) << thb) | ((y & ~((1 << thb)-1)) * lineStride);

For example, if a linear 16×12 image calculates a pixel offset relative to the start of the buffer as:

pixelOffset = x | (y * 16);

a 16×12 image comprised of 4×4 tiles may calculate the pixel offset relative to the start of the buffer as:

pixelOffset = (x & 3) | ((y & 3) << 2) | ((x & ~3) << 2) | ((y & ~3) * 16);

Figure 8. 4×4 tiled order texel offsets from coordinates (consecutive values linked in red to show the pattern)

Textures which have dimensions that are not a multiple of the tile size require padding.

For so long as the size of a tile fits into an on-chip cache and can be filled efficiently by the memory subsystem, this approach has the benefit that only one in $1\over n$ transitions between vertically-adjacent lines requires accessing outside a tile of height n. The larger the tile, the greater the probability that vertically-adjacent accesses fall inside the same scan line. However, horizontally-adjacent texels that cross a tile boundary are made more distant in memory the larger the tile size, and tile sizes which do not conveniently fit the cache and bus sizes of the memory subsystem have inefficiencies; thus the tile size is an architectural trade-off.

While any non-linear representation is typically referred to as “tiling”, some hardware implementations actually use a more complex layout in order to provide further locality of reference. One such scheme is Morton order:

Figure 9. Morton order texel offsets from coordinates (consecutive values linked in red)

uint32_t mortonOffset(uint32_t x, uint32_t y,
                      uint32_t width, uint32_t height) {

    const uint32_t minDim = (width <= height) ? width : height;
    uint32_t offset = 0, shift = 0, mask;

    // Tests xy bounds AND that width and height != 0
    assert(x < width && y < height);

    // Smaller size must be a power of 2
    assert((minDim & (minDim - 1)) == 0);

    // Larger size must be a multiple of the smaller
    assert(width % minDim == 0 && height % minDim == 0);

    for (mask = 1; mask < minDim; mask <<= 1) {
        offset |= (((y & mask) << 1) | (x & mask)) << shift;
        shift++;
    }

    // At least one of width and height will have run out of most-significant bits
    offset |= ((x | y) >> shift) << (shift * 2);
    return offset;
}

Note that this particular implementation assumes that the smaller of width and height is a power of two, and the larger is a multiple of the smaller. A practical hardware implementation of the calculation is likely much more efficient than this C code would imply.

Another approach, with even more correlation between locality in x,y space and memory offset, is Hilbert curve order:

Figure 10. Hilbert curve order texel offsets from coordinates (consecutive values linked in red)

uint32_t h(uint32_t size, uint32_t x, uint32_t y) {
    const uint32_t z = x ^ y;
    uint32_t offset = 0;

    while (size >>= 1) { // Iterate in decreasing block size order
        // Accumulate preceding blocks of size * size
        offset += size * (((size & x) << 1) + (size & z));
        y = z ^ x; // Transpose x and y
        if (!(size & y)) x = (size & x) ? ~y : y; // Conditionally swap/mirror
    }
    return offset;
}

uint32_t hilbertOffset(uint32_t x, uint32_t y, uint32_t width, uint32_t height) {
    const uint32_t minDim = (width <= height) ? width : height;

    // Tests xy bounds AND that width and height != 0
    assert(x < width && y < height);

    // Smaller size must be a power of 2
    assert((minDim & (minDim - 1)) == 0);

    // Larger size must be a multiple of the smaller
    assert(width % minDim == 0 && height % minDim == 0);

    if (width < height) return (width * width) * (y / width) + h(width, y % width, x);
    else return (height * height) * (x / height) + h(height, x % height, y);
}

This code assumes that the smaller of width and height is a power of two, with the larger a multiple of the smaller.

Some implementations will mix these approaches — for example, having linear tiles arranged in Morton order, or Morton order texels within tiles which are themselves in linear order. Indeed, for non-square areas, the tiling, Morton and Hilbert orderings shown here can be considered as a linear row of square areas with edges of the shorter dimension.

In all these cases, the relationship between coordinates and the memory storage location of the buffer depends on the image size and, other than needing to know the amount of memory occupied by a single texel, is orthogonal to the “format”. Tiling schemes tend to be complemented by proprietary hardware that performs the coordinate-to-address mapping and depends on cache details, so many APIs expose only a linear layout to end-users, keeping the tiled representation opaque. The variety of possible mappings between coordinates and addresses mandates leaving the calculation to the application.

These storage approaches can be extended to additional dimensions, for example treating a 3-D image as being made up of 2-D “planes”, which are distinct from the concept of storing channels or bits of data in separate planes. In a linear layout, 3-dimensional textures can be accessed as:

Here, the plane stride is the offset between the start of one contiguous 2-D plane of data and the next. The plane stride is therefore height × row stride plus any additional padding.

Again, non-linear approaches can be used to increase the correlation between storage address and coordinates.

uint32_t tileOffset3D(uint32_t x, uint32_t y, uint32_t z,
                      uint32_t twb, uint32_t thb, uint32_t tdb,
                      uint32_t lineStride, uint32_t planeStride) {
  // twb = tile width bits (log2 of tile width)
  // thb = tile height bits (log2 of tile height)
  // tdb = tile depth bits (log2 of tile depth)
  return (x & ((1 << twb) - 1)) |
         ((y & ((1 << thb) - 1)) << twb) |
         ((z & ((1 << tdb) - 1)) << (twb + thb)) |
         ((x & ~((1 << twb) - 1)) << (thb + tdb)) |
         ((y & ~((1 << thb) - 1)) << tdb) * lineStride |
         ((z & ~((1 << tdb) - 1)) * planeStride);
}

uint32_t mortonOffset3D(uint32_t x, uint32_t y, uint32_t z,
                        uint32_t width, uint32_t height, uint32_t depth) {
    const uint32_t max = width | height | depth;
    uint32_t offset = 0, shift = 0, mask;
    for (mask = 1; max > mask; mask <<= 1) {
        if (width > mask)  offset |= (x & mask) << shift++;
        if (height > mask) offset |= (y & mask) << shift++;
        if (depth > mask)  offset |= (z & mask) << shift++;
        --shift;
    }
    return offset;
}

There are multiple 3D variations on the Hilbert curve; one such is this:

uint32_t hilbertOffset3D(uint32_t x, uint32_t y, uint32_t z, uint32_t size) {
    // "Harmonious" 3D Hilbert curve for cube of power-of-two edge "size":
    // http://herman.haverkort.net/recursive_tilings_and_space-filling_curves
    uint32_t offset = 0;
    while (size >>= 1) {
        uint32_t tx = (size & x) > 0, ty = (size & y) > 0, tz = (size & z) > 0;
        switch (tx + 2 * ty + 4 * tz) {
        case 0: tx =  z; ty =  y; tz =  x; break;
        case 1: tx =  x; ty =  z; tz =  y; offset += size * size * size; break;
        case 2: tx = ~y; ty = ~x; tz =  z; offset += size * size * size * 3; break;
        case 3: tx =  z; ty =  x; tz =  y; offset += size * size * size * 2; break;
        case 4: tx = ~z; ty =  y; tz = ~x; offset += size * size * size * 7; break;
        case 5: tx =  x; ty = ~z; tz = ~y; offset += size * size * size * 6; break;
        case 6: tx = ~y; ty = ~x; tz =  z; offset += size * size * size * 4; break;
        case 7: tx = ~z; ty =  x; tz = ~y; offset += size * size * size * 5; break;
        }
        x = tx; y = ty; z = tz;
    }
    return offset;
}

The examples provided so far have assumed that a unique value from each color channel is present at each access coordinate. However, some common representations break this assumption.

One reason for this variation comes from representing the image in the Y′C_BC_R color model, described in Section 15.1; in this description, the Y′ channel represents the intensity of light, with C_B and C_R channels describing how the color differs from a gray value of the same intensity in the blue and red axes. Since the human eye is more sensitive to high spatial frequencies in brightness than in the hue of a color, the C_B and C_R channels can be recorded at lower spatial resolution than the Y′ channel with little loss in visual quality, saving storage space and bandwidth.

In one common representation known colloquially as YUV 4:2:2, each horizontal pair of Y′ values has a single C_B and C_R value shared for the pair. For example, Figure 11 shows a 6-element 1-D buffer with one Y′ value for each element, but as shown in Figure 12 the C_B and C_R values are shared across pairs of elements.

Figure 11. 1-D Y′C_BC_R 4:2:2 buffer, texels associated with Y′ values

Figure 12. 1-D Y′C_BC_R 4:2:2 buffer, texels associated with C_B and C_R values

In this case, we can say that the 2×1 coordinate region forms a texel block which contains two Y′ values, one C_B value and one C_R value; our bulk-data buffer or image is composed of a repeating pattern of these texel blocks.

Note that this representation raises a question: while we have assumed so far that accessing a value at texel coordinates will provide the value contained in the texel, how should the shared C_B and C_R values relate to the coordinates of the Y′ channel? Each texel coordinate represents a value in the coordinate space of the image or buffer, which can be considered as a sample of the value of a continuous surface at that location. The preferred means of reconstructing this surface is left to the application: since this specification only defines the values in the image and not how they are used, it is concerned with the sample values rather than the reconstruction algorithm or means of access. For example, in graphics APIs the coordinates used to access a 2-D texture may offset the sample locations of a texture by half a coordinate relative to the origin of sample space, and filtering between samples is typically used to implement antialiasing. However, to interpret the data correctly, any application still needs to know the theoretical location associated with the samples, so that information is within the remit of this specification.

Our Y′ samples should fall naturally on our native coordinates. However, the C_B and C_R sample locations (which are typically at the same location as each other) could be considered as located coincident with one or other of the Y′ values as shown in Figure 13, or could be defined as falling at the mid-point between them as in Figure 14. Different representations have chosen either of these alternatives — in some cases, choosing a different option for each coordinate axis. The application can choose how to treat these sample locations: in some cases it may suffice to duplicate C_B and C_R across the pair of Y′ values, in others bilinear or bicubic filtering may be more appropriate.

Figure 13. 1-D Y′C_BC_R 4:2:2 buffer, C_B and C_R cosited with even Y′ values

Figure 14. 1-D Y′C_BC_R 4:2:2 buffer, C_B and C_R midpoint between Y′ values

Traditional APIs have described the C_B and C_R as having 2×1 downsampling in this format (there are half as many samples available in the horizontal axis for these channels).

This concept can be extended to more dimensions. Commonly, a two-dimensional image stored in Y′C_BC_R format may store the C_B and C_R channels downsampled by a factor of two in each dimension (“2×2 downsampling”, also known for historical reasons as “4:2:0”). This approach is used in, for example, many JPEG images and MPEG video streams.

Because there are twice as many rows of Y′ data as there are C_B and C_R data, it is convenient to record the channels as separate planes as shown in Figure 15 (with 2×2 texel blocks outlined in red); in addition, image compression schemes often work with the channels independently, which is amenable to planar storage.

Figure 15. 2-D Y′C_BC_R 4:2:0 planes, downsampled C_B and C_R at midpoint between Y′ values

In this case, the C_B and C_R planes are half the width and half the height of the Y′ plane, and also have half the line stride. Therefore if we store one byte per channel, the offsets for each plane in a linear representation can be calculated as:

A description based on downsampling factors is sufficient in the case of common Y′C_BC_R layouts, but does not extend conveniently to all representations. For example, one common representation used in camera sensors is a Bayer pattern, in which there is only one of the red, green and blue channels at each sample location: one red and blue value per 2×2 texel block, and two green values offset diagonally, as in Figure 16.

Figure 16. An 18×12 ^RG/_GB Bayer filter pattern (repeating pattern outlined in yellow)

A Bayer pattern can then be used to sample an image, as shown in Figure 17, and this sampled version can later be used to reconstruct the original image by relying on heuristic correlations between the channels. Technology for image processing continues to develop, so in many cases it is valuable to record the “raw” sensor data for later processing, and to pass the raw data unmodified to a range of algorithms; the choice of algorithm for reconstructing an image from samples is beyond the remit of this specification.

Figure 17. A Bayer-sampled image with a repeating 2×2 ^RG/_GB texel block (outlined in yellow)

In the Bayer representation, the red and blue channels can be considered to be downsampled by a factor of two in each dimension. The two green channels per 2×2 repeating block mean that the “downsampling factor” for the green channel is effectively $\sqrt{2}$ in each direction.

More complex layouts are not uncommon. For example, the X-Trans sample arrangement developed by Fujifilm for their digital camera range, shown in Figure 18, consists of 6×6 texel blocks, with each sample, as in Bayer, corresponding to a single channel. In X-Trans, each block contains eight red, eight blue and twenty green samples; red and blue are “downsampled” by $\sqrt{3\over 2}$ and green is “downsampled” by $3\over\sqrt{5}$ .

Figure 18. An 18×12 X-Trans image (repeating 6×6 pattern outlined in yellow)

Allowing for possible alternative orientations of the samples (such as whether the Bayer pattern starts with a row containing red or blue samples, and whether the first sample is green or red/blue), trying to encode these sample patterns implicitly is difficult.

Instead of an implicit representation, the descriptor specifies a texel block, which is described in terms of its finite extent in each of four dimensions, and which identifies the location of each sample within that region. The texel block is therefore a self-contained, repeating, axis-aligned pattern in the coordinate space of the image. This description conveniently corresponds to the concept of compressed texel blocks used in common block-based texture compression schemes, which similarly divide the image into (typically) small rectangular regions which are encoded collectively. The bounds of the texel block are chosen to be aligned to integer coordinates.

Although it is most common to consider one- and two-dimensional textures and texel blocks, it can be convenient to record additional dimensions; for example, the ASTC compressed texture format described in Section 23 can support compressed texel blocks with three dimensions. For convenience of encoding, this specification uses a four-dimensional space to define sample locations within a texel block — there is no meaning imposed on how those dimensions should be interpreted.

In many formats, all color channels refer to the same location, and the texel block defines a 1×1×1×1 region — that is, a single texel.

Tiling schemes for texel addressing can also be seen to break the image into rectangular (or higher-dimensional) sub-regions, and in many schemes these sub-regions are repeating and axis-aligned. Within reason, it is possible to define some coordinate-to-texel mapping in terms of a texel block; for example, instead of a simple tiled layout of 4×4 texels, it would be possible to describe the image in terms of a linear pattern of texel blocks, each of which contain 4×4 samples. In general, this is not a useful approach: it is very verbose to list each sample individually, it does not extend well to larger block sizes (or to infinite ranges in approaches such as Morton order), it does not handle special cases well (such as the “tail” of a mip chain), and encodings such as Hilbert order do not have a repeating mapping. In most contexts where these concepts exist, tiling is not considered to be part of a “format”.

The description above has shown that the data contributing to a texel may be stored in separate locations in memory; for example, R, G and B stored in separate planes may need to be combined to produce a single texel. For the purposes of this specification, a plane is defined to be a contiguous region of bytes in memory that contributes to a single texel block.

This interpretation contradicts the traditional interpretation of downsampled channels: if two rows of Y′ data correspond to a single row of C_B and C_R (in a linear, non-tiled memory layout), the Y′ channel contribution to the texel block is not “a contiguous region of bytes”. Instead, each row of Y′ contributing to a texel block can be treated as a separate “plane”.

In linear layout, this can be represented by offsetting rows of Y′ data with odd y coordinates by the row stride of the original Y′ plane; each new Y′ plane’s stride is then double that of the original Y′ plane, as in Figure 19 (c.f. Figure 15). If the planes of a 6×4 Y′C_BC_R 4:2:0 texture are stored consecutively in memory with no padding, which might be described in a traditional API as Table 3, the representation used by this specification would be that shown in Table 4.

There is no expectation that an API must actually use this representation in accessing the data: it is simple for an API with explicit support for the special case of integer chroma downsampling to detect interleaved planes and to deduce that they should be treated as a single plane of double vertical resolution. Many APIs will not support the full flexibility of formats supported by this specification, and will map to a more restrictive internal representation.

The Khronos Basic Descriptor Block indicates a number of bytes contributing to the texel block from each of up to eight planes — if more than eight planes are required, an extension is needed. Eight planes are enough to encode, for example:

If a plane contributes less than a byte to a texel (or a non-integer number of bytes), the texel block size should be expanded to cover more texels until a whole number of bytes are covered — q.v. Table 92.

Interlaced Y′CBCR data may associate chroma channels with Y′ samples only from the same field, not the frame as a whole. This distinction is not made explicit, in part because a number of interlacing schemes exist. One suggested convention for interlaced data is that the field of a sample be encoded in the fourth sample coordinate (the first field as samplePosition3 = 0, the second field as samplePosition3 = 1, etc.) This interpretation is not mandated to allow other reasons for encoding four-dimensional texels, although it is consistent with the fourth dimension representing “time”.

Figure 19. 2-D Y′C_BC_R 4:2:0 planes, with separate even and odd Y′ planes

For the purposes of description, the bytes contributed by each plane are considered to be concatenated into a single contiguous logical bit stream. This “concatenation” of bits is purely conceptual for the purposes of determining the interpretation of the bits that contribute to the texel block, and is not expected to be the way that actual decoders would process the data.

That is, if planes zero and one contribute two bytes of data each and planes two and three contribute one byte of data each, this bit stream would consist of the two bytes from plane zero followed by the two bytes from plane one, then the byte from plane two, then the byte from plane three.

The data format descriptor then describes a series of samples that are contained within the texel block. Each sample represents a series of contiguous bits from the bit stream, interpreted in little-endian order, and is associated with a single channel and four-dimensional coordinate offset within the texel block; in formats for which only a single texel is being described, this coordinate offset will always be 0,0,0,0.

The descriptor block for a Y′C_BC_R 4:2:0 layout is shown in Table 96. Figure 20 shows this representation graphically: the three disjoint regions for each channels and the texel block covering them are shown on the left, represented in this specification as four planes of contiguous bytes. These are concatenated into a 48-bit logical bit stream (shown in blue, in top-to-bottom order); these samples then describe the contributions from these logical bits, with geometric location of the sample at the right.

Figure 20. Y′C_BC_R 4:2:0 described in six samples

Consecutive samples with the same channel, location and flags may describe additional bits that contribute to the same final value, and appear in increasing order. For example, a 16-bit big-endian word in memory can be described by one sample describing bits 0..7, coming from the higher byte address, followed by one sample describing bits 8..15, coming from the lower address. These samples comprise a single virtual sample, of 16 bits.

Figure 21 shows how multiple contributions to single value can be used to represent a channel which, due to a big-endian native word type (high bits stored in lower byte addresses), is not contiguous in a little-endian representation (low bits stored in lower byte addresses). Here, channels are comprised of consecutive bits from a 16-bit big-endian word; the bits of each channel cease to be contiguous when the memory bytes are interpreted in little-endian order for the logical bit stream. The bit contributions to each channel from each bit location are shown in superscript, as later in this specification; the channel contributions start at bit 0 with the first sample contributing to the value, and are deduced implicitly from the number of bits each sample contributes. This example assumes that we are describing a single texel, with the same 0,0,0,0 coordinates (not shown) for each sample. The descriptor block for this format is shown in Table 97.

Figure 21. RGB 565 big-endian encoding

In Figure 21, bit 0 of the logical bit stream corresponds to bit 3 of the virtual sample that describes the green channel; therefore the first channel to be encoded is green. Bits 0 to 2 of the green channel virtual sample correspond to bits 13..15 of the logical bit stream, so these are described by the first sample. The second sample continues describing bits 3..5 of the green channel virtual sample, in bits 0..2 of the logical bit stream. The next bit from the logical bit stream that has not yet been described is bit 3, which corresponds to bit 0 of the red channel. The third sample is therefore used to describe bits 3..7 of the logical bit stream as a 5-bit red channel. The last sample encodes the remaining 5-bit blue channel that forms bits 8..12 of the logical bit stream. Note that in the case where some bits of the data are ignored, they do not need to be covered by samples; bits may also appear repeatedly if they contribute to multiple samples.

The precision to which sample coordinates are specified depends on the size of the texel block: coordinates in a 256×256 texel block can be specified only to integer-coordinate granularity, but offsets within a texel block that is only a single coordinate wide are specified to the precision of $1\over 256$ of a coordinate; this approach allows large texel blocks, half-texel offsets for representations such as Y′C_BC_R 4:2:0, and precise fractional offsets for recording multisampled pattern locations.

The sequence of bits in the (virtual) sample defines a numerical range which may be interpreted as a fixed-point or floating-point value and signed or unsigned. Many common representations specify this range. For example, 8-bit RGB data may be interpreted in “unsigned normalized” form as meaning “0.0” (zero color contribution) when the channel bits are 0 and “1.0” (maximum color contribution) when the channel is 255. In Y′C_BC_R “narrow-range” encoding, there is head-room and foot-room to the encoding: “black” is encoded in the Y′ channel as the value 16, and “white” is encoded as 235, as shown in Section 16.1 encoding”. To allow the bit pattern of simply-encoded numerical encodings to be mapped to the desired values, each sample has an upper and lower value associated with it, usually representing 1.0 and either 0.0 or -1.0 depending on whether the sample is signed.

Note that it is typically part of the “format” to indicate the numbers which are being encoded; how the application chooses to process these numbers is not part of the “format”. For example, some APIs may have two separate “formats”, in which the 8-bit pattern 0x01 may be interpreted as either the float value 1.0 or the integer value 1; for the purposes of this specification, these formats are identical — “1” means “1”.

A sample has an associated color channel in the color model of the descriptor block — for example, if the descriptor block indicates an RGB color model, the sample’s channel field could specify the R channel. The descriptor block enumerates a range of common color models, color primary values and transfer functions, which apply to the samples.

There is some redundancy in the data format descriptor when it comes to the ordering of planes and samples. In the interests of simplifying comparison between formats, it is helpful to define a canonical ordering.

The order of samples should be defined by the following rules:

Finally:

The data format descriptor describes only concepts which are conventionally part of the “format”. Additional information is required to access the image data:

For example, if texels are laid out in memory in linear order:

int numPlanes;
char *planeBaseAddresses[8];
unsigned int strides[8][4];

// Note: The strides here are assumed to be in units of the
// corresponding dimension of the texel block
char *planeAddress(uint32_t descriptor, int plane, float coords[4]) {
  return planeBaseAddresses[plane]
    // Block stride
    + ((int) (coords[0] / descriptorSize(descriptor, 0))) * strides[plane][0]
    // Row stride
    + ((int) (coords[1] / descriptorSize(descriptor, 1))) * strides[plane][1]
    // Plane stride
    + ((int) (coords[2] / descriptorSize(descriptor, 2))) * strides[plane][2]
    // Volume(?) stride
    + ((int) (coords[3] / descriptorSize(descriptor, 3))) * strides[plane][3];
}

decodeTexelGivenCoords(uint32_t *descriptor, float coords[4]) {
  char *addresses[8];
  for (int i = 0; i < numPlanes; ++i) {
    addresses[i] = planeAddress(i, coords);
  }

  // processTexel reads bytesPlane[] bytes from each address and
  // decodes the concatenated data according to the descriptor
  processTexel(descriptor, addresses, coords);
}

The processTexel function would typically operate on the coordinates having taken the remainder of dividing them by the texel block size. For example, if the texel block size is 2×2, the block and row stride provide the offset in bytes between adjacent blocks in the first two dimensions. processTexel would then work with the data corresponding to the 2×2 region with coords[0] and coords[1] in the range [0..2). For formats that describe a single texel, coords can be considered to be an integer. Note that for more complex formats such as Bayer layouts, reconstructing an R,G,B value at a location may require information from more than one texel block, depending on the algorithm used, in a manner analogous to sampling using bilinear filtering.

The stride values may be stored explicitly, or derived implicitly from the bytesPlane values and image size information, with application-specific knowledge about alignment constraints.

Despite being designed to balance size against flexibility, the data format container described here is too unwieldy to be expected to be used directly in most APIs, which will generally support only a subset of possible formats. The expectation is that APIs and users will define data descriptors in memory, but have API-specific names for the formats that the API supports. If these names are enumeration values, a mapping can be provided by having an array of pointers to the data descriptors, indexed by the enumeration. It may commonly be necessary to provide API-specific supplementary information in the same array structure, particularly where the API natively associates concepts with the data which is not uniquely associated with the content.

In this approach, it is likely that an API would predefine a number of common data formats which are natively supported. If there is a desire to support dynamic creation of data formats, this array could be made extensible with a manager assigning unique handles.

Even where an API supports only a fixed set of formats, it is flexible to provide a comparison with user-provided format descriptors in order to establish whether a format is compatible (and differs only in a manner irrelevant to the API).

Some APIs have the concept of a native data type for a format, if each channel is stored separately. Since this specification describes a number of bytes per plane and separately contiguous bit sequences, there is no such explicit concept. However, if a sample’s bitOffset and bitLength are byte-aligned and no further samples contribute to the same value, the bitLength trivially defines a little-endian native data type size. Big-endian data types can be identified by observing that in a big-endian format, a sequence of bits in the top bits of byte n may continue in the low bits of byte n - 1. Finally, “packed” formats consist of consecutive bit sequences per channel in either little- or big-endian order; little-endian sequences are a single stand-alone sample, and a big-endian sequence consists of a number of samples adjacent to byte boundaries in decreasing byte order (see Figure 21); the packed field size can typically be deduced from the bytesPlane0 value. There is no way to distinguish a logically “packed”, byte-aligned samples from distinct but consecutively-stored channels that have the same in-memory representation.

Each descriptor block has the same prefix, shown in Table 6.

The vendorId is a 17-bit value uniquely assigned to organizations. If the organization has a 16-bit identifier assigned by the PCI SIG, this should occupy bits 0..15, with bit 16 set to 0. Other organizations should apply to Khronos of a unique identifier, which will be assigned consecutively starting with 65536. The identifier 0x1FFFF is reserved for internal use which is guaranteed not to clash with third-party implementations; this identifier should not be shipped in libraries to avoid conflicts with development code.

The descriptorType is a unique 15-bit identifier defined by the vendor to distinguish between potential data representations.

Prior to version 1.3 of the Khronos Data Format Specification, the vendorId field was 16-bit, and purely assigned through the auspices of this specification; the descriptorType was consequently also 16-bit. Since no vendor has requested an identifier and Khronos does not have a descriptor block with type 1, this change should not cause any ambiguity. This change is intended to allow consistency with the vendor IDs used by the Vulkan specification.

The versionNumber is vendor-defined, and is intended to allow for backwards-compatible updates to existing descriptor blocks.

The DescriptorBlockSize indicates the size in bytes of this Descriptor Block, remembering that there may be multiple Descriptor Blocks within one container, as shown in Table 7. The descriptorBlockSize therefore gives the offset between the start of the current Descriptor Block and the start of the next — so the size includes the vendorId, descriptorType, versionNumber and descriptorBlockSize fields, which collectively contribute 8 bytes.

Having an explicit descriptorBlockSize allows implementations to skip a descriptor block whose format is unknown, allowing known data to be interpreted and unknown information to be ignored. Some descriptor block types may not be of a uniform size, and may vary according to the content within.

This specification initially describes only one type of stand-alone descriptor block, plus two extension blocks which modify the description in the first. Future revisions may define additional descriptor block types for additional applications — for example, to describe data with a large number of channels or pixels described in an arbitrary color space. Vendors can also implement proprietary descriptor blocks to hold vendor-specific information within the standard descriptor.

Unless otherwise specified, descriptor blocks can appear in any order, to make it easier to add and remove additional informative descriptor blocks to a preexisting data format descriptor as part of processing. Descriptor blocks that provide additional capabilities beyond a basic scheme (such as the descriptor block for supporting additional planes beyond the Khronos Basic Descriptor Block) should not be present unless their additional capabilities are needed; that is, redundancy should be resolved so as to minimize the number of descriptor blocks in the data format descriptor.

The vendorId for the Basic Data Format Descriptor Block is 0, defined as KHR_DF_VENDORID_KHRONOS in the enum khr_df_vendorid_e.

khr_df_vendorid_e vendorId = KHR_DF_MASK_VENDORID & (bdb[KHR_DF_WORD_VENDORID] >> KHR_DF_SHIFT_VENDORID);

The descriptorType for the Basic Data Format Descriptor Block is 0, a value reserved in the enum of Khronos-specific descriptor types, khr_df_khr_descriptortype_e, as KHR_DF_KHR_DESCRIPTORTYPE_BASICFORMAT.

khr_df_descriptortype_e descriptorType = KHR_DF_MASK_DESCRIPTORTYPE & (bdb[KHR_DF_WORD_DESCRIPTORTYPE] >> KHR_DF_SHIFT_DESCRIPTORTYPE);

The versionNumber relating to the Basic Data Format Descriptor Block as described in this specification is 2.

	The versionNumber is incremented to indicate an incompatible change in the descriptor. The addition of enumerant values, for example to represent more compressed texel formats, does not constitute an “incompatible change”, and implementations should be resilient against enumerants that have been added in later minor updates.

uint32_t versionNumber = KHR_DF_MASK_VERSIONNUMBER & (bdb[KHR_DF_WORD_VERSIONNUMBER] >> KHR_DF_SHIFT_VERSIONNUMBER);

The memory size of the Basic Data Format Descriptor Block depends on the number of samples contained within it. The memory requirements for this format are 24 bytes of shared data plus 16 bytes per sample. The descriptorBlockSize is measured in bytes.

uint32_t descriptorBlockSize = KHR_DF_MASK_DESCRIPTORBLOCKSIZE & (bdb[KHR_DF_WORD_DESCRIPTORBLOCKSIZE] >> KHR_DF_SHIFT_DESCRIPTORBLOCKSIZE);

The colorModel determines the set of color (or other data) channels which may be encoded within the data, though there is no requirement that all of the possible channels from the colorModel be present. Most data fits into a small number of common color models, but compressed texture formats each have their own color model enumeration. Note that the data need not actually represent a color — this is just the most common type of content using this descriptor. Some standards use color container for this concept.

The available color models are described in the khr_df_model_e enumeration, and are represented as an unsigned 8-bit value.

khr_df_model_e colorModel = KHR_DF_MASK_MODEL & (bdb[KHR_DF_WORD_MODEL] >> KHR_DF_SHIFT_MODEL);

Note that the numbering of the component channels is chosen such that those channel types which are common across multiple color models have the same enumeration value. That is, alpha is always encoded as channel ID 15, depth is always encoded as channel ID 14, and stencil is always encoded as channel ID 13. Luma/Luminance is always in channel ID 0. This numbering convention is intended to simplify code which can process a range of color models. Note that there is no guarantee that models which do not support these channels will not use this channel ID. Particularly, RGB formats do not have luma in channel 0, and a 16-channel undefined format is not obligated to represent alpha in any way in channel number 15.

The value of each enumerant is shown in parentheses following the enumerant name.

5.5.3. KHR_DF_MODEL_YUVSDA (= 2)

This color model represents color differences with three channels, nominally luma (Y′) and two color-difference chroma channels, U (C_B) and V (C_R), supplemented by channels for alpha, depth and stencil, as shown in Table 16. These formats are distinguished by C_B and C_R being a delta between the Y′ channel and the blue and red channels respectively, rather than requiring a full color matrix. The conversion between Y′C_BC_R and RGB color spaces is defined in this case by the choice of value in the colorPrimaries field as described in Section 15.1.

	Most single-channel luma/luminance monochrome data formats should select KHR_DF_MODEL_YUVSDA and use only the Y channel, unless there is a reason to do otherwise.

Table 16. Basic Data Format YUVSDA channels

Channel number	Name		Description
0	KHR_DF_CHANNEL_YUVSDA_Y		Y/Y′ (luma/luminance)
1	KHR_DF_CHANNEL_YUVSDA_CB		CB (alias for U)
1	KHR_DF_CHANNEL_YUVSDA_U		U (alias for CB)
2	KHR_DF_CHANNEL_YUVSDA_CR		CR (alias for V)
2	KHR_DF_CHANNEL_YUVSDA_V		V (alias for CR)
13	KHR_DF_CHANNEL_YUVSDA_STENCIL		Stencil
14	KHR_DF_CHANNEL_YUVSDA_DEPTH		Depth
15	KHR_DF_CHANNEL_YUVSDA_ALPHA		Alpha (opacity)

Terminology for this color model is often abused. This model is based on the idea of creating a representation of monochrome light intensity as a weighted average of color channels, then calculating color differences by subtracting two of the color channels from this monochrome value. Proper names vary for each variant of the ensuing numbers, but YUV is colloquially used for all of them. In the television standards from which this terminology is derived, Y′CBCR is more formally used to describe the representation of these color differences. See Section 15.1 for more detail.

A number of compressed formats are supported as part of khr_df_model_e. In general, these formats will have the texel block dimensions of the compression block size. Most contain a single sample of channel type 0 at offset 0,0 — where further samples are required, they should also be sited at 0,0. By convention, models which have multiple channels that are disjoint in memory have these channel locations described independently as separate samples; this can simplify some decoders.

The ASTC family of formats have a number of possible channels, and are distinguished by samples which reference some set of these channels. The texelBlockDimension fields determine the compression ratio for ASTC and PVRTC.

Compressed formats necessarily do not have an equivalent integer representation in which to describe the sampleLower and sampleUpper ranges — in particular, some have different ranges on a block-by-block format. Floating-point compressed formats have lower and upper limits specified in floating point format, since this representation indicates the output of compressed decoding. Integer compressed formats with a lower and upper of 0 and UINT32_MAX (for unsigned formats) or INT32_MIN and INT32_MAX (for signed formats) are assumed to map the full representable range to 0..1 or -1..1 respectively.

If a format has a non-linear transfer function, any samples with channel ID 15 (that is, the format has separate alpha encoding, for example KHR_DF_BC2_ALPHA) should set the KHR_DF_SAMPLE_DATATYPE_LINEAR bit for that sample.

Example descriptors for compressed formats are provided after each model in this section.