Difference between revisions of "Images"

Latest revision as of 15:38, 2 August 2021

The GDISP module comes with a built-in image decoder. The decoder allows it to open an image of various formats and display it. As the GFILE module is being used internally, the image can be stored on different sources such as the internal flash or external memory like an SD card.

Before you continue to read this documentation, please consider using the ImageBox widget as it is much simpler to use.

API reference

The API reference for the image support can be found here.

RAM Usage

Image decoders use RAM to decode and display images. Whilst the image handlers have been written from scratch to use as little RAM as possible, for small microcontrollers with limited RAM the image formats used should be choosen carefully. The image handlers do not allocate RAM to store the full decompressed bitmap as do most other image decoders, instead the image is decoded again if it needs to be displayed. The only RAM used therefore is:

• Some RAM to hold information on the image itself - typically 200 to 300 bytes. This RAM is maintained while the image is opened. It may vary slightly for some images with some image formats (due to holding palettes etc).
• RAM allocated during the decoding process and freed once the decoding is done. The GIF image format requires around 12K of RAM to decode an image. BMP and NATIVE images do not require any extra RAM for decoding.
• If you decide to cache the image then RAM is required for the full decoded image. This should not be considered for low memory micros. For example, caching a 320x240 image on a 2 bytes per pixel display will take 150K of RAM (plus the normal decoding RAM).
• Stack space. You may need to increase your available stack space if you get exceptions when trying to decode images. Some image formats require a few hundred bytes of stack space to decode an image.

Our image decoders have been written from the ground up to keep the image decoders as lean and mean as possible. Our current decoders use significantly less than other decoding libraries available.

Accounting

It is possible to examine the exact amount of memory that is used by an image. Setting GDISP_NEED_IMAGE_ACCOUNTING to TRUE in the configuration file will add the two members memused and maxmemused to the image struct (the gdispImage data type). The first one gives information about the currently used memory, the second one keeps track of the maximum amount of memory that was ever used by this image.

Caching

It's possible to cache a decoded image into your ram using the gdispImageCache() call. When you don't cache an image, it will always be re-read again from flash, decoded and then displayed. When you cache your image, you just have to load it from RAM and display it. This is way faster, especially for PNG, JPG and GIF formats since these require quite complex decoding algorithms. However, caching an image requires A LOT of RAM. Especially when you're using multiframe GIF images or large sized images.

If you cache your image you still have to open() the image before it can be displayed. When close() is called, it will free all memory used by the decoder including any cached images.

Calling the caching routine doesn't guarantee that the image will be cached. For example, if not enough RAM is available the image won't be cached. Since caching is fully optional, the image will still be drawn when you call the drawing routine. It will simply be decoded again.

A word about close()

The gdispImageClose() function performs a complete cleanup. Calling this routine will not only clean up all memory used by the decoder and the caching, it will also close the file system. Therefore, this should really just be called when you're done with the image and you won't re-draw it for a longer time period.

Color Palettes

Certain image formats support color palettes. The following functions are available to access and modify the color palettes:

uint16_t   gdispImageGetPaletteSize(gdispImage *img);
gColor     gdispImageGetPalette(gdispImage *img, uint16_t index);
gBool      gdispImageAdjustPalette(gdispImage *img, uint16_t index, color_t newColor);

Please check the API reference to learn more about these functions.

Image formats

µGFX does currently come with the following image decoders:

We plan to add decoders for other formats such as JPG in the future.

NATIVE

The native image format consists of an 8 byte header followed by an array of bytes in the format your display controller accepts. Whilst hardware dependant, this format displays extremely quickly and with the least amount of RAM usage. It is ideal for images stored in FLASH. The header is:

{ 'N', 'I', width.hi, width.lo, height.hi, height.lo, format.hi, format.lo }

Where the format word is equal to GDISP_PIXELFORMAT which is defined in the conf.h file of your GDISP driver.

BMP

The BMP format is very performance friendly as it is a bitmap of pixels. There's only little to no decoding required to display a BMP image. Therefore, it is often used for UI designs. The advantage over the Native format is it's portability.

GIF

The GIF decoder can handle animated images and transparency out of the box.

Animated images

The GIF decoder simply advances the image pointer by one frame on each gdispImageDraw() call. Therefore, displaying an animated image is simply a matter of calling gdispDrawImage() in a loop. However, the user manually has to impliment the delay before loading the next image. This can be done like this:

while (1) {
    gdispImageDraw(&myImage, 0, 0, myImage.width, myImage.height, 0, 0)
    delay = gdispImageNext(&myImage);
 
    gfxSleepMilliseconds(delay);
}

Note: This is just an illustrative example. A real world program would have to check for error return codes etc. A full example can be found under /demos/modules/gdisp/images_animated.

PNG

The PNG image decoder can handle all PNG features including transparency except for interlacing. All the various PNG sub-formats and features can be turned on or off in the configuration file in order to decrease code size. By default all supported formats are enabled whenever PNG images are enabled.

Alpha & Transparency

The decoder supports both transparency and alpha. When a background color is specified in the PNG file alpha and transparency are handled by color blending with the background color. If a background color has not been specified by the PNG file then any pixel alpha below GDISP_NEED_PNG_ALPHACLIFF is just not displayed.

RAM usage

Compared to other formats such as BMP or GIF, PNG images require vast amounts of RAM to be decoded. Therefore, using PNG images is usually discouraged on very small embedded systems where RAM is a scarce resource.

PNG images have the following RAM requirements:

• 64 bytes (or less) for the gdispImage structure and internal PNG image information
• Around 34k bytes + 2 image scan lines used only during image decoding

This means that even super huge PNG images of say 3072x2304 can be fully decoded and displayed using around 40k bytes of RAM.

Unfortunately there is nothing that can be done to further reduce the amount of RAM usage. There are a couple of big memory hogs in PNG decoding...

• A sliding window of 32K is needed for the inflate decompression. While some images may use less, because the inflate window size is non-deterministic while decoding the maximum sliding window has to be allocated up front.
• 2 scan lines of uncompressed data are required to perform PNG filtering. For a 16-bit RGB with alpha that amounts to (1 + 8 bytes per pixel * width) * 2. Obviously for an 8-bit palette based PNG it is (1 + 1 * width) * 2, and for a 1-bit palette PNG it is image (1 + width / 8) * 2

Unfortunately these overheads cannot be reduced as they are required by the PNG specification.

Most PNG decoders (and decoder libraries) require:

1. A full copy of the input file in memory
2. The PNG header information and palette storage overheads
3. The 32K sliding window and scanline buffers (although this is often integrated into #4 below)
4. A full copy of the decompressed byte stream in memory
5. A full copy of the output image in PNG color format after scanline filtering
6. A full copy of the output image in output device color format.

Obviously this is a huge amount of RAM, well beyond the capabilities of most embedded devices. We therefore wrote our decoder from scratch to remove those overheads where-ever possible. For our decoder we only need #2 and #3 and furthermore #3 is only needed while the decoding is actually happening.

GIF vs. PNG

We have not performed any speed tests comparing GIF's with PNG's. Both use similar technology to decode the image but the differences will determine the relative speeds. They use different compression algorithm's. GIF is also 8-bit palette pixel format only whereas PNG supports a multitude of internal pixel formats. Both support transparency but only PNG supports alpha blending. With GIF we support interlacing but not with PNG due to complexity. GIF supports animation, PNG does not. In conclusion, the differences in decoding speed between PNG and GIF is going to depend very much on the actual image.

There is however one exception to the above, when a GIF image is cached to RAM using gdispImageCache() we cache the decompressed byte stream. For PNG we cache the compressed byte stream. In this situation it is likely that cached GIF images will display faster than cached PNG images but will take more RAM to cache.

Examples

The following code pieces shows how to use the GDISP image decoder correctly. Please refer to the GFILE article to learn how to use the different file systems.

#include "gfx.h"
 
/**
 * The image file must be stored on a GFILE file-system.
 * Use either GFILE_NEED_NATIVEFS or GFILE_NEED_ROMFS (or both).
 *
 * The ROMFS uses the file "romfs_files.h" to describe the set of files in the ROMFS.
 */
 
static gdispImage myImage;
 
int main(void) {
	gCoord swidth, sheight;
 
	// Initialize uGFX and the underlying system
	gfxInit();
 
	// Get the display dimensions
	swidth = gdispGetWidth();
	sheight = gdispGetHeight();
 
	// Set up IO for our image
	gdispImageOpenFile(&myImage, "myImage.bmp");
	gdispImageDraw(&myImage, 0, 0, swidth, sheight, 0, 0);
	gdispImageClose(&myImage);
 
	while(1) {
		gfxSleepMilliseconds(1000);
	}
 
	return 0;
}

Troubleshooting

When troubleshooting, please check the return value of each function in order to track down the problem.

No image is shown

It is possible that the program compiles fine and that it runs without any crashes, but no image is being displayed. This behavior can be caused by a variety of different issues:

• The corresponding decoder has not been enabled in the configuration file
• The image could not be opened
• The image decoder could not allocate enough memory
• When loading an image from a file, it's possible that the maximum number of file handles has been reached and opening the file fails. In that case, increase GFILE_MAX_GFILES in the configuration file or close an already opened file that is no longer needed.