Images

Image Formats

In order to maximize the speed and efficiency of the system, the source images must be in a multi-resolution format. This allows for rapid access to individual image tiles at any resolution with minimal server overhead. The is especially important for very high resolution images. The IIPImage server supports both TIFF and JPEG2000 formats. TIFF is supported by IIPImage via the libtiff library with files saved in tiled multi-resolution (pyramid) format. JPEG2000 is supported through OpenJPEG or through the non-open source Kakadu JPEG2000 codec, which provides better performance.

Images can be in either 8, 16 or 32 bits per channel format and in greyscale, sRGB or CIE L*a*b* colour space.

Tiled Multi-Resolution TIFF

Tiled Multi-Resolution Pyramid Structure

The TIFF image format is an extremely flexible and powerful container allowing almost any possible type of image or even scientific data to be efficiently stored and compressed. Images can be stored in any bit depth: from 1 bit for bi-level images, through standard 8 bit images through to 32 or 64 bit floating point data. Images can consist of 1 or more channels allowing even multispectral and hyperspectral imagery to be stored in a single file. This image data can be stored in uncompressed raw form or compressed losslessly using Deflate or LZW or compressed lossily with JPEG or WebP compression for smaller file sizes.

TIFF also allows image data to be structured in ways that allow for fast random access by means of tiling and through the creation of multi-resolution image pyramids. Tiling allows random regions from an image to be quickly decoded without the need to decode the entire image. Multiple resolutions can also be stored in a pyramid structure to allow fast access to images at any size. The combination of tiling and multi-resolution pyramids provides extremely fast random access to any part of an image at any size, allowing gigapixel or even terapixel large image or data sets to be handled comfortably.

Tiled Multi-Resolution TIFF (or Tiled Pyramid TIFF) are standard TIFF files with each tiled resolution stored as a separate layer within the TIFF. This example TIFF image can be used to test your IIPImage server.

Tiled multi-resolution TIFFs are widely supported and can be read by most image processing libraries and applications including Photoshop, GIMP, ImageMagick and many others. A number of applications can also create tiled multi-resolution TIFF. Notably VIPS, ImageMagick, GDAL, Bio-Formats and STIFF.

Conversion using VIPS

The recommended way to create tiled multi-resolution TIFF images is using the high performance VIPS image processing suite. To generate a tiled pyramid TIFF image use nip2, the VIPS GUI or create one directly from the command line:

vips im_vips2tiff <source_image> <output_image.tif>:<compression>,tile:<size>,pyramid
  • source_image: can be any kind of image supported by VIPS
  • output_image.tif: your TIFF output image
  • compression: either none, deflate, LZW or JPEG
  • size: pixel dimensions of tiles, eg: 256×256

For example, to generate a TIFF with deflate compression and a tile size of 256×256, use:

vips im_vips2tiff source_image output_image.tif:deflate,tile:256x256,pyramid

It is also possible to generate tiled pyramid TIFF images with JPEG compression at quality 75:

vips im_vips2tiff source_image output_image.tif:jpeg:75,tile:256x256,pyramid

The latest version of libtiff (version 4+) includes BigTIFF support, allowing images to go beyond the standard TIFF size limit of 4GB. To use this, make sure that both VIPS and IIPImage have been compiled and are linked to the correct version of libtiff. The conversion syntax for creating an image with VIPS is slightly different:

vips im_vips2tiff source_image output_image.tif:jpeg:75,tile:256x256,pyramid,,,,8

where the 8 indicates that the TIFF should be BigTIFF enabled. (The 3 blank commas before the 8 are for the optional parameters format,resolution and ICC profile. See the VIPS documentation for more details).

From VIPS version 7.28 onwards, a new sequential mode is available within VIPS which is both faster and more efficient in terms of disk access. To enable this, use the following syntax (type vips tiffsave on the command line to see a full list of options):

vips tiffsave source_image output_image.tif --tile --pyramid --compression deflate --tile-width 256 --tile-height 256

Conversion using ImageMagick

You can also use ImageMagick (version 6.4.7-10 and upwards) to create Tiled Pyramid TIFF. In this case use the convert command. For example, to generate a 256×256 pyramid tiled tiff using JPEG compression:

convert input -define tiff:tile-geometry=256x256 -compress jpeg 'ptif:output.tif'
  • input: source image – can be any kind of image supported by ImageMagick (almost all)
  • ptif: specify your image format as pyramid tiff
  • output.tif: output image – *MUST* have tif or ptif extension;
  • 256×256: the tile size

Conversion using GDAL

GDAL is suite of tools for handling geospatial data formats and is part of the The Open Source Geospatial Foundation. The gdaladdo tool can be used to create tiled multi-resolution TIFF. For example, to create a JPEG-compressed tiled multi-resolution TIFF with a tile size of 256×256 pixels:

gdaladdo --config GDAL_TIFF_OVR_BLOCKSIZE 256 --config COMPRESS_OVERVIEW JPEG -r average -ro input.tif

This will create a file with an .ovr extension (input.tif.ovr), but which is in fact a standard TIFF, which can be used directly by IIPImage. The GDAL_TIFF_OVR_BLOCKSIZE parameter defines the tile size, –config COMPRESS_OVERVIEW sets the compression used (if any) and -r sets the resampling algorithm (nearest neighbour by default). See the gdaladdo web page for more information.

Conversion using Bio-Formats

Bio-Formats is a standalone Java library for reading and writing life sciences image file formats developed by the OME Open Microscopy consortium. The bfconvert tool can be used to create tiled multi-resolution TIFF from any supported microscopy format. For example to create a tiled pyramid TIFF with 4 resolution levels, a tile size of 256×256 and lossless Deflate compression:

bfconvert -pyramid-resolutions 4 -pyramid-scale 2 -tilex 256 -tiley 256 -compression zlib input.tif output.tif

Conversion using STIFF

It is also possible to create a Tiled Pyramid TIFF from scientific data in FITS format using STIFF. STIFF is a FITS to TIFF conversion package for astronomy data that  allows control of gamma, contrast,  brightness and color saturation and is fully compatible with IIPImage. For example to combine 3 FITS images into a single tiled multi-resolution TIFF with the specified min and max settings:

stiff -IMAGE TYPE TIFF-PYRAMID d1 i.fits d1 r.fits d1 g.fits -GAMMA FAC 1.1 -COLOUR SAT 2.2 -MAX TYPE MANUAL -MAX LEVEL 550,350,160 -MIN LEVEL 0.00001 -SATUR LEVEL 10000 -COMPRESSION TYPE DEFLATE -OUTFILE NAME d1 gri.ptif

See the STIFF documentation for more details on how to use it and the options available.

JPEG2000

JPEG2000 is an advanced wavelet-based image compression and coding format that gives improved compression over standard JPEG and much greater flexibility with support for multi-resolution coding, multiple bit-depths, region-of-interest, quality layers and lossless compression. Images can be in 1, 8 or 16 bits per channel and both the jp2 (JPEG2000 Part 1 ISO/IEC 15444-1), jpx  (JPEG2000 Part 2 ISO/IEC 15444-2) and jph (JPEG2000 Part 15 ISO/IEC 15444-15) formats are supported. Note that jpc raw JPEG2000 codestreams are not yet supported.

If you wish to use JPEG2000, you will need to compile the IIPImage server with either OpenJPEG or Kakadu. OpenJPEG is an open source library that provides full support for JPEG2000. The Kakadu JPEG2000 codec is a widely used commercial codec which provides significantly faster JPEG2000 decoding. It is. however, not open source –  you will have to purchase a copy of the SDK from Kakadu in order to use it (if you are interested in obtaining pre-built binaries of the IIPImage server with Kakadu included for commercial use, please contact us).

Optimizing JPEG2000 Encoding Parameters

Any JPEG2000 image can be used with IIPImage, but in order to get the best performance, you should encode your images in an optimal way for fast random access. The key parameters for this are:

  • Resolution levels: A sufficient number of resolution levels should be created. The number required depends on the size of the image. Each resolution is a factor of 2 smaller than the next, so for images of around 2000×2000 pixels in size, 5 levels should be sufficient, for 4000×4000 6 levels and for 8000×8000 7 levels etc.
  • Resolution-based progression order (RPCL): In order to allow efficient access to different resolutions, a resolution-based progression order should be used by setting the progression order to Resolution –  Component – Position – Layer (RPCL)
  • Precincts (256×256): Precincts allow decoding to be carried out only of specific regions (in the same way that tiling works with TIFF). Precinct sizes should be the same as the size of the tiles that will be requested by tile-based viewers (typically 256×256 pixels)
  • Code blocks (64×64): Code blocks help further with random access and should be as large as possible, with the maximum allowed size being 64×64 pixels.
  • Markers: Adding markers to a JPEG2000 image can make significant improvements to decoding speed. PLT packet length markers add a packet index to the file header which allows faster access to resolution levels and regions within an image. TLM tile part length markers provide a coarser index that can be used if a decoder does not support PLT markers. Although not related to decoding speed, the use of packet markers such as SOP (Start of Packet) or EPH (End of Packet Header) is also recommended and provides a level of error resiliency and allows decoding to continue even in the presence of file corruption.
  • HTJ2K: If you are using iipsrv with a recent version of Kakadu (>8.3) or OpenJPEG (>2.5), encoding files using the High Throughput extension to JPEG2000 (HTJ2K) can also make a significant difference to decoding performance with up to 2x faster decoding speed for lossy compression.

JPEG2000 supports the encoding of quality layers. The decoding of specific quality layers is supported by IIPImage, but this necessarily slows down decoding. Quality layers in JPEG2000 should, therefore, only be created during encoding if you wish to use this feature to allow IIPImage to limit decoding to lower quality versions of the image.

It is also possible to use tiling within JPEG2000, but the existence of precincts and code blocks largely replicates the features of the tiled pyramid structure used by TIFF making JPEG2000 tiling unnecessary and often even counter-productive for fast random access. Unlike with tiled pyramid TIFF, tiling with JPEG2000 is only applied to the full size image and not the sub-resolutions. Accessing a lower resolution, therefore, requires accessing every tile, which will make decoding speed slower. Moreover, it is not recommended to use tiling when using lossy JPEG2000 compression as compression artifacts can be visible at tile boundaries (compression artifacts are not an issue with lossless encoding).

Encoding with Kakadu

In order to encode using Kakadu, you can use the kdu_compress command line tool. The key parameters are Clayers (number of quality layers), Clevels (number of resolutions), Cprecincts (precinct size – which are tiles in the wavelet domain), Cblk (code block size), Corder (encoding compression order – RPCL for resolution encoding) and rate (compression rate).

For example, to create a lossy encoded JPEG2000 (part 1) image with 1 quality layer, 7 resolutions, precincts of size 256×256, code blocks of size 64×64, multi-resolution encoding and a compression rate of 2.5:

kdu_compress -i input.tif -o output.jp2 -rate 2.5 Clayers=1 Clevels=7 Cprecincts="{256,256}" Corder=RPCL Cblk="{64,64}" ORGgen_plt=yes ORGplt_parts=R ORGtparts=R ORGgen_tlm=8 Cuse_sop=yes

The ORGgen_plt and ORGplt_parts parameters enable PLT markers and specify that these should be divided according to resolution level (R). ORGgen_tlm and ORGtparts enables tile part length markers and defines how these are divided. ORGgen_tlm should be set to the number of resolutions levels + 1. Thus, if you have asked for 7 resolution levels, there should be 8 TLM marker segments. ORGtparts should be set to resolution level (R).

The Cuse_sop=yes parameter adds packet synchronization markers for error robustness.

To create the same lossy encoded image using the faster HTJ2K to produce an output file in jph format:

kdu_compress -i input.tif -o output.jph Cmodes=HT -rate 2.5 Clevels=7 Cprecincts="{256,256}" Corder=RPCL Cblk="{64,64}" ORGgen_plt=yes ORGplt_parts=R ORGtparts=R ORGgen_tlm=8 Cuse_sop=yes

Although not recommended for performance reasons, if you wish, nonetheless, to add tiling, this can be done through the Stiles parameter. For example, to use a a tile size of 1024×1024 pixels, add the following parameters to the above command:

Stiles="{1024,1024}"

It is also possible to encode losslessly using the Creversible=yes parameter. Lossless encoding with JPEG2000 is capable of reducing file size to around 40% of the original raw data size. Several quality layers can also be included if you wish to keep a lossless archive image, but allow users to have fast access to only a lower quality lossy version of the image.

Encoding with OpenJPEG

OpenJPEG also provides command line tools to encode images. To create a lossy encoded JPEG2000 (part 1) image with 1 quality layer, 7 resolutions, precincts of size 256×256, code blocks of size 64×64, multi-resolution encoding, a compression rate of 2.5 and including PLT, TLM and SOP markers:

opj_compress -i input.tif -o output.jp2 -r 2.5 -n 7 -c "[256,256]" -b "64,64" -p RPCL -SOP -PLT -TLM -TP R -SOP

If no rate parameter is specified, encoding is performed losslessly. If tiling is required, add the following parameters, where -t specifies the desired tile size:

-t 1024,1024

Image File Names

The image paths given to the IIPImage server (via the FIF command for IIP requests or IIIF for IIIF requests) must be absolute system paths on the server machine (eg. FIF=/images/test.tif) and not paths relative to the web server document root location. Don’t forget to make sure the IIPImage server process owner has read access on the images! On Windows, you should still use forward slashes “/” for the image path, not “\” and there is no need for the “c:” prefix.

In addition, the server’s FILESYSTEM_PREFIX configuration variable allows you to define a prefix that is pre-pended to each path. For example, if FILESYSTEM_PREFIX is set to “/mnt/images/”, all paths provided to the server will have this prefix added to the beginning. Thus a request for an image 001.tif will be mapped to /mnt/images/001.tif.

Similarly, the FILESYSTEM_SUFFIX configuration variable allows you to define a suffix that is added to the file name. For example, if you set FILESYSTEM_SUFFIX to “.tif”, a request to an image 001 will be mapped to 001.tif.

Note that iipsrv uses magic file signatures to determine the input image type and does not make use of image suffixes such as .tif or .jp2. It is, therefore, possible to use arbitrary suffix-less image names.