First Project Report - October 21, 2008:
The goal of the related work section is to better understand approaches to image salience, and to develop an understanding of what is feasible for an implementation. We explore two types of work: the first on image saliency, and specific algorithms. The second type of work explored includes specific case studies, to discover issues and techniques used in various implementations, as well as develop an understanding of what approaches to image salience are feasible for specific target hardware.
Image Salience
While salience is a general, subjective term to define importance of any region of an image, many image salience techniques focus on the preservation of image details. For example, approaches will preserve high gradient regions, or deform the image while preserving the strucutre of detected edges. Other approaches rely on the existence of a salience map, which defines a one-to-one mapping between pixels of an image and a salience value. The following papers, while certainly not an exhaustive list, explore some different approaches and concepts of interest.
In Seam Carving for Content-Aware Image Resizing, the image gradient is used to define an energy metric, and seams (8-connected paths travelling from either from the top to the bottom of the image, or the left to the right) are removed or added where energy is low.
In Detail Preserving Shape Deformation in Image Editing, a system is presented that allows the deformation of source images, such that the texture detail of the image is preserved. The approach works by growing curvilinear coordinates around the feature, and then resynthesizing the texture from source to target.
In Stylization and Abstraction of Photographs, a method is proposed to produce stylized version of images, by first segmenting the image, and colouring the segments uniformly. Further, stylized strokes are drawn along the paths of detected edges. While the paper covers using an eye-tracker to determine the salient region - an interesting point is that the stylized reproductions draw the eye to the same salient regions in the original image.
Case Studies
In Region-Based Visual Attention Analysis with Its Application in Image Browsing on Small Displays, image segmentation is used to estimate the salient regions. The saliency of segmented region is determined partly by its position, and partly by its contrast when all regions are coloured uniformly.
MobiPicture - Browsing Pictures on Mobile Devices, presents an implementation that runs on Windows Mobile, that allows browsing of images and that includes some useful features: "smart" thumbnail view, automatic zoom in/out, and automatic navigation. These features work due to the proposed "Image Attention Model", which when applied identifies a handful of "attention objects" in a given image. The model is based on four concepts: Region-of-Interest, Attention Value, Minimal Perceptible Size, and Minimal Perceptible Time.
The target hardware is the i-mate JAQ3, a commodity Pocket PC which runs Windows Mobile 5.0.
The following points summarize the relevant features of this device:
The Texas Instruments OMAP850
Within this microprocessor, the core CPU is an ARM ARM926EJ-S, a 200MHz processor which supports the ARMv5TEJ instruction set. Most notable is that the processor is a 32-bit RISC CPU, which supports the 32-bit ARM instructions, as well as 16-bit Thumb instructions for improved performance, which may find use during implementation. In addition, this particular processor is described as "Jazelle technology enhanced", meaning that it was designed to better run applications in Java - in particular, supporting low-level instructions for Java bytecode exectuion. This technology may make development of an application in Java to run on the device a possible alternative to the intended development of a program for the Windows Mobile OS.
Windows Mobile Development
Where Windows Mobile is the target platform, Visual Studio 2008 will be used during development. If the application is written using managed code (.NET) rather than native code (plain C++), using Visual Studio may in fact be a necessity. The Windows Mobile 5.0 SDK for Pocket PC includes sample code (and more importantly, documentation) to get an application up and running quickly.
Device Display
The display has a resolution of 320x240 and 16-bit color depth, whereby the red and blue components have 5 bits (32 distinct intensities) and the green component has 6 bits (64 distinct intensities). While most images are stored in a 24-bit RGB format, where each colour channel has 8 bits, it may be possible to improve performance by first transcoding the image data to 16-bit RGB, prior to computation.