BlueAdmiral.com

• text - different fonts and to produce special effects such as colour and fill.
• audio - various audio file formats include Microsoft WAV (wave) and MIDI, which is a more compact representation of sound.
• still images - pixels can be 0 or 1 ('white' or 'black') or hi-res colour images with 8, 16 or 24 bits per pixel.
• digital video - usually stored as a sequence of frames. For realistic playback, the transmission, compression, and decompression of digitized continuous frames requires transfer rates of 30 frames per second. If audio is required as well, the audio and video must be interleaved so that the sequences can be timed properly. Microsoft's AVI format can synchronize playback of audio and video.
• graphical objects - such as 2- and 3-dimensional images.

Most of these data types require a lot of storage space. An average page of text may require about 2 KB; 75 minutes of high-fidelity music may need 100MB; a full page, still image varies from about 10KB for black-and-white to several megabytes for colour; a video frame may require 1 MB of storage so a video clip lasting a second needs something of the order of 3 0MB. Compression techniques are available to reduce these storage requirements but it will be clear that MM still needs a lot of storage space.

MMDBs require all the basic attributes of a database management system such as a transaction manager, query optimizer, recovery manager etc. as well as special storage structures and specialized search and querying modules.

Typical architecture of a MMDB application

Since existing relational and OO databases comprise the basic requirements of any database, it is natural that many multimedia and imaging DB applications are constructed within such existing systems. In order to support such applications, many DBMS vendors offer facilities suitable for MM. These include:

• long bit and byte strings
• BLOBS
• paths or references of images where the actual image stored elsewhere, such as on an optical storage subsystem. The reasons for this are that document imaging systems need on-line, near-line and off-line storage of images, including archiving. This may be achieved by the use of optical jukeboxes but most commercial DBMSs do not directly support optical storage subsystems (Informix Online/optical is an exception).
• Content retrieval capabilities. In conventional relational and OO DBs querying is based on the attributes of objects. However, information retrieval and document imaging systems require searching the content of documents. This ability can be
  generalized to still images, audio and video.

Characteristics of Multimedia and Imaging Databases

1. Support for imaging and multimedia data types plus
2. the capacity to handle many MM objects plus
3. support for suitable storage management plus
4. database capabilities (transaction/concurrency control, integrity, etc) plus
5. information retrieval capabilities (including exact match and probabilistic retrieval)

Question1
Take each of the above points in turn (except number 3 which will be considered shortly) and describe briefly what is required

Hierarchical Storage Management

It has already been pointed out that MM objects can require a lot of memory for storage. It will also be necessary to have an appropriate storage mechanism so that the system can keep track of objects that are swapped between near-line and on-line and inform the user when an object is stored off-line. In order to do this, the preferred mechanism is that of hierarchical storage management. This is based on the idea of managing a hierarchy of on-line, near-line and off-line storage media. Each of these levels has a particular performance. capacity and cost.

• RAM - best performance, smallest capacity, highest cost, little permanence.
• Hard-drive - good performance, reasonable capacity. fairly high cost, some online storage capabilities.
• optical storage - on-line with a drive or near-line with a jukebox. Acceptable performance when on-line but slow when near-line. High capacity, reasonable cost (less than preceding levels). Can be used for archiving which is permanent e.g. WORMdevices, CD-ROM and recordable compact discs.
• optical media stored off line - stored in cabinets, on shelves etc. Unlimited capacity. very cheap, lasts much longer than magnetic media and therefore good for archiving. Poor performance in the sense that the user has to take the discs off the shelf and put them in the drive!

Diagram

Requirements for a MMDBMS
A prime requirement for a multimedia DBMS is to manage this hierarchy of devices. Through caching and archival capabilities, objects that are no longer accessed are automatically stored on slower media in the hierarchy and eventually may be taken off-line. It is possible for advanced relational, object-relational and object-oriented DBMSs to cope with these requirements to some extent by assigning large segments or clusters to store multimedia objects. The segments are then mapped onto disc volumes and the application developer can use volumes on the storage subsystem to manage the multimedia object. However, there are problems with very large databases. Treating a volume as a large storage space without considering the performance and functionality of the storage subsystem will cause serious problems or result in the system not working at all.

Searching for and retrieving data
Although there are similarities with retrieving data from conventional DBs, in MMDBs it is often necessary to find objects that satisfy the user's query as closely as possible rather than finding an exact match. It should be possible to rank the results of searches. Queries may involve record-attribute searching as well as content-based searching and the query optimizer must take this into account.

Spatial data types
In many MM systems elements may have a spatial relationship with one another e.g. in GISs. The user may wish to query the DB using spatial predicates such as location, position with respect to others (object 1 is to left of object2 or object 1 is
contained within object2, etc.). Such MMDBMSs require support for queries of this type.

Interactive querying
How do you query a MM object? If you know that you want the whole of a particular named image or a particular video clip then the query is straightforward. However, frequently the query may be more complex and require some interactive exchange between the user and the DB as the user attempts to refine his query. In MMDBs it is common to have domains or pick lists of various existing objects so that the user can construct a query interactively. Query-by-example functions may also be available where the user builds an example MM object from existing domain elements. For example, the user may first ask to be shown images of all the types of fabrics stored using a GUI. They might then pick a fabric and say: show me all the fabrics of similar texture and different colours.
Content retrieval is not as precise as querying records or object attributes and various weights may be applied to extracted features. For example, values of important features such as colour, texture or shape are weighted and the distance between these sets of attributes in the sample and returned images is computed. This can be used as a measure of how closely the returned images fit the query example image. Another approach is demonstrated by Excalibur, a system which recognises patterns using a neural network. For example, the user may provide an image, such as a fingerprint, and ask the system to find those images that match it exactly or are nearest to it.

Question 2
Describe some other content-based searches that might be run on a multi-media database.

Automatic feature extraction and indexing
When records are inserted into a relational database the attribute values of the object must be specified precisely. Indexes are normally specified by the user or DB designer. However, for MM systems, tools may be available to extract the important features of MM objects and even automatically produce indexes. For example, with advanced document management systems, paper documents may be converted into scanned digitized images that are subsequently recognized by OCR products and the contents may be automatically indexed. Hence, the bulk of the attribute, or feature, data entry is performed automatically.

Performance issues
MMDBMSs must be able to provide good performance for real-time querying and updating. Some of the features that influence this are:

• indexing - most DBs use single key access structures such as B-trees which can be used for retrieving ranges as well as precise matches. For MM purposes, spatial and multidimensional indexes are also useful. Two dimensional objects have X and Y co-ordinates ( multi-dimensional objects will have more) and special structures such as R trees or grid files will provide better access time.
• content-retrieval indexing - special indexes are required for this. For example, the index for a video could contain the frame number of the start of each clip or scene.
• organising BLOBs - interfaces for BLOBs may allow the user to access and update byte or bit streams and so positional indexes are required to provide fast access to continuous streams of bytes or bits starting at a certain position.
• query optimization - MMDBs are large and manage many complex objects. Query optimization is vital for providing reasonable performance.

Relational Databases and Multi-media

It should be clear by now that, although MMDBMSs require many of the features that are also possessed by relational database management systems, they also have many requirements that are not met by conventional systems. However, in view of the current dominance of relational systems, it is reasonable to consider whether relational DBMSs can be extended to provide at least some MM capabilities.

Variable length fields
Most of the data types supported by relational DBs are fixed length and so the length of each record is also fixed However, many database vendors also provide variable length fields with the aim of supporting at least some MM data types - text, digitized audio, still images etc. (You are familiar with Oracle's VARCHAR2, for example, and various vendors provide data types such as BLOBs, IMAGE. CHARACTER VARYING). Unfortunately, there is no uniformity about these. Even with varying length character fields there are various maximum sizes - some may be as little as 32 or 64 KB per field value which is not much use for most MM purposes (see figures quoted earlier for storage requirements). However, such small fields can be useful for memos or other simple text additions to records.

SQL92 provides some support for variable length attributes and also provides BIT and BIT VARYING for storing bit-mapped graphics. SQL3 will provide more support for MM data types when it finally appears.

Ouestion 3
What support is SQL3 likely to offer for MM data types?

BLOBS
Classical DB theory says that fields in a relational database must be atomic. However, for large MM data it is not reasonable to assume that all the bytes in the long field will be read or updated as a whole. Although many vendors provide support for BLOBS through various data types (BLOB, VARCHAR VARYING. BIT VARYING) there is no single convention for manipulating such fields.

Embedding SQL statements in application programming languages(e.g. C or C-H-). In these applications, data is interchanged between SQL and variables set up in the host programming language. There are several ways of doing this - for example, rows of data may be processed one at a time using a cursor operation or data may be inserted, updated, deleted without using a cursor. An application programming interface (API) is required that controls the exchange of information with the database server that manages the SQL database.

One of the key features of APIs is that they should be able to access BLOBs from a database and integrate them with front-end applications that can manipulate large quantities of space-intensive MM data. Clearly, it would be useful if substrings of the MM data could be retrieved and updated - apart from anything else, this would reduce the overhead (buffer or cache space) and communication.

As mentioned above, various mechanisms are used for retrieving values from a database to a host programming language and there is no need to go into details here. The main issues involved in manipulating these variable-length fields are:

• size is not known in advance
• the data stored in these field may be quite large and cannot be read as a whole. So the data must be managed a piece at a time and byte strings read as needed.

Examples of MM systems based on Relational DBs

InterBase
This relational database system has built in support for BLOBs. The BLOBs are stored in collections of 'segments' - the size of which can be specified by the user and are basically a fixed length 'page' or I/O block.. InterBase has a proprietary high-level language programme interface as well as a standard SQL interface. The individual segments that comprise a BLOB can be read and or updated.

Sybase SQL server allows users to declare columns as TEXT and IMAGE data types which can be very large (2GB). Sybase has an enhanced version of SQL called TransactSQL which allows some manipulation of the TEXT and IMAGE data types such as finding the first occurrence of a particular "pattern" in the column. The column values of TEXT/IMAGE contain pointers to the first page of the MM column and these pages are stored separately from the tables for the database. The pages on which the object is stored form a linked list.

XDP from Plexus
This is an imaging database engine based on the INFORMIX Turbo relational DBMS. Unlike Sybase, this system does support hierarchical storage subsystems and manages magnetic discs, optical discs, and optical jukeboxes with on-line, near-line and off-line facilities. Records and the images associated with them are stored in different locations but both image/text structures and records can be manipulated and updated consistently in the same transaction.

Question 4
Review the main limitations of using a relational database as the basis for a
multimedia database system.