Oracle8i Application Developer's Guide - Advanced Queuing
Release 8.1.5

A68005-01

Library

Product

Contents

Index

Prev Next

1
Introduction

In this chapter we introduce Oracle Advanced Queuing (AQ) by considering the requirements for complex information handling in a distributed environment under the following headings:

The Need for Queuing in Messaging Systems

Consider the following application scenario.

The operations of a large bookseller, BooksOnLine, are based on an online book ordering system which automates activities across the various departments involved in the entire sale process.The front end of the system is an order entry application which is used to enter new orders.These incoming orders are processed by an order processing application which validates and records the order. Shipping departments located at regional warehouses are then responsible for ensuring that these orders are shipped in a timely fashion. There are three regional warehouses: one serving the East Region, one serving the West Region, and a third warehouse for shipping International orders. Once an order has been shipped, the order information is routed to a central billing department which handles payment processing.The customer service department, located at its own site, is responsible for maintaining order status and handling inquiries about orders.

Message Systems

This scenario describes an application in which messages come from and are disbursed to multiple clients (nodes) in a distributed computing environment. Messages are not only passed back and forth between clients and servers but are also intricately interleaved between processes on different servers. The integration of the various component applications consist of multi-step processes in which each step is triggered by one or more messages, and which may then give rises to one or more messages.

These applications can be viewed as message systems. For instance, the application should be able to implement content-based routing, content-based subscription, and content-based querying.

Such message systems need to exhibit high performance characteristics as might be measured by the following metrics:

Message systems should also exhibit high scalability. A system should continue to exhibit high performance as the number of programs using the application increase, as the number of messages increase, and as the size of the message warehouse increases.

Synchronous Communication as an Application Model

One way of modeling this intercommunication of messages, termed synchronous, on-line or connected, is based on the request-reply paradigm. In this model a program sends a request to another program and waits (blocks) until the reply arrives. This close coupling of the sender and receiver of the message is suitable for programs that need to get a reply before they can proceed.

Traditional client/server architectures are based on this model.Its major drawback is that all the component programs must be available and running for the application to work. In the event of network or machine failure, or even if the needed program is busy, the entire application grinds to a halt.

Asynchronous Messaging as an Application Model

In the asynchronous, disconnected or deferred model programs in the role of producers place messages in a queue and then proceed with their work. Programs in the role of consumers retrieve requests from the queue and act on them. This model is well suited for applications that can continue with their work after placing a request in the queue because they are not blocked waiting for a reply. It is also suited to applications that can continue with their work until there is a message to retrieve.This decoupling of 'requests for service' from 'supply of services' increases efficiency, and provides the infrastructure for complex scheduling.

Message Persistence

Handling an intricate scheduling of message-passing is not the only challenge. Unfortunately, networks, computing hardware, and software applications will all fail from time to time. For deferred execution to work correctly in the presence of network, machine and application failures, messages that constitute requests for service must be stored persistently, and processed exactly once.In other words, messaging must be persistent.

Being able to preserve messages is fundamental. Applications may have to deal with multiple unprocessed messages arriving simultaneously from external clients or from programs internal to the application, and in such situations they may not have the necessary resources. Similarly, the communication links between databases may not be available all the time or may be reserved for some other purpose. If the system falls short in its capacity to deal with these messages immediately, the application must be able to store the messages until they can be processed. By the same token, external clients or internal programs may not be ready to receive messages that have been processed.

Even more importantly, messaging systems need message persistence so they can deal with priorities: messages arriving later may be of higher priority than messages arriving earlier; messages arriving earlier may have to wait for messages arriving later before actions are executed; the same message may have to be accessed by different processes; and so on. Such priorities may not be fixed. One crucial dimension of handling the dynamic aspect of message persistence has to do with windows of opportunity that grow and shrink.It may be that messages in a specific queue become more important than messages in other queues, and so need to be processed with less delay or interference from messages in other queues. Similarly, it may be more pressing to send messages to some destinations than to others.

Finally, message persistence is crucial because the control component of the message can be as important as the payload data. For instance, the time that messages are received or dispatched can be a crucial part of the message. It may be central to analyzing periods of greatest demand, or for evaluating the lag between receiving and completing an order, and son on. Put more formally: the message may need to retain importance as a business asset after it has been executed.Tracking and documentation should be the responsibility of the messaging system, not the developer.

Features of Advanced Queuing (AQ)

By integrating transaction processing with queuing technology, persistent messaging in the form of Advanced Queuing is made possible. The following overview considers the features of Oracle AQ under four headings:

General Features

The following features apply to all aspects of Oracle AQ.

SQL Access

Messages are placed in normal rows in a database table, and so can be queried using standard SQL. This means that you can use SQL to access the message properties, the message history and the payload. All available SQL technology, such as indexes, can be used to optimize the access to messages.

Integrated Database Level Operational Support

Standard database features such as recovery, restart and enterprise manager are supported. Oracle AQ queues are implemented in database tables, hence all the operational benefits of high availability, scalability and reliability are applicable to queue data. In addition, database development and management tools can be used with queues. For instance, queue tables can be imported and exported.

Structured Payload

Users can use object types to structure and manage message payloads. RDBMSs in general have had a far richer typing system than messaging systems. Since Oracle8i is an object-relational DBMS, it supports both traditional relational types as well as user-defined types. Many powerful features are enabled as a result of having strongly typed content i.e. content whose format is defined by an external type system. These include:

To see this feature applied in the context of the BooksOnLine scenario, refer to Structured Payload in Chapter 2, "Implementing AQ -- A Sample Application")

Retention and Message History

Users of AQ can specify that messages be retained after consumption. The systems administrator can specify the duration for which messages will be retained. Oracle AQ stores information about the history of each message, preserving the queue and message properties of delay, expiration, and retention for messages destined for local or remote recipients. The information contains the ENQUEUE/DEQUEUE time and the identification of the transaction that executed each request. This allows users to keep a history of relevant messages. The history can be used for tracking, data warehouse and data mining operations.

To see this feature applied in the context of the BooksOnLine scenario, refer to Retention and Message History)

Tracking and Event Journals

If messages are retained they can be related to each other. For example: if a message m2 is produced as a result of the consumption of message m1, m1 is related to m2. This allows users to track sequences of related messages. These sequences represent 'event journals' which are often constructed by applications. Oracle AQ is designed to let applications create event journals automatically.

Integrated Transactions

The integration of control information with content (data payload) simplifies application development and management.

Queue Level Access Control

With Oracle 8i, an owner of an 8.1 style queue can grant or revoke queue level privileges on the queue. DBAs can grant or revoke new AQ system level privileges to any database user. DBAs can also make any database user an AQ administrator.

To see this feature applied in the context of the BooksOnLine scenario, refer to Queue Level Access Control).

Non-Persistent Queues

AQ can deliver non-persistent messages asynchronously to subscribers. These messages can be event-driven and do not persist beyond the failure of the system (or instance). AQ supports persistent and non-persistent messages with a common API.

To see this feature applied in the context of the BooksOnLine scenario, refer to Non-Persistent Queues).

Publish/ Subscribe Support

A combination of features are introduced to allow a publish/subscribe style of messaging between applications. These features include rule-based subscribers, message propagation, the listen feature and notification capabilities.

Support for OPS Environments

With Oracle8i release 8.1.5, an application can specify the instance affinity for a queue-table. When AQ is used with parallel server and multiple instances, this information is used to partition the queue-tables between instances for queue-monitor scheduling. The queue-table is monitored by the queue-monitors of the instance specified by the user. If an instance affinity is not specified, the queue-tables will be arbitrarily partitioned among the available instances. There can be 'pinging' between the application accessing the queue-table and the queue-monitor monitoring it. Specifying the instance-affinity does not prevent the application from accessing the queue-table and its queues from other instances.

This feature prevents 'pinging' between queue monitors and AQ propagation jobs running in different instances. In Oracle8i release 8.1.5 an instance affinity (primary and secondary) can be specified for a queue table. When AQ is used with parallel server and multiple instances, this information is used to partition the queue-tables between instances for queue-monitor scheduling as well as for propagation. At any time, the queue table is affiliated to one instance. In the absence of an explicitly specified affinity, any available instance is made the owner of the queue table. If the owner of the queue table dies, the secondary instance or some available instance takes over the ownership for the queue table.

To see this feature applied in the context of the BooksOnLine scenario, refer to Support for Oracle Parallel Server (OPS)).

Support for Statistics Views

Basic statistics about queues in the database are available via the GV$AQ view.

Reliability and Recoverability

The standard database reliability and recoverability characteristics apply to queue data.

ENQUEUE Features

The following features apply to the process of producing messages by enqueuing them into a queue.

Correlation Identifier

Users can assign an identifier to each message, thus providing a means to retrieve specific messages at a later time.

Subscription & Recipient Lists

A single message can be designed to be consumed by multiple consumers. A queue administrator can specify the list of subscribers who can retrieve messages from a queue. Different queues can have different subscribers, and a consumer program can be a subscriber to more than one queue. Further, specific messages in a queue can be directed toward specific recipients who may or may not be subscribers to the queue, thereby overriding the subscriber list.

You can design a single message for consumption by multiple consumers in a number of different ways. The consumers who are allowed to retrieve the message are specified as explicit recipients of the message by the user or application that enqueues the message. Every explicit recipient is an agent identified by name, address and protocol.

A queue administrator may also specify a default list of recipients who can retrieve all the messages from a specific queue. These implicit recipients become subscribers to the queue by being specified in s default list. If a message is enqueued without specifying any explicit recipients, the message is delivered to all the designated subscribers.

A rule-based subscriber is one that has a rule associated with it in the default recipient list. A rule based subscriber will be sent a message with no explicit recipients specified only if the associated rule evaluated to TRUE for the message. Different queues can have different subscribers, and the same recipient can be a subscriber to more than one queue. Further, specific messages in a queue can be directed toward specific recipients who may or may not be subscribers to the queue, thereby over-riding the subscriber list.

A recipient may be specified only by its name, in which case the recipient must dequeue the message from the queue in which message was enqueued. It may be specified by its name and an address with a protocol value of 0. The address should be the name of another queue in the same database or another Oracle8i database (identified by the database link) in which case the message is propagated to the specified queue and can be dequeued by a consumer with the specified name. If the recipient's name is NULL, the message is propagated to the specified queue in the address and can be dequeued by the subscribers of the queue specified in the address. If the protocol field is nonzero, the name and address field is not interpreted by the system and the message can be dequeued by special consumer (see third party support in the propagation section).

To see this feature applied in the context of the BooksOnLine scenario, refer to Subscriptions and Recipient Lists).

Priority and Ordering of Messages in Enqueuing

It is possible to specify the priority of the enqueued message. An enqueued message can also have its exact position in the queue specified. This means that users have three options to specify the order in which messages are consumed: (a) a sort order specifies which properties are used to order all message in a queue; (b) a priority can be assigned to each message; (c) a sequence deviation allows you to position a message in relation to other messages. Further, if several consumers act on the same queue, a consumer will get the first message that is available for immediate consumption. A message that is in the process of being consumed by another consumer will be skipped.

To see this feature applied in the context of the BooksOnLine scenario, refer to Priority and Ordering of Messages).

Message Grouping

Messages belonging to one queue can be grouped to form a set that can only be consumed by one user at a time. This requires the queue be created in a queue table that is enabled for message grouping. All messages belonging to a group have to be created in the same transaction and all messages created in one transaction belong to the same group. This feature allows users to segment complex messages into simple messages, e.g., messages directed to a queue containing invoices could be constructed as a group of messages starting with the header message, followed by messages representing details, followed by the trailer message.

To see this feature applied in the context of the BooksOnLine scenario, refer to Message Grouping).

Propagation

This feature enables applications to communicate with each other without having to be connected to the same database or to the same Queue. Messages can be propagated from one Oracle AQ to another, irrespective of whether these are local or remote. The propagation is done using database links, and Net8.

To see this feature applied in the context of the BooksOnLine scenario, refer to Asynchronous Notifications).

Sender Identification

Applications can mark the messages they send with a custom identification. Oracle also automatically identifies the queue from which a message was dequeued. This allows applications to track the pathway of a propagated message, or of a string messages within the same database.

Time Specification and Scheduling

Delay interval and/or expiration intervals can be specified for an enqueued message, thereby providing windows of execution. A message can be marked as available for processing only after a specified time elapses (a delay time) and has to be consumed before a specified time limit expires.

Rule-based Subscribers

A message can be delivered to multiple recipients based on message properties or message content. Users define a rule based subscription for a given queue as the mechanism to specify interest in receiving messages of interest. Rules can be specified based on message properties and message data (for object and raw payloads). Subscriber rules are then used to evaluate recipients for message delivery.

To see this feature applied in the context of the BooksOnLine scenario, refer to Rule-based Subscription).

Asynchronous Notification

OCI clients can use the new call OCISubscriptionRegister to register a callback for message notification. The client issues a registration call which specifies a subscription name and a callback. When messages for the subscription are received, the callback is invoked. The callback may then issue an explicit dequeue to retrieve the message.

To see this feature applied in the context of the BooksOnLine scenario, refer to Asynchronous Notifications).

DEQUEUE Features

Multiple Recipients

A message in queue can be retrieved by multiple recipients without there being multiple copies of the same message.

To see this feature applied in the context of the BooksOnLine scenario, refer to Multiple Recipients).

Local and Remote Recipients

Designated recipients can be located locally and/or at remote sites.

To see this feature applied in the context of the BooksOnLine scenario, refer to Local and Remote Recipients).

Navigation of Messages in Dequeuing

Users have several options to select a message from a queue. They can select the first message or once they have selected a message and established a position, they can retrieve the next. The selection is influenced by the ordering or can be limited by specifying a correlation identifier. Users can also retrieve a specific message using the message identifier.

To see this feature applied in the context of the BooksOnLine scenario, refer to Message Navigation in Dequeue).

Modes of Dequeuing

A DEQUEUE request can either browse or remove a message. If a message is browsed it remains available for further processing, if a message is removed, it is not available any more for DEQUEUE requests. Depending on the queue properties a removed message may be retained in the queue table.

To see this feature applied in the context of the BooksOnLine scenario, refer to Modes of Dequeuing).

Optimization of Waiting for the Arrival of Messages

A DEQUEUE could be issued against an empty queue. To avoid polling for the arrival of a new message a user can specify if and for how long the request is allowed to wait for the arrival of a message.

To see this feature applied in the context of the BooksOnLine scenario, refer to Optimization of Waiting for Arrival of Messages).

Retries with Delays

A message has to be consumed exactly once. If an attempt to dequeue a message fails and the transaction is rolled back, the message will be made available for reprocessing after some user specified delay elapses. Reprocessing will be attempted up to the user-specified limit.

To see this feature applied in the context of the BooksOnLine scenario, refer to Retry with Delay Interval).

Optional Transaction Protection

ENQUEUE/DEQUEUE requests are normally part of a transaction that contains the requests, thereby providing the desired transactional behavior. Users can, however, specify that a specific request is a transaction by itself making the result of that request immediately visible to other transactions. This means that messages can be made visible to the external world either as soon as the ENQUEUE or DEQUEUE statement is issued, or only after the transaction is committed.

Exception Handling

A message may not be consumed within given constraints, i.e. within the window of execution or within the limits of the retries. If such a condition arises, the message will be moved to a user-specified exception queue.

To see this feature applied in the context of the BooksOnLine scenario, refer to Exception Handling).

Listen Capability (Wait on Multiple Queues)

The listen call is a blocking call that can be used to wait for messages on multiple queues. It can be used by a gateway application to monitor a set of queues. An application can also use it to wait for messages on a list of subscriptions. If the listen returns successfully, a dequeue must be used to retrieve the message.

To see this feature applied in the context of the BooksOnLine scenario, refer to Listen Capability).

Dequeue Message Header with No Payload

The new dequeue mode REMOVE_NODATA can be used to remove a message from a queue without retrieving the payload. This mode will be useful for applications that want to delete messages with huge payloads and aren't interested in the payload contents.

Propagation Features

Automated Coordination of Enqueuing and Dequeuing

As already noted, recipients can be local or remote. Oracle8i does not support distributed object types, hence remote enqueuing or dequeuing using a standard database link does not work. However, you can use AQ's message propagation to enqueue to a remote queue.

For example, you can connect to database X and enqueue the message in a queue, say "DROPBOX" located in database X. You can configure AQ so that all messages enqueued in queue "DROPBOX" will be automatically propagated to another queue in a database Y, regardless whether database Y is local or remote. AQ will automatically check if the type of the remote queue in database Y is structurally equivalent to the type of the local queue in database X, and propagate the message.

Recipients of propagated messages can be either applications or queues. If the recipient is a queue, the actual recipients will be determined by the subscription list associated with the recipient queue.If the queues are remote, messages will be propagated using the specified database link. Only AQ to AQ message propagation is supported.

Propagation of Messages with LOBs

Propagation handles payloads with LOB attributes.

To see this feature applied in the context of the BooksOnLine scenario, refer to Propagation of Messages with LOB Attributes).

Propagation Scheduling

Messages can be scheduled to propagate from a queue to local or remote destinations. Administrators can specify the start time, the propagation window and a function to determine the next propagation window (for periodic schedules).

Enhanced Propagation Scheduling Capabilities

Detailed run-time information about propagation is gathered and stored in the DBA_QUEUE_SCHEDULES view for each propagation schedule. This information can be used by queue designers and administrators to fix problems or tune performance. For example, available statistics about the total and average number of message/bytes propagated can be used to tune schedules. Similarly, errors reported by the view can be used to diagnose and fix problems. The view also describes additional information such as the session ID of the session handling the propagation, and the process name of the job queue process handling the propagation. To see this feature applied in the context of the BooksOnLine scenario, refer to Enhanced Propagation Scheduling Capabilities).

Third Party Support

Advanced Queueing allows messages to be enqueued in queues that can then be propagated to different messaging systems by third party propagators. If the protocol number for a recipient is in the range 128 - 255, the address of the recipient is not interpreted by AQ and so the message is not propagated by the Advanced Queuing system. Instead a third party propagator can then dequeue the message by specifying a reserved consumer name in the dequeue operation. The reserved consumer names are of the form AQ$_P# where # is the protocol number in the range 128 - 255. For example, the consumer name AQ$_P128 can be used to dequeue messages for recipients with protocol number 128. The list of recipients for a message with the specific protocol number is returned in the recipient_list message property on dequeue.

Primary Components of Advanced Queuing (AQ)

By integrating transaction processing with queuing technology, persistent messaging in the form of Advanced Queuing is made possible.

Message

A message is the smallest unit of information inserted into and retrieved from a queue. A message consists of

The control information represents message properties used by AQ to manage messages. The payload data is the information stored in the queue and is transparent to Oracle AQ. A message can reside in only one queue. A message is created by the enqueue call and consumed by the dequeue call.

Queue

A queue is a repository for messages. There are two types of queues: user queues, also known as normal queues, and exception queues. The user queue is for normal message processing. Messages are transferred to an exception queue if they can not be retrieved and processed for some reason. Queues can be created, altered, started, stopped, and dropped by using the Oracle AQ administrative interfaces (see Chapter 4, "Administrative Interface: Basic Operations").

Queue Table

Queues are stored in queue tables. Each queue table is a database table and contains one or more queues. Each queue table contains a default exception queue. Figure 1-1, "Basic Queues" shows the relationship between messages, queues, and queue tables.

Agent

An agent is a queue user. This could be an end user or an application.There are two types of agents:

Any number of producers and consumers may be accessing the queue at a given time. Agents insert messages into a queue and retrieve messages from the queue by using the Oracle AQ operational interfaces (see Chapter 6, "Operational Interface: Basic Operations")

An agent is identified by its name, address and protocol (see "Agent" in Chapter 3, "Managing Oracle AQ" for formal description of this data structure).

Recipient

The recipient of a message may be specified by its name only, in which case the recipient must dequeue the message from the queue in which the message was enqueued. The recipient may be specified by name and an address with a protocol value of 0. The address should be the name of another queue in the same database or another Oracle8 database (identified by the database link) in which case the message is propagated to the specified queue and can be dequeued by a consumer with the specified name. If the recipient's name is NULL, the message is propagated to the specified queue in the address and can be dequeued by the subscribers of the queue specified in the address. If the protocol field is nonzero, the name and address field is not interpreted by the system and the message can be dequeued by special consumer (see third party support in the propagation section).

Recipient and Subscription Lists

A single message can be designed for consumption by multiple consumers. There are two ways to do this.

Different queues can have different subscribers, and the same recipient can be a subscriber to more than one queue. Further, specific messages in a queue can be directed toward specific recipients who may or may not be subscribers to the queue, thereby over-riding the subscriber list.

Rule

A rule is used to define one or more subscribers' interest in subscribing to messages that conform to that rule. The messages that meet this criterion are then delivered to the interested subscribers. Put another way: a rule filters for messages in a queue on a topic in which a subscriber is interested.

A rule is specified as a boolean expression (one that evaluates to true or false) using syntax similar to the WHERE clause of a SQL query. This boolean expression can include conditions on

Rule Based Subscriber

A rule-based subscriber is a subscriber that has rule associated with it in the default recipient list. A rule-based subscriber is sent a message that has no explicit recipients specified if the associated rule evaluates to TRUE for the message.

Queue Monitor

The queue monitor (QMNn) is a background process that monitors the messages in the queues. It provides the mechanism for message delay, expiration and retry delay. The also QMNn also performs garbage collection for the queue table and its indexes and index-organized tables. It is possible to start a maximum of 10 multiple queue monitors at the same time. You start the desired number of queue monitors by setting the dynamic init.ora parameter aq_tm_processes. The queue monitor wakes up every minute, or whenever there is work to be done, for instance, if a message is to be marked as expired or as ready to be processed.

Modeling Queue Entities

Figure 1-1 Basic Queues


The preceding figure portrays a queue table that contains two queues, and one exception queue:

Basic Queuing

Basic Queuing -- One Producer, One Consumer

At its most basic, one producer may enqueue different messages into one queue. Each message will be dequeued and processed once by one of the consumers. A message will stay in the queue until a consumer dequeues it or the message expires. A producer may stipulate a delay before the message is available to be consumed, and a time after which the message expires. Likewise, a consumer may wait when trying to dequeue a message if no message is available. Note that an agent program, or application, can act as both a producer and a consumer.

Basic Queuing -- Many Producers, One Consumer

At a slightly higher level of complexity, many producers may enqueue messages into a queue, all of which are processed by one consumer.

Basic Queuing -- Many Producers, Many Consumers of Discrete Messages

In this next stage, many producers may enqueue messages, each message being processed by a different consumer depending on type and correlation identifier. The figure below shows this scenario.

Illustrating Basic Queuing

Figure Figure 1-2, "Modeling Basic Queuing" (below) portrays a queue table that contains one queue into which messages are being enqueued and from which messages are being dequeued.

Producers

The figure indicates that there are 6 producers of messages, although only four are shown. This assumes that two other producers (P4 and P5) have the right to enqueue messages even though there are no messages enqueued by them at the moment portrayed by the figure. The figure shows:

Consumers

According to the figure, there are 3 consumers of messages, representing the total population of consumers. The figure shows:

Figure 1-2 Modeling Basic Queuing


Illustrating Client-Server Communication Using AQ

The previous figure portrayed the enqueuing of multiple messages by a set of producers, and the dequeuing of messages by a set of consumers. What may not be readily evident in that sketch is the notion of time, and the advantages offered by Oracle AQ.

Client-Server applications normally execute in a synchronous manner, with all the disadvantages of that tight coupling described above. Figure 1-3, "Client-Server Communication Using AQ" demonstrates the asynchronous alternative using AQ. In this example Application B (a server) provides service to Application A (a client) using a request/response queue.

Figure 1-3 Client-Server Communication Using AQ


  1. Application A enqueues a request into the request queue.

  2. Application B dequeues the request.

  3. Application B processes the request.

  4. Application B enqueues the result in the response queue.

  5. Application A dequeues the result from the response queue.

In this way the client does not have to wait to establish a connection with the server, and the server dequeues the message at its own pace. When the server is finished processing the message, there is no need for the client to be waiting to receive the result. In this way a process of double-deferral frees both client and server.


Note:

The various enqueue and dequeue operations are part of different transactions.  


Multiple-Consumer Dequeuing of the Same Message

A message can only be enqueued into one queue at a time. If a producer had to insert the same message into several queues in order to reach different consumers, this would require management of a very large number of queues. Oracle AQ provides two mechanisms to allow for multiple consumers to dequeue the same message: queue subscribers and message recipients. The queue must reside in a queue table that is created with multiple consumer option to allow for subscriber and recipient lists. Each message remains in the queue until it is consumed by all its intended consumers.

Queue Subscribers

Using this approach, multiple consumer-subscribers are associated with a queue. This will cause all messages enqueued in the queue to be made available to be consumed by each of the queue subscribers. The subscribers to the queue can be changed dynamically without any change to the messages or message producers. Subscribers to the queue are added and removed by using the Oracle AQ administrative package. The diagram below shows multiple producers enqueuing messages into queue, each of which is consumed by multiple consumer-subscribers.

Message Recipients

A message producer can submit a list of recipients at the time a message is enqueued. This allows for a unique set of recipients for each message in the queue. The recipient list associated with the message overrides the subscriber list associated with the queue, if there is one. The recipients need not be in the subscriber list. However, recipients may be selected from among the subscribers.

Figure 1-4 Multiple-Consumer Dequeuing of the Same Message


Illustrating Multiple-Consumer Dequeuing of the Same Message

Figure 1-4 describes the case in which three consumers are all listed as subscribers of a queue. This is the same as saying that they all subscribe to all the messages that might ever be enqueued into that queue. The drawing illustrates a number of important points:

Figure 1-5 Communication Using a Multi-Consumer Queue


Figure 1-5 illustrates the same technology from a dynamic perspective. This examples concerns a scenario in which more than one application needs the result produced by an application. Every message enqueued by Application A is dequeued by Application B and Application C. To make this possible, the multiple consumer queue is specially configured with Application B and Application C as queue subscribers. Consequently, they are implicit recipients of every message placed in the queue.


Note:

Queue subscribers can be applications or other queues.  


Figure 1-6 Dequeuing of Specified Messages by Specified Recipients


Illustrating Dequeuing of Specified Messages by Specified Recipients

Figure 1-6 shows how a message can be specified for one or more recipients. In this case, Message 5 is specified to be dequeued by Recipient-1 and Recipient-2. As described by the drawing, neither of the recipients is one of the 3 subscribers to the queue.

Figure 1-7 Explicit and Implicit Recipients of Messages


We earlier referred to subscribers as "implicit recipients" in that they are able to dequeue all the messages placed into a specific queue. This is like subscribing to a magazine and thereby implicitly gaining access to all its articles. The category of consumers that we have referred to as recipients may also be viewed as "explicit recipients" in that they are designated targets of particular messages.

Figure 1-7 shows how Oracle AQ can adjust dynamically to accommodate both kinds of consumers. In this scenario Application B and Application C are implicit recipients (subscribers). But messages can also be explicitly directed toward specific consumers (recipients) who may or may not be subscribers to the queue. The list of such recipients is specified in the enqueue call for that message and overrides the list of subscribers for that queue. In the figure, Application D is specified as the sole recipient of a message enqueued by Application A.


Note:

Multiple producers may simultaneously enqueue messages aimed at different targeted recipients.  


Illustrating the Implementation of Workflows using AQ

Figure 1-8 illustrates the use of AQ for implementing workflows, also knows as chained application transactions. It shows a workflow consisting of 4 steps performed by Applications A, B, C and D. The queues are used to buffer the flow of information between different processing stages of the business process. By specifying delay interval and expiration time for a message, a window of execution can be provided for each of the applications.

Figure 1-8 Implementing Workflows using AQ


From a workflow perspective, the passing of messages is a business asset above and beyond the value of the payload data. Hence, AQ supports the optional retention of messages for analysis of historical patterns and prediction of future trends. For instance, two of the three application scenarios at the head of the chapter are founded in an implementation of workflow analysis.


Note:

The contents of the messages 1, 2 and 3 can be the same or different. Even when they are different, messages may contain parts of the of the contents of previous messages.  


Illustrating the Implementation of Publish/Subscribe using AQ

Figure 1-9 illustrates the use of AQ for implementing a publish/subscribe messaging scheme between applications. Application A is a publisher application which is publishing messages to a queue. Applications B, C, D are subscriber applications. Application A publishes messages anonymously to a queue. These messages are then delivered to subscriber applications based on the rules specified by each application. Subscriber applications can specify interest in messages by defining a rule on message properties and message data content.

In the example shown, applications B has subscribed with rule "priority=1", application C has subscribed with rule "priority > 1" and application D has subscribed with rule "priority = 3". Application A enqueues 3 messages (priority 3, 1, 2). Application B receives a single message (priority 1), application C receives two messages (priority 2, 3) and application D receives a single message (priority 3). Thus, message recipients are computed dynamically based on message properties and content. Additionally, the figure also illustrates how application D uses asynchronous notification for message delivery. Application D registers for messages on the queue. When messages arrive, application D is notified and can then dequeue the messages.

Figure 1-9 Implementing Publish/Subscribe using AQ


From a workflow perspective, the passing of messages is a business asset above and beyond the value of the payload data. Hence, AQ supports the optional retention of messages for analysis of historical patterns and prediction of future trends. For instance, two of the three application scenarios at the head of the chapter are founded in an implementation of workflow analysis.

Message Propagation

Fanning-Out of Messages

In AQ, message recipients can be either consumers or other queues. If the message recipient is a queue, the actual recipients are determined by the subscribers to the queue (which may in turn be other queues). Thus it is possible to fan-out messages to a large number of recipients without requiring them all to dequeue messages from a single queue.

For example: A queue, Source, may have as its subscribers queues dispatch1@dest1 and dispatch2@dest2. Queue dispatch1@dest1 may in turn have as its subscribers the queues outerreach1@dest3 and outerreach2@dest4, while queue dispatch2@dest2 has as subscribers the queue outerreach3@dest21 and outerreach4@dest4. In this way, messages enqueued in Source will be propagated to all the subscribers of four different queues.

Funneling-in of Messages

Another use of queues as a message recipient is the ability to combine messages from different queues into a single queue. This process is sometimes described as "compositing"

For example, if queue composite@endpoint is a subscriber to both queues funnel1@source1 and funnel2@source2 then the subscribers to queue composite@endpoint can get all messages enqueued in those queues as well as messages enqueued directly into itself.

Figure 1-10 Message Propagation


Illustration of Message Propagation

Figure 1-10 illustrates applications on different databases communicating via AQ. Each application has an inbox and an outbox for handling incoming and outgoing messages. An application enqueues a message into its outbox irrespective of whether the message has to be sent to an application that is local (on the same node) or remote (on a different node).

Likewise, an application is not concerned as to whether a message originates locally or remotely. In all cases, an application dequeues messages from its inbox.

Oracle AQ facilitates all this interchange, treating messages on the same basis.

Programmatic Environments for Working with AQ

Oracle now offers you different environments for working with AQ:

AQ and XA

You must specify "Objects=T" in the xa_open string if you want to use the AQ OCI interface. This forces XA to initialize the client side cache in Objects mode. You do not need to do this if you plan to use AQ through PL/SQL wrappers from OCI or Pro*C.

You must use AQ navigation option carefully when you are using AQ from XA. XA cancels cursor fetch state after an xa_end. Hence, if you want to continue dequeuing between services (i.e. xa_start/xa_end boundaries) you must reset the dequeue position by using the FIRST_MESSAGE navigation option. Otherwise you will get an ORA-25237 (navigation used out of sequence).


For more information about deploying AQ with XA, see:

 

Compatibility

Certain features only will function if compatibility is set to '8.1'. As shown in Table 1-1, you may have to set the compatible parameter of the init.ora and/or the compatible parameter of the queue table.

Table 1-1 Compatibility Settings Required to Make Use of New Features
Feature  Init.ora compatible = ' 8.1.x'  queue table compatible = ' 8.1' 

Queue Level Access Control  

X  

X  

Non-Persistent Queues  

X  

automatically created  

Support for OPS Environments  

X  

 

Rule-based Subscribers for Publish/Subscribe  

X  

X  

Asynchronous Notification  

X  

 

Sender Identification  

X  

X  

Separate storage of history management information  

X  

X  


For more information, see:

 

Restrictions

The following restrictions currently apply.

Auto-commit features in DBMS_AQADM package

The auto_commit parameters in CREATE_QUEUE_TABLE, DROP_QUEUE_TABLE, CREATE_QUEUE, DROP_QUEUE and ALTER_QUEUE calls in DBMS_AQADM package are deprecated for 8.1.5 and subsequent releases. Oracle continues to support this parameter in the interface for backward compatibility purpose.

Collection Types in Message Payloads

You cannot construct a message payload using a collection type that is not itself contained within an object. You also cannot currently use a nested table even as an embedded object within a message payload. However, you can create an object type that contains one or more VARRAYs, and create a queue table that is founded on this object type.

For example, the following operations are allowed:

CREATE TYPE number_varray AS VARRAY(32) OF NUMBER; 
CREATE TYPE embedded_varray AS OBJECT (col1 number_varray); 
EXECUTE DBMS_AQADM.CREATE_QUEUE_TABLE( 
  queue_table           =>     'QT', 
  queue_payload_type    =>     'embedded_varray'); 
 

Object Type Payload Support in AQ Java API

The AQ Java classes in release 8.1.5 does not allow enqueuing and dequeuing object type payloads, only raw type payloads are supported.

Synonyms on Queue Tables and Queue

All AQ PL/SQL calls do not resolve synonyms on queues and queue tables. Even though you can create a synonyms, you should not apply the synonym to the AQ interface.

Pluggable Tablespace does not Work For 8.0 Compatible Multiconsumer Queues

Any tablespace which contains 8.0 compatible multiconsumer queue tables should not be transported using the pluggable tablespace mechanism. The mechanism will work, however, with tablespaces that contain only single consumer queues as well as 8.1 compatible multiconsumer queues. Before you can export a tablespace in pluggable mode, you have to alter the tablespace to read-only mode. If you try to import a read-only tablespace which contain 8.0 compatible multiconsumer queues, you will get an Oracle error indicating that you cannot update the queue table index at import time.

Tablespace point-in-time recovery

AQ currently does not support tablespace point in time recovery. Creating a queue table in a tablespace will disable that particular tablespace for point-in-time recovery.

Propagation from Object Queues

Note that AQ does not support propagation from Object queues that have BFILE or REF attributes in the payload.

Non-Persistent Queues

Currently you can create only non-persistent queues of RAW type.You are limited in that you can send messages only to subscribers and explicitly specified recipients who are local. Propagation is not supported from non-persistent queues. And in retrieving messages, you cannot use the dequeue call but must instead employ the asynchronous notification mechanism, registering for the notification by mean of OCISubcriptionRegister.

Reference to Demos

The following demos may be found in the $ORACLE_HOME/demo directory.:

Table 1-2
Demo & Locations  Topic 

aqdemo00.sql  

Main driver of demo  

aqdemo01.sql  

Create queue tables and queues using AQ administration interface  

aqdemo02.sql  

Load the demo package  

aqdemo03.sql  

Submit the event handler as a job to Job Queue  

aqdemo04.sql  

Enqueue messages  

newaqdemo00.sql  

Create users, message types, tables etc.  

newaqdemo01.sql  

Set up queue_tables, queues, subscribers and set up  

newaqdemo02.sql  

Enqueue messages  

newaqdemo03.sql  

Installs dequeue procedures  

newaqdemo04.sql  

Performs 'blocking dequeue'  

newaqdemo05.sql  

Performs 'listen' for multiple agents  

newaqdemo06.sql  

Cleans up users, queue_tables, queues, subscribers etc. (cleanup script)  

ociaqdemo00.c  

Enqueue messages  

ociaqdemo01.c  

Performs blocking dequeue  

ociaqdemo02.c  

Performs 'Listen' for multiple agents  




Prev

Next
Oracle
Copyright © 1999 Oracle Corporation.

All Rights Reserved.

Library

Product

Contents

Index