TECHNOTE 1059: Open Transport Application Performance, P3

On Improving Open Transport Network Application Performance

Open Transport Event Processing

Open Transport on Mac OS
The Mac OS Open Transport API is a superset of the industry standard X/Open Transport Interface (XTI) specification. XTI orginated in a UNIX preemptive multitasking environment, where it is quite acceptable for a task to issue blocking I/O requests which can cause a process to sleep until the request is completed. However since the current Mac OS utilizes a cooperative rather than preemptive multitasking architecture, blocking I/O is unacceptable. In order to adress this, Apple extended the XTI API to support asynchronous notfication of I/O completion.
Creating the kind of applications that take full advantage of Open Transport on the Macintosh, such as high performance network servers, necessitate the employment of these XTI extensions. Understanding these extensions is important to not only developers who are writing new Macintosh network server applications but also to those who attempt import code based on UNIX sockets.

Event Processing Strategies
Open Transport uses an event mechanism to notify your application that something has occurred which demands it's immediate attention, such as data arival or connection / disconnection requests. Open Transport events can also be used to signal that a function executing asynchronously has completed. These network events should not be confused with the kinds of events processed by the Event Manager, rather they are unique to Open Transport.
Your application can gather events from Open Transport two ways; it can either periodically poll for them or install a notfier function that Open Transport will call when an event occurs.
Polling for Endpoint Events
It is possible to process Open Transport events by calling the OTLook function from your applications main event loop to poll for incoming data or connection requests. OTLook will return information on events pending for a given EndpointRef.
The problem with this strategy, as with most forms of polling from an event loop, is linked to the way the Mac OS EventManager does event processing. To provide the cooperative multitasking used by the Mac OS, the WaitNextEvent function may occasionally allocate processing time to service other running applications. Consequently you have no control of how long it takes for other applications to call WaitNextEvent. This places your application's performance at the mercy of other apps running on the system. This is illustrated in Figure 1.

FIGURE 1. Polling for Endpoint Events

Warning:
Processing OT events in an application's event loop will result in unpredictable packet processing delays, because the round trip time from when your server receives a packet to when it responds will depend on factors external to your application.

Using a Notifier Function to Handle OT Events
As an alternative to polling, you should take avantage of an extension available the Mac OS implementation of Open Transport. Your application can specify a callback routine or notifier that Open Transport will call when an event occurs for a specific endpoint. This notifer can be installed automatically at endpoint creation with OTAsyncOpenEndpoint or later time with the OTInstallNotifier function.
When an event occur for an endpoint the notfifer will be invoked and passed the follwing items:

a user specified context pointer
a code identifying the event that occured
a result or error code
an optional pointer to additional information (the cookie parameter)

This is illustrated in Figure 2.

FIGURE 2. Using a Notifier to Process Endpoint Events

This is illustrated below:
typedef struct
{
	EndpointRef	epRef;	
	....
		
}epInfoRec;
	

pascal void myNotifier(void* myContextPtr, OTEventCode event,
                                     OTResult result, void* cookie)
    {
    
    	epInfoRec *epp = myContextPtr;
    	
        switch(code)
        { 
            case T_OPENCOMPLETE:    // Endpoint opened
                    epp->epRef = (EndpointRef)cookie
                ...
                
            case T_DATA:            // Network Data avail
                ...
             
            case T_DISCONNECT:      // Endpoint is disconnected
                ....
                
        }
    };


// opening a TCP Endpoint

epInfoRec epRec;

err = OTAsyncOpenEndpoint( 
            OTCreateConfiguration ("tcp"),0, NULL, &myNotifier, &epRec);
In the above example, The application creates an TCP endpoint with the OTAsyncOpenEndpoint function, passing in the address of the myNotifier callback routine, and a pointer to the user defined structure, epRec, to be passed to the notifier. Once the endpoint plumbing has sucessfully completed, Open Transport will call the notifier routine with the follwing information:

ContextPtr The address of epRec

OTEventCode T_OPENCOMPLETE

OTResult kOTNoError (hopefully)

cookie The EndpointRef of the newly created endpoint

More information on Open Transport event codes and values passed to notifiers can found beginning on page 2-17 of Inside Mac: OpenTransport.
Lazy Evaluation
Use a notfier and defer all events to event loop.

FIGURE 3. Lazy Evaluation

Do everything Synchronously from a thread
Open Transport 1.1.1 introduced the SyncIdleEvents feature, which was intended to facilitate Notifier/Thread Manager interaction. What the feature does is call your notifier at a time when it is safe to call YieldtoThread. Since YieldtoThread will eventually cause the Thread Manager to switch to a thread that calls WaitNextEvent, this presents the same unpredictable latency. For this reason, I would suggest not to use this strategy in a high performance server.

Getting the Most Out of Open Transport By Using Notifiers
Since the notifier mechanism is the most immediate way for an application to discover what endpoint events are occurring, your code should attempt to respond to most actions directly in the notifier.
In general, network server code should avoid deferring incoming packet processing to System task time.
A better strategy to processing incoming packets is to receive and initiate all I/O from the notifier. As a rule, if you can start an OTSnd or an asynchronous File Manager operation from a notifier, you should do it. This is the most important piece of advice you can follow if you want to extract the best performance from Open Transport. Network applications developers, especially those designing servers, should heed this recommendation. Packet-response time typically is used as a measure of server performance.
immune to mouse downs, application activity continues in background.
possibly an issue with spending too much time in interrupt stack..
Specifically, in a Notifier you should be able to perform the following tasks:

Accept and hand off connections

Receive and process all incoming data

Start asynchronous I/O operations, e.g., File Manager

Send network data

Tear down network connections.

Combination of above
leads to synchronization problems
Notifier Hints and Guidelines
In the Mac OS, you have three execution contexts:

System task - WaitNextEvent

Deferred task - Secondary Interrupt, when the interrupt mask is zero

Primary interrupt - I/O completion, VBL or Time Manager tasks
Because proper use of Open Transport notifiers is the key to improved server performance, here are some hints and guidelines to help you use notifiers more effectively:
Receive and process network data from the notifier. Don't defer processing to SystemTask, since this directly affects your turnaround time.

Treat the notifier code path as a critical section; assume you are locking the operating system from other tasks: keep it short and simple. If you must perform a lengthy operation, this is the place to consider using lazy evaulation with a Deferred Task.

Open Transport will never run a notifier during a Primary Interrupt, only at System Task or Deferred Task time. A Deferred Task, also known as Secondary Interrupt, occurs when the interrupt mask is zero, i.e., on the way out of a Primary Interrupt.

Since it is also possible for Open Transport to run a notifier during the execution of or returning from an OT routine, you should never call OT at Primary Interrupt time, except to schedule a Deferred Task.

You should never make a synchronous OT call from inside a notifier. Doing this will cause Open Transport to return kOTStateChangeErr in order to prevent you from deadlocking

You can use completion events to gate endpoint action, i.e., you can use T_OPENCOMPLETE to initiate an OTBind or T_DISCONNECTCOMPLETE to gate an OTUnBind. This method will prevent you from receiving a kOTStateChangeErr for calling a function before an endpoint is ready.

The normal XTI events (T_DATA, T_LISTEN ...) and the completion events (T_OPENCOMPLETE, T_BINDCOMPLETE ...) will not reenter the notifier . The events that do are T_MEMORYRELEASED and some of the high priority notifications, such as kOTProviderWillClose. For example:

Your notifier receives a T_GODATA

The notifier responds by calling OTSnd

Notifier reenters with T_MEMORYRELEASED

OTSnd returns.

Starting with Open Transport 1.1.1, you can use the OTEnterNotifier and OTLeaveNotifier functions to prevent your notifier from being entered during a critical section of code.

For a list of OT functions and in what context you can call them, refer to Appendix F in the Open Transport Client Developer Note.

Interrupt-Safe Functions
One of the major reasons that developers have shied away from processing packets in a notifier is you can't call the Mac Toolbox functions that move memory at interrupt time. A number of fast, interrupt-safe functions, however, are available from Open Transport. These functions are the same for both Mac OS 7 and Mac OS 8.
Many developers may overlook the functions available in the OT library. The Open Transport Client Developer Note and the <Opentransport.h> include file ought to provide you with the information you need.
IM:OT 1.1.1 should cover this info.
Memory Management
OTAllocMem/OTFreeMem can be safely called from a notifier. Keep in mind that the memory pool used by OTAllocMem is allocated from the application memory pool, which, due to the Memory Manager's constraints, can only be filled at task time.
Therefore, if you allocate memory from the Open Transport memory pool from an interrupt or deferred task, you should be prepared to handle a failure resulting from an temporarily depleted memory pool, which can only be replenished at the System Task time.
Strategy for handling kENOMEMErr in notfier.
PreAllocate important structures. (eg OTCreateDeferredTask / OTScheduleDeferredTask)

Fall back to SystemTask

List Management
OTLIFO functions can be used to implement an interrupt-safe LIFO or FIFO list. Here's how:

Create a OTLIFO list.
Populate it at interrupt or notifier time by using OTLIFOEnqueue.
Use the OTLIFOStealList to atomically remove the list and then call OTReverseList to flip it around, so that it will become a FIFO list.
You can then use OTLIFODequeue to remove individual elements from the list.

Semaphores
There are also a number of Atomic functions and macros, such as OTAtomicSet/Clear/TestBit, OTCompareAndSwap, OTAtomicAdd, OTClearLock/OTAcquireLock that can be used to administrate interrupt-safe semaphores.

Open Transport Deferred Tasks: A Closer Look
Open Transport Deferred Tasks provides a way to simplify working with primary interrupts, such as IO completions , VBL tasks, or Time Manager tasks . Remember that a Deferred Task, also known as Secondary Interrupt, occurs on the way out of processing a primary interrupt, after the interrupt mask has been lowered to zero. Deferred Tasks are in effect a priority above SystemTask, but still can be interrupted by a Primary Interrupt such as packet reception.
Although this is a form of lazy evaluation, it is more predictable than defering to SystemTask time. OTCreateDeferredTask can be used to setup a block of code that can be scheduled from primary interrupts to run at next Deferred Task time. Just pass OTCreateDeferredTask a pointer to the function you wish to schedule and an contextInfo argument, and it returns you a reference that can be used later to schedule the function.
	dtRef = OTCreateDeferredTask(taskproc,  contextInfo);
You can then use OTScheduleDeferredTask to schedule the function associated with the reference to run at the next Deferred Task time. Once scheduled , the function pointed to by taskproc will be called back at the appropriate time and passed contextInfo as a parameter.
	if(  OTScheduleDeferredTask(dtRef) ) ... ;
The OTScheduleDeferredTask will return true if the function was scheduled, false if not. If the function was not scheduled, and the dtCookie parameter is valid, then this indicates that the function is already scheduled to run.
Since OTScheduleDeferredTask needs to allocate memory, it is best to call it at SystemTask Time, otherwise you will run the risk of having to process a kENOMEMErr. The best strategy is to create all deferred tasks at Systemtask and then schedule then to run whenever you need them (notifiers)

Note:
Although you can call OTScheduleDeferredTask at any time a during primary interrupt, you must bracket the call with a OTEnterInterrupt/OTLeaveInterrupt pair, a better approach is use OTScheduleInterruptTask which combines the the functions.
Warning:
Since Open Transport does not keep track of outstanding Deferred Task requests, it is your application's responsibility before quitting to ensure that all outstanding Deferred Task requests have either fired, or have been cancelled with the OTDestroyDeferredTask call.

Avoiding Synchronization Problems
If you mix the processing of OTRcv in different interrupt contexts, such as notifier and Deferred Task or SystemTask time, you should be aware that certain synchronization problems can occur.
How do you avoid this problem.....
Diagram here..
For example:
You call OTRcv from your main thread.
There is no pending data, so OT returns a kOTNoDataErr.
Just then, an inbound data packet interrupts OT, causing it to step down to deferred task time to process the data.
OT calls your notifier with a T_DATA event, which you ignore.
Although the OTRcv in your main thread completes with a kOTNoDataErr, you have no way of knowing that you got the T_DATA event, and you won't get another one until you read to kOTNoDataErr again.
The result: your application hangs

[Prev] p. 1 2 3 4 5 6 7 8 [Next]

ContextPtr	The address of epRec
OTEventCode	`T_OPENCOMPLETE`
OTResult	`kOTNoError` (hopefully)
cookie	The `EndpointRef` of the newly created endpoint