Event Service

When using events within an MPI application, a thread needs to be created to distribute any events that occur on the controller. This involves calling mpiControlProcessEvents and then waiting for controller interrupts inside a loop. You may implement your own event service routine if you want to maintain tight control on thread priorities or how the thread is executed. However, the MPI library provides a standard event service routine. This page discusses this standard routine.

The standard event service routine provides a large number of options:

• Polling for events or use controller interrupts to be notified when events occur.

• Setting the thread priority of the event service routine.

• Using callback functions or thread notification about events that have occurred.

• Notification of and handle any errors that occur within the event service routine.

Implementation

The standard event service executes a loop where it calls mpiControlProcessEvents, then either waits for a new controller interrupt or if the thread is configured for it puts the thread to sleep for a period of time. If an error occurs, an error counter is incremented and the error is passed to a user-specified error handler. If the error counter exceeds the error threshold value, then event service thread exits. The mpiControlProcessEvents routine gathers events from the controller and passes them to all notify objects and event callback routines associated with the controller.

Below is pseudo-code representing the design of the standard event service routine:

Pseudo-Code for the Standard Event Service Routine

MPI_RESULT

    ServiceThreadMain(MPIControl                        control,

                      MPIControlEventServiceErrorConfig eventServiceConfig)

{

    MPI_RESULT returnValue = MPIMessageOK;

    MPI_BOOL   exitThread = FALSE;

    int32_t       errorCount;

 

    while (not signaled to exit) {

        if (returnValue == MPIMessageOK) {

            returnValue =

                mpiControlProcessEvents(control);

        }

 

        if (returnValue != MPIMessageOK) {

            ++data->errorCount;

 

            if (eventServiceConfig.errorHandler != NULL) {

                MPIControlEventServiceErrorParams errorParams;

 

                errorParams.control    = control;

                errorParams.message    = returnValue;

                errorParams.errorCount = errorCount;

 

                if (eventServiceConfig.errorHandler(&errorParams) == TRUE) {

                    /* If errorHandler() returns true, treat as fatal error */

                    data->exitThread = TRUE;

                }

            }

 

            if (eventServiceConfig.errorThreshold >= 0) {

                if (errorCount > eventServiceConfig.errorThreshold)

                {

                    exitThread = TRUE;

                }

            }

        }

 

        if (exitThread != TRUE) {

            if (configured to use interrupts) {

                returnValue =

                    mpiControlInterruptWait(control, ...);

            } else {

                mpiPlatformSleep(polling time);

            }

        }

    }

 

    return returnValue;

}

Interupts vs. Polliing

The controller has a circular buffer which by default holds 256 event messages. If event service thread does not call mpiControlEvents(…) before 256 events occur, event messages are overwritten as new events occur. For interrupt driven service threads, this is usually not an issue. For a polling service thread, this could be a potential issue if the service thread does not have enough CPU cycles. In the worst-case scenario, events could be lost.

The best way to avoid lost events is to use an interrupt driven service routine. If your application uses polling, the best approach is to make sure the service thread polls for events at a high enough rate to avoid controller message buffer rollover. Determining the worst case service latency and the maximum event message rate can be difficult. A simple method is to estimate the event message frequency and make sure polling occurs at least once for every 64 events, which results in a safety factor of 4. If you know that some events occur more frequently than others, then you may want to increase the polling frequency.

Callback Functions vs. Thread Notification

When an event occurs, an application may be notified by either a callback function or by thread notification using MPINotify objects.

Callback functions (sometimes called delegates) are what most modern frameworks use (including .NET, MFC, ActiveX controls, wxWidgets) and are a very popular way of dealing with events. For quick actions that do not need to run in a specific thread, callback functions are the right solution. This includes actions such as logging events or setting the value of some variable shared by another thread in the application.

Thread notification allows threads in the application to wait for events. For example, if a thread commanded motion and needed to wait for the motion to complete before continuing, it could use an MPINotify object to be notified when a Motion Done event (MPIEventTypeMOTION_DONE) occurred.

Callback Function Guidelines

Each thread resides in its own thread and it is therefore possible to call any MPI function from within the callback function. In practice however, it is recommended that objects that are intended to be used outside the callback function are not created or deleted from within the callback function. This makes tracking object handles difficult and can easily lead to memory leaks or accessing deleted objects.

It is recommended that the callback function does not take a int32_t time to execute. There are two operations that will wait for the callback function to complete: mpiControlDelete(…) and processing of events from the same callback function. If two events that are processed by a single callback function and occur close in time, then the processing of the second event will not start until the processing of the first callback function completes.

Some common operations that occur in callback functions are:

• Setting a variable to be read elsewhere in the program.

• Changing the state of a graphical element. For example, to disable or enable a button.

• Signaling another thread to wake up.

• Using MPINotify objects to send event information to another thread waiting for the event information.

• Calling mpiControlEventCallbackRemove(…) to remove the event callback function from the controller.

• Logging information to a file, to a remote computer, or a shared memory location.

Error Handling

The event service thread allows applications to monitor and handle errors that occur in the thread To enable error handling, you must configure an MPIControlEventServiceErrorConfig structure and pass a pointer to this structure when calling mpiControlEventServiceStart(…).

An error handler must have the same type as the following function prototype:

MPI_BOOL MPI_DECL2

    myEventServiceErrorHandler(MPIControlEventServiceErrorParams *params);

A typical error handler logs the error. By returning FALSE, the error handler signals to event service routine to keep. For example:

MPI_BOOL MPI_DECL2

    myEventServiceErrorHandler(MPIControlEventServiceErrorParams *params)

{

    fprintf(logFile,

            "Event service thread Error occurred:\n"

            "  Error Count = %d  Error = 0x%x (%s)\n",

            params->errorCount,

            params->message,

            mpiMessage(params->message, NULL));

    return FALSE;
}

The error handler can also make a decision to halt the event service routine immediately by returning TRUE:

MPI_BOOL MPI_DECL2

    myEventServiceErrorHandler(MPIControlEventServiceErrorParams *params)

{

    fprintf(logFile,

            "Event service thread error occurred:\n"

            "  Error Count = %d  Error = 0x%x (%s)\n",

            params->errorCount,

            params->message,

            mpiMessage(params->message, NULL));

    if (params->message == MPIControlMessageEVENTS_LOST)

    {

        fprintf(logFile, "Exiting event service thread");

        return TRUE;

    }

    return FALSE;

}

In additional to an error handler, the event service routine can be configured to keep running in the presence of a certain number of errors. For this configuration, you must configure the errorThreshold member of MPIControlEventServiceErrorConfig. The error threshold is the number of errors the event service routine reports to the error handler and then ignores before the routine exits. When the event service routine encounters error number (1 + MPIControLEventServiceErrorConfig.errorThreshold), the event service routine exits, even if the error handler returns FALSE for that error.

Note: MEI strongly recommends that applications implement an error handler. This allows any errors that occur in the event service thread to be logged.

Common Questions

Why should I implement an error handler?

There are operations that are beyond the control of the MPI library that may cause an error to occur. For example, if an application has several threads running and the controller produces many events in a short period of time, it is possible the event service thread may not be able to process all the events before the controller’s event buffer fills up. In this case, the error handler receives the MPIControlMessageEVENTS_LOST error message. Other conditions that causes errors include out-of-memory conditions and when the client application deletes a MPINotify object before removing it from the controller’s notify list.

Note: MEI does its best to make the MPI library robust and bug-free. However, if a bug does exist in the MPI library that causes an error, the bug is difficult to track down without error logging. Therefore, logging is important to easily identify errors and bugs in the client application or MPI library and result in less machine down-time.

Does the error counter ever reset itself?

The error counter does not reset itself. However, the error counter may be reset by calling mpiControlEventServiceErrorCountReset(…).

If the error threshold was set to 10, and you received on average 1 error per month, and the system ran for 10 months, would the event service thread exit on the 10th month when it recieved the 10th error?

Yes, the service handler will fail on the 10th error if you do not call mpiControlEventServiceErrorCountReset(…).

What is a recommended value for the error threshold?

Values in the range of 1 to 500 are recommended.

Allowing the event service thread to handle a small number of errors without exiting allows the application to continue running if intermittent errors occur. Large thresholds (above one thousand) should be avoided in case the error ended up being a problem that did not disappear. In this scenario, the event service thread spins in a tight loop continually encountering the error.