Ulma (our ultralight monitor architecture) needs to support multi-modal instrumentation, in which different events can result in different monitor behavior. The instrumentation should be "ultralight" (superfast) as well as capable of being changed at runtime (disabled entirely, configured to report via sharedmem, socket, coexpression, etc). To get there, I first present Alamo's instrumentation and what I have learned, and then discuss what the Ulma instrumentation should look like.

Alamo uses instrumentation macros such as EV(code,value) which can be defined at compile time as empty, or as a function call, a co-expression switch, or whatever. Initially (done by Ralph Griswold and Gregg Townsend) they were all-or-nothing, wrote to log files, and turning on instrumentation caused a huge slowdown, especially considering the extremely frequent events such as VM instructions. The instrumentation was in the VM and shared by all programs, not done separately on a per-program or per-execution-monitor-basis. One could build custom VM's with specific events of interest available and others not instrumented, but that was unattractive. Much later, I noticed that only about 30 functions in the VM were instrumented, and modified the VM to have two copies of each of those functions: one instrumented and one uninstrumented. Each program keeps track of what event types (if any) it needs to report, and uses the instrumented versions of the functions that can possibly report events of the type(s) for which it is being monitored. A single VM, with dual copies of 30 of its functions, is good for regular program executions as well as for monitoring.

For Ulma we want many of these same properies. We want instrumentation that can be turned on and off at runtime...but we also want an instrumentation macro that can be switched to different reporting methods (sharedmem, socket, etc) at runtime, rather than at compile time. If an instrumented program wants to monitor itself (i.e. the "one process model"), it should be able to plugin a function to call; from the same compiled binary, if that instrumentation site were being monitored by a different co-expression or a different process, the instrumentation should do the right thing. Also, some monitors may wish to install some of their instrumentation as one-process calls while others are IPC or context switches, or a one-process instrumentation function might, if during its analysis it identified a special-case or higher-order behavior, trigger an IPC or context-switch reporting of the event.

How do we go about creating such an instrumentation model? Via a data structure. Compared with Alamo, which has for each event type a bit (in the eventmask bitset) plus a set (value mask) plus a single hardwired event reporter function (coexpression switch) plus a per-program global variable which says where to send events, for each event type in Ulma we want:

In Alamo there are about 120 event types; in Ulma we might have fewer or more, but in general, we are looking at around 1KB to hold this information on a per-event-type basis: no problem.



Edit Me