Multi-Core CPUs Help Embedded Systems Reduce Interrupt Latency, And Increase Software Reliability

By Paul Fischer, TenAsys Corp.

Multi-core processors aren't just good for improving the performance of servers and office computers. They're delivering big system-level benefits for embedded computing applications as well. For example, multi-core Intel® processors are making it much easier to improve real-time Windows performance. For those systems that require the user-interface and enterprise connectivity of Windows—but also need deterministic real-time control—it's possible to dedicate one or more central-processing-unit (CPU) cores to processing real-time tasks. Other core(s) can be dedicated to general-purpose processing. This approach enables real-time processes to operate unencumbered by non-real-time applications. By isolating real-time processes from general-purpose processes on different cores, jitter and response time are greatly improved. The result is a more reliable and higher-performance real-time system.

From a software point of view, making such an environment work requires the creation of multiple virtual machines. Here, a general-purpose operating system (OS) like Windows runs unmodified on one virtual machine. The real-time operating system (RTOS) runs on the other. Such an arrangement can use the hardware features of the CPU to keep each virtual machine from affecting the other.This virtual-machine approach makes it very easy to host two operating systems on a dual-core CPU, as each virtual machine can be mapped directly onto its own physical core (see Figure 1). One CPU core hosts the RTOS while the other hosts Windows. The Intel® Core™ Duo processor provides a mechanism for signaling between the two virtual machines. It also allows for directing interrupts to specific CPU cores. As a result, real-time interrupts are always handled by the CPU core that's hosting the RTOS.

Dual-OS virtual-machine systems exist today. The INtime RTOS for Windows from TenAsys is such an example. It has been deployed with real-time applications operating at cycle times of 500 to 1000 microseconds on single-core desktop and industrial motherboard platforms (uniprocessor Intel® Pentium® 4 processors running at speeds of 1 to 3 GHz). For real-time applications demanding faster cycle times, the optimum solution was to use a faster processor. Generally, that's no longer true. A better alternative is to use a multi-core CPU.

When two virtual machines share a CPU—as is the case with the single-core processor designs—they must maintain a full machine context in order to switch between the two operating systems. Saving and restoring these contexts compromises event-response latency and cycle times. Such compromises can contribute 10 to 30 microseconds to the worst-case timer interrupt jitter. For a cycle time of 1000 microseconds, 10 to 30 microseconds of worst-case interrupt latency represents a jitter variation of only a few percent.

As cycle-time rates increase to say 50 to 200 microseconds, however, 10 to 30 microseconds of timer jitter represents a significant number. When jitter becomes a significant percentage of the cycle time, it adversely affects the stability and quality of the control algorithm. This degrades the stability margin of closed-loop control systems—especially naturally unstable systems like position-feedback motion-control loops.

Dedicated Cores

Dedicating one CPU core to the RTOS virtual machine on a multi-core platform allows virtually 100% of that core's CPU instruction cycles to be available for running real-time threads. The CPU cycles of the remaining cores become the exclusive property of general-purpose virtual machine(s). Contention for key processor resources, such as pipelines, cache, and the floating-point unit, are avoided when a CPU core is dedicated to the RTOS environment. Coordination between the RTOS core and Windows core(s) is accomplished using the built-in interprocessor interrupt mechanism, eliminating inter-OS context switch times.

On a real-time Windows dual-core system, real-time interrupt latencies are reduced by an order of magnitude—to 1 to 3 microseconds. Loop cycle times in the 50-to-200-microseconds range execute with very high precision. The net result is an order-of-magnitude improvement in quality and bandwidth for the control algorithms deployed on a dual-OS system.

With multiple virtual machines one can contain each OS to ensure that runaway processes in Windows never affect those in the RTOS and vice versa. A virtual-machine approach also allows real-time applications to run in user mode rather than kernel mode. The net result is improved reliability and robustness because real-time processes run in separately managed memory segments. These segments are distinct from those used by Windows. They provide address isolation and protection between real-time processes and non-deterministic Windows code. Development and debugging also are simplified, as all of the processes are run in protected user mode rather than unprotected kernel mode. There's no speed penalty for this approach—only increased reliability and safety!

Single Development Platform

Even though the developer is working with multiple operating systems running in separate virtual machines, they reside on a single hardware platform. This offers the advantage of hosting a single development environment on which to develop code for both parts of the solution: the deterministic real-time part and the enterprise-level part. New integrated development environments (IDEs) support such an approach. For example, Microsoft Visual Studio can be used to develop Windows and INtime applications on the same system (see Figure 2).

With the INtime development tools integrated into Visual Studio, real-time Windows applications can be edited, compiled, debugged, and maintained with the Visual Studio IDE. There's no reason to purchase or learn to use other development tools. Using an IDE like Visual Studio, developers can debug real-time applications at full speed—avoiding the need to use a simulator or debug over a communication interface. Debugging real-time code in a simulated environment dilutes the effectiveness of one's development effort, as a simulator isn't the actual run-time environment. Using a simulator or communication link, it's very easy to miss subtle timing-specific problems that can only be found when running directly on the real-time kernel. The INtime Visual Studio debugger plug-in captures and identifies real-time process faults caused by divide-by-zero errors, bad pointer accesses, page faults, stack faults, and other CPU exceptions.

Multi-core-processor components promise to deliver greater performance, real-time responsiveness, and lower power consumption to embedded systems. At the same time, they promise to keep system cost under control. The potential software complexity of implementing multi-core systems is eased by basing the processor usage models around a virtual-machine approach in which the real-time and general-purpose OS portions of the system can be hosted on dedicated processor execution cores. The final key is to employ integrated development environments that support design and execution on multi-core systems.

Paul Fischer is a senior technical marketing engineer at TenAsys Corporation. Paul's experience with INtime goes back to 1997, when the product was first introduced. He has logged more than 25 years of experience building and writing about real-time and embedded systems in a variety of engineering and marketing roles. Fischer has an MSE from UC Berkeley and a BSME from the University of Minnesota.