Headlines

Headlines

IEEE 1588 in Network Processors for Next-Generation Industrial Automation Solutions

By Puneet Sharma, Intel Corporation

Industrial Automation Trends

Many designers of embedded and industrial automation equipment find it increasingly hard to maintain the pace of innovation using proprietary software, operating systems (OSs) and application-specific integrated circuits (ASICs). The steadily mounting cost of custom networks in industrial automation, combined with the increasing speed and reliability of commercial-off-the-shelf (COTS) networking components, is driving the current migration from proprietary networks to industry-standard networks including Ethernet.

This article provides an overview of the move to Ethernet-based COTS networking solutions in the industrial control and automation market segment. It includes an overview of the IEEE 1588 protocol and describes how the pioneering integration of IEEE 1588 technology in the Intel® IXP465 network processor delivers a cost-effective and highly accurate time synchronization solution for a wide range of Ethernet-based embedded applications.

Why Ethernet?

One of the fundamental explanations for the current shift to Ethernet is that commercial networks, communications networks, and industrial automation networks share common characteristics. Manufacturers of networking equipment must deliver increasingly higher performance in ever-shrinking devices and high-performance, reliable networks are needed to connect once-isolated environments to enable data sharing and to support real-time decision making. In addition, the choice of a networking standard is becoming an important issue for embedded developers, as networking and communications features are added to a broad range of embedded platforms.

Ethernet provides an opportunity for growth in industrial automation because it delivers all the advantages of a proven and reliable network standard, and Ethernet enables a uniform, plant-wide networking infrastructure at all levels of deployment, from the enterprise system down to the field level. Another advantage of Ethernet is that it allows standards-based IT technology to be deployed on the factory floor.

The use of standard IT technologies in these areas simplifies the adoption of a broad range of networking capabilities, including remote access, Web services and remote software updates, all from a centralized location. The advantages of Ethernet include higher network performance in the industrial control and automation sector, simpler updates and management, and a networking environment of proven reliability.

These attributes seem to make Ethernet networks the obvious choice for migration of an industrial automation network to a COTS network, but before this can happen, a gap must be overcome.

A Time-Synchronization Solution is Needed

Time-synchronization protocols, such as Network Time Protocol (NTP), are not reliable enough and do not provide sufficient resolution to be useful for industrial automation applications. Synchronizing nodes for industrial automation applications, such as motion control, require the implementation of a new time-synchronization protocol.

The IEEE 1588 standard is poised for adoption as the time-synchronization solution of choice on Ethernet-based systems. This technology is not Ethernet-specific, and it can be used on many network types for time synchronization. The strength of IEEE 1588 technology is its ability to deliver highly accurate clock synchronization for multiple control nodes in a distributed network. This overcomes the major weakness of standard Ethernet technology in industrial automation applications.

Intel® IXP465 Network Processor

Intel has integrated IEEE 1588 and Ethernet technologies into the Intel® IXP465 network processor to help meet the special networking requirements of next-generation industrial control and automation applications. This network processor builds on the Intel® IXP42X product line of network processors, extending Intel XScale® technology into a broad range of industrial control and automation applications that require communications and networking functionality along with a range of other integrated capabilities.

Figure 1 is a block diagram of the Intel IXP465 network processor, depicting functional units including the Intel XScale core, network processor engines (NPE) and a broad range of standard interfaces for enhanced design flexibility. The Intel XScale core, with speeds of up to 667 MHz, provides processing headroom for industrial automation applications. The network processor also provides integrated capabilities, including hardware-enhanced cryptography, built-in LAN, WAN, I2C and Synchronous Serial Port (SSP) interfaces to reduce overall system costs and simplify development.

Figure 1 - Intel® IXP465 Network Processor Block Diagram.

The Intel IXP465 includes several other enhancements for embedded networking applications:

  • Reliability is enhanced by error checking and correcting (ECC) support on the DDR1-266 controller, parity on most internal memories, and spread-spectrum clocking.
  • Low power dissipation with extended temperature support eliminates the need for fans and enables small-form-factor designs.
  • Open, industry-standard technologies are included, such as Ethernet, the TCP/IP software suite, and Universal Serial Bus (USB).
  • Integrated IEEE 1588 hardware-assist logic for time synchronization of multiple clocks in a distributed Ethernet system. This hardware-assist logic, when complemented by software running on the Intel XScale core, can be used to implement a full source- or sink-capable IEEE 1588-compliant Ethernet node.

The Intel IXP465 network processor is supported by a robust application-development environment, including prevalidated development infrastructures and operating systems, development platforms, software components and debug tools. Development tools are provided by Intel, members of the Intel® Communications Alliance and other third-party providers.

IEEE 1588 Overview

As measurement and control systems become more complex and support larger numbers of nodes, they increasingly require distributed network technologies. Most of these applications will be enhanced by having an accurate distributionwide sense of time achieved by having local clocks at each node, which can be a sensor, actuator, or other device.

The problem is that each of these individual clocks tends to drift apart due to instabilities inherent in source oscillators, in addition to environmental conditions including temperature, air circulation, mechanical stresses, vibration and aging [1]. This requires some kind of correction mechanism to synchronize individual clocks so that they maintain accurate distributionwide timing.

The IEEE-1588 Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems [2] defines a message-based protocol. Its purpose is to enable accurate synchronization of clocks with varying precision, resolution and oscillator stability in distributed systems that support multicast messaging, including Ethernet.

The IEEE 1588 protocol defines a ‘Best Master Clock (BMC) Algorithm’ in which each clock in a distributed system identifies the most accurate clock and labels it ‘master.’ All other ‘slave’ clocks synchronize to this master.

The protocol defines several periodic messages, independent of the distributed network technology, that trigger a clock to capture a time-stamp and communicate time-stamp information between the master and slave clocks. This method of using time-stamps enables each slave clock in a distribution to analyze the master clock’s timestamp and the network propagation delay. This allows the slave clock to determine the delta from the master in its synchronization algorithm. The IEEE 1588 protocol does not define the method a slave clock must use to adjust itself to synchronize with the master clock, nor does it define the method by which timestamps are captured.

IEEE 1588 and Ethernet

Four 1588 messages are encapsulated within standard Ethernet frames. These include Sync, Follow_up, Delay_Req and Delay_Resp.

In order for a slave node to synchronize with a master node, it must know two pieces of information: 1) how much the slave node’s clock is off from the master node-determined by the Sync and Follow_up message pair, and 2) what is the network propagation delay-determined by the Delay_Req and Delay_Resp pair.

As shown in Figure 2, the overall operation is as follows:

Figure 2 - The IEEE 1588 time-synchronization message exchange sequence on an Ethernet network.

  1. The master sends “Sync” messages to the slaves periodically (minimum interval specified is once every second). The outbound sync message triggers the master to capture a timestamp of its local system clock. The inbound sync message at the slaves triggers the slaves to capture a timestamp of its local system clock.
  2. The master sends “Follow up” messages to the slaves that contain the master timestamp of the previous “Sync” message. The slaves use the two timestamps to figure out the delta between the master and slave clocks.

At this point each slave has the first required piece of information-the slave and master system time delta. Now the slave needs the network propagation delay. Note that there may be a varied number of switches and/or routers between the master and slave nodes.

  1. The slave sends a “Delay Request” message to the master. The outbound Delay_Req message triggers the slave to capture a time-stamp of its local system time. The inbound Delay_Req at the master triggers the master to capture a timestamp of its local system time.
  2. The master sends “Delay Response” messages back to the slaves that contain the master timestamp from the receipt of previous “Delay Request” message. The slave now uses the two time-stamps to calculate the propagation delay between the master and slave node. Note that a fundamental assumption of the IEEE 1588 protocol is that propagation delay in network is symmetrical. At this point the slave uses the two pieces of information to adjust its system clock to synchronize with the master.

Switches and/or routers may induce a significant propagation delay between master and slave clock nodes. However the accuracy of time synchronization is dependant on how accurately propagation delay is measured, relative to how deterministic it is within the network. Note that switches and routers themselves may implement IEEE 1588 time synchronization, thus acting as master and slave simultaneously.

Accuracy of time synchronization also depends on how accurately the time-stamp is captured when triggered by a 1588 message. Hardware assisted triggering at the physical level, as opposed to firmware-interrupt triggering or application-level triggering, delivers the most accurate time-stamp capture. The accuracy of physical level time-stamps delivers time synchronization in the order of nanoseconds to submicroseconds [3].

Integrating IEEE 1588 Logic into the Network Processor

Intel is one of the first vendors to integrate a hardware-assisted IEEE 1588 implementation that delivers nanosecond to submicrosecond time-synchronization accuracy. The Intel IXP465 network processor supports IEEE 1588 time synchronization on three MII ports for Ethernet-based applications, each of which can act in master and slave mode configurations.

Integrating this logic into the Intel IXP465 network processor rather than off-chip in an FPGA achieves significant savings for cost and printed-circuit-board area. Figure 3 shows the position of the integrated IEEE 1588 logic in the network processor and the monitoring of three Ethernet ports and one auxiliary port, for hardware-assisted time-stamp triggering.

Figure 3 - IEEE 1588 logic within the Intel® IXP465 network processor.

This logic also supports one non-MII interface, called ‘auxiliary,’ via general-purpose I/O pins (GPIO), which is also configurable as a master or a slave. The auxiliary interface may be used to allow synchronization with a non-Ethernet based node, for example-a Global Positioning Satellite (GPS) or other clock.

An IEEE 1588 software API is provided in the standard Intel® IXP400 Software, v2.0, which runs on the Intel XScale core. This increases the designer’s flexibility to implement any range of the specified IEEE 1588 feature set. A clock synchronization algorithm can be implemented using a ‘frequency scaling’ method of system clock adjustment. This delivers uninterrupted system clock adjustment, suitable for real-time motion control. Another advantage of the frequency-scaling method is that it delivers a high-precision time-synchronization lock using only standard inexpensive oscillators.

Rockwell Automation Case Study

Until recently, Rockwell Automation, an established worldwide leader in power, control and information solutions for industrial automation, relied on an assortment of general-purpose processors and custom ASICs in thousands of different product types to meet the rigorous performance requirements of their solutions. It was a costly strategy requiring numerous specialty engineering resources. Rockwell Automation designers believed they could achieve significant time and cost efficiencies by moving many of their solutions to a single processing architecture-without sacrificing performance or features. For further information, please read “Intel® Network Processors Drive New Communications Capabilities for Rockwell* Automation Case Study, October 2004: www.intel.com/design/network/casestudies/rockwell.htm.

Summary

The IEEE 1588 hardware logic provides best accuracy time synchronization with reliable nanosecond to submicrosecond sync accuracy, as well as allowing for redundancy channels and switch/router implementations. The IEEE 1588 frequency scaling clock adjustment provides an uninterrupted clock adjustment for real-time applications proven to provide high accuracy with low-cost oscillators.

The Intel IXP465 network processor provides a unique combination of processor performance, integrated networking and reliability features that make it an ideal solution for emerging industrial control and automation applications. Through the pioneering integration of IEEE 1588 technology, the Intel IXP465 network processor delivers a cost-effective and highly accurate time-synchronization solution for a wide range of Ethernet-based embedded applications. And the IEEE 1588 software API in the Intel® IXP400 software provides a flexible Intel XScale core-based programming model.

References

[1] A Frequency Compensated Clock for Precision Synchronization using IEEE 1588 Protocol and its Application to Ethernet. Sivaram Balasubramanian, Kendal R. Harris and Anatoly Moldovansky, Rockwell Automation. September 24, 2003.

http://ieee1588.nist.gov/z_RA_1_FreqCompClk.pdf

[2] IEEE-1588™ Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems. John C. Eidson (Agilent Laboratories), Michael C.Fischer (Fischer Consulting) and Joe White (U.S. Naval Research Laboratory). November 8, 2002.

http://ieee1588.nist.gov/PTTI_draft_final.pdf

[3] IEEE 1588: Running Real-time on Ethernet, Dirk S. Mohl, Hirschmann Electronics, The OnLine Industrial Ethernet ook, Issue 17, November 2003.

http://ethernet.industrial-networking.com/ieb/articledisplay.asp?id=173

Puneet Sharma is currently a technical marketing engineer for Intel New Products Architecture on the Intel® IXP4XX product line of network processors. He is responsible for leading new product system solutions with top-tier customers in embedded, telecom and print-imaging market segments and for driving new product feature definitions and standards. Previously, Sharma worked as a silicon design engineer on the Intel® IXP42X, Intel® IXP2350 and Intel® IXP28XX network processors. Sharma also served as a Sort/E-Test automation engineer and software lead at Ireland FAB Operations (IFO). He holds a bachelor’s degree in computer engineering, summa cum laude, and a master’s degree in electronics and biomedical engineering, both from the University of Limerick, Ireland.

All information provided related to future Intel products and plans is preliminary and subject to change at any time, without notice.

Copyright © Intel Corporation 2005. All rights reserved. Reproduced by Extension Media with permission. This article was originally published in Intel’s Technology@Intel Magazine.