[ntpwg] NTP vs. PTP

[TS Glassey]

I am working on an adaptation of 1588 that reduces its messaging to a ICMP 
protocol. Anyone else interested?

T.
----- Original Message ----- 
From: "Greg Dowd" <GDowd at symmetricom.com>
To: "David L. Mills" <mills at udel.edu>; "NTP Working Group" 
<ntpwg at lists.ntp.isc.org>
Sent: Wednesday, October 24, 2007 1:25 PM
Subject: Re: [ntpwg] NTP vs. PTP


> Costs fall.  Our first NTP server (1991) sold for almost $20k.  Alma VME
> crate with Matrix CPU and double card GPS.  The big difference really
> seems to reside in the implementation.  1588 is typically implemented in
> hardware and hence has better possibility of sync accuracy.  NTP is
> typically implemented in software, worse, even non-realtime software so
> it can suffer.  The truth is you can build 1588 in software (see ixxat
> site) and you can build ntp in hardware (see symmetricom site).
>
> As 1588 is newer, there seem to be a few additions based on what we've
> learned; follow up messages allow even software based implementations
> improved timestamp resolution and transparent clock definitions allow
> compensation for dwell time in switching elements.
>
> On the other hand, ntp has had a lot more system engineering built in
> with more flexible concurrent modes of configuration and more rigor in
> the authentication models.
>
> As for the description of 1588 usage, that could also map to synchronous
> ethernet.
>
>
>
> -----Original Message-----
> From: ntpwg-bounces+gdowd=symmetricom.com at lists.ntp.org
> [mailto:ntpwg-bounces+gdowd=symmetricom.com at lists.ntp.org] On Behalf Of
> David L. Mills
> Sent: Wednesday, October 24, 2007 11:42 AM
> To: NTP Working Group
> Subject: Re: [ntpwg] NTP vs. PTP
>
> Martin,
>
> The real downside of 1588 is the huge cost of the grandmaster. But, 1588
> is a really good way to synchronize interface clocks on a LAN. Assume a
> good NTP server with GPS and PPS. Expect the system clock to hold a few
> microseconds. Run 1588 on the LAN to synchronize the intervace clocks
> time and frequency, but not necessarily the exact right time and
> frequency. At periodic intervals measure the offset between the 1588
> interface clock and the system clock and broadcast that over the LAN.
>
> Piggyback the offset in an NTP extension field for compatiblity. As
> these packets arrive, discipline the system clock to the same offset.
> 1588 is used intact and interrupts don't matter.
>
> Dave
>
> Martin Burnicki wrote:
>
>> Hi Brad,
>>
>> Brad Knowles wrote:
>>
>>> From what little I can tell, it seems to me that IEEE 1588 is much
>>> more oriented towards a closed single-entity local area (or
>>> near-local area) network environment, where you don't care if you're
>>> sync'ed to any given external reference, or how close or far you may
>>> be from The One True Time, you just care that all your internal
>>> slaves are sync'ed to your given master, and that you have some
>>> pretty tight tolerances on how far out of sync the slaves can get.
>>
>>
>> Yep. However, it depends. Basically IEEE 1588 (PTP) is capable of
>> sync'ing all clients to a server/grandmaster with high accuracy. If
>> the application does not only rely on the _same_ time for all nodes
>> but also on the _right_ time the you just have to take care that the
>> grandmaster is sync'ed e.g. to a GPS receiver.
>>
>> BTW, the same is essentially true for NTP.
>>
>>> In contrast, it seems to me that NTP works well on the broader WAN
>>> multi-entity internet (global and beyond), and that it can work just
>>> fine on smaller scale single-entity networks where sub-nanosecond
>>> accuracy timekeeping is not required.
>>>
>>>
>>> The two protocols seem to complement each other pretty well, and it
>>> would appear that the folks at Meinberg agree.
>>
>>
>> I absolutely agree.
>>
>> The main difference between NTP and PTP is that the statistics
>> implemented by the NTP algorithms yield quite good results even over
>> WAN connections, and just using standard hardware.
>>
>> PTP can achieve much higher accuracy than NTP, but this is _only_ if
>> special hardware is used which explicitely supports PTP. And this
>> basically applies to both PTP v1 and v2.
>>
>> Imagine you want to send a packet with "the right time" from one PC to
>
>> another.
>>
>> 1.) The sending program picks up a time stamp and puts it into a
>> network packet.
>>
>> 2.) The network packet is then passed to the IP protocol stack where
>> it is passed down from the sending user space application to the
>> network drivers which partially run in kernel space, where it finally
>> ends up in a send queue. This introduces a delay which depends on the
>> CPU power and the system load (interrupt requests).
>>
>> 3.) The network driver waits until the network wire is unused and
>> starts to transmit the packet. If there's a collision on the wire then
>
>> transmission is aborted and retried after a random delay.
>> This also causes a random delay.
>>
>> 4.) Once the packet is on the wire the propagation delay is pretty
>> constant, depending on the length of the wire. If there's a network
>> hub between the sender and the receiver then this also introduces an
>> additional delay, which is pretty constant, though.
>> If there's a router or switch between the 2 nodes then the packet may
>> be queued for an undetermined amount of time, which also results in an
>
>> unknown delay.
>>
>> 5.) If the packet arrives at its destination then the network driver
>> generates an interrupt request to let the packet be fetched by the
>> protocol drivers. It also takes an unknown amount of time until this
>> is done, depending on the CPU power, whether there are
>> higher-prioritized interrupts just being handled, etc.
>>
>> 6.) Finally the packet is moved up the protocol stack, moved from
>> kernel space back to user space, and passed to the application which
>> then takes a time stamp of its own system time in order to compute the
>
>> difference to the time stamp from the incoming packet.
>>
>> There's a bunch of unknown delays there, isn't it?
>>
>> The only delay which is constant and can be measured is the
>> propagation delay on the cable.
>>
>> The other effects introduce a more or less random delay which can only
>
>> be estimated by statistical methods. That's what the NTP algorithms do
>
>> with quite good results. The advantage is that no special hardware is
>> required, but the disavantage is that you can only yield a limited
>> accuracy.
>>
>> In order to compensate the receive delay (i.e. when a packet comes in
>> from the wire until it arrives at the application) you have to take a
>> time stamp when a packet comes in from the wire. This is done by a
>> time stamp unit which includes a pattern matcher which has to identify
>
>> incoming packets in the bit stream from the wire, and take a time
>> stamp if such a packet is detected.
>> Both the network card driver and the application have to provide a way
>
>> (API
>> call) which lets the application retrieve that time stamp from the NIC
>
>> driver and assign it to the associated (right) packet. This way the
>> application can compute the difference between the time it has
>> received the packet and the time the packet has arrived from the wire,
>
>> and account for that delay.
>>
>> Obviously the same has to be done for outgoing packets, i.e. determine
>
>> the time interval from when the packet is sent by the application
>> until it really goes onto the wire. The calculated delay has to be
>> passed to the receiver which has to account for that delay.
>> Unfortunately the time stamp can only be taken when the packet goes
>> out onto the wire, so when the time stamp is available the packet has
>> already been sent. The PTP protocol accounts for this situation by
>> sending a so-called follow-up packet which contains the time stamp of
>> the previous packet. The receiver then gets the original packet which
>> is time-stamped when coming in, plus the follow-up packet which
>> contains the transmission delay and can thus account for both the
> delays.
>>
>> Using a point-to-point connection between the transmitting and the
>> receiving node you can yield an accuracy down to a couple of
>> nanoseconds by hardware timestamping.
>>
>> However, there's still a last delay which is not yet known by our
>> server and client. This is the propagation delay across intermediate
>> nodes like switches and routers. For example, if a switch receives an
>> incoming packet at one port, and the outgoing port is just be used by
>> another packet then the incoming packet is queued internally in a
>> FIFO. This can take up to several tens of milliseconds (!), depending
>> on the network load and the queue depth.
>>
>> The problem is that neither the transmitting nor the receiving node
>> can determine whether a packet has been passed on directly, or has
>> been queued, and for how long it has been queued.
>>
>> So even if both your endpoints provide a way for hardware time
>> stamping, a single standard switch between them can screw up the
>> accuracy. The only way to avoid this are either to use "dumb" hubs
>> which just duplicate the packets without queueing them, or to use
>> special switches which are aware of special timing packets and handle
>> them in a special way.
>>
>> The PTP protocol defines a special "transparent" or "boundary clock"
>> which can
>> be implemented in switches or routers in order to handle the PTP
>> packets in a special way which compensates the switch's delays. The
>> Hirschmann switch which we offer as part of our PTP starter kit
>> http://www.meinberg.de/english/ptp-starterkit
>> supports the PTP protocol that way.
>>
>> Since standard PC NICs don't provide hardware TSU support nowadays,
>> you must install a special PTP NIC (normally a PCI card) in each of
>> the PCs anyway.
>>
>> Here is a short summary on the 3 ways to assign timestamps to specific
>
>> outgoing and incoming network packets:
>>
>> 1.) Inside the application, if a packet is sent or received. This is
>> very portable since it runs completely in user space, and this is the
>> way currently used by NTP. You can also run this on operating systems
>> where you don't have access to the source code of the NIC drivers.
>> However, this is the most inaccurate way since the time stamps depend
>> on the latency of the IP stack, and on network collisions.
>>
>> 2.) Inside the NIC driver, e.g. in the interrupt service routine. This
>
>> avoids the latencies of the IP stack, but you must modify the source
>> code of the driver, or have a driver which takes time stamps if
>> certain packets are being sent or received. Also, you don't know the
>> latency due to network collisions when sending packets.
>>
>> 3.) By the NIC hardware. You need a time stamp unit (TSU) which
>> listens on the data lines between the MAC and the PHY. The TSU
>> contains a pattern matching unit which detects the pattern of the
>> desired packet type in the serial bit stream on those data lines. If
>> such a pattern is detected then the pattern matching unit must capture
>
>> a time stamp of a high speed counter which is also part of the TSU.
>> Since this method takes time stamps when a packet goes on the wire or
>> arrives on the wire is also capable to eliminate the latency due to
>> network collisions.
>> The driver must be able to read those time stamps from the TSU and
>> either assign them to the packets or pass them up the protocol stack
>> to the application which then assigns the time stamps to the packets.
>> The TSU can be implemented in a programmable logic device like an ASIC
>
>> or FPGA.
>>
>> The latter method is the most accurate, but it requires a NIC with a
>> TSU, and a kernel driver which supports the TSU. The problem here is
>> that most commonly used NICs have the MAC and the PHY in a single
>> chip, and the data lines on which the TSU must listen is not
>> accessible outside the chip.
>> However, there are also some new NIC chips available which have a
>> built-in TSU.
>>
>> As a conclusion you can say that
>>
>> 1.) NTP can achieve pretty good accuracy in both small and large
>> networks where you don't know which routes the packets take, and you
>> can not and don't have to rely on special hardware support for the
>> protocol.
>>
>> 2.) PTP can yield very high accuracy provided that the network
>> infrastructure fully supports the protocol. Obviously this is easier
>> to implement in a closed network where the administrators have full
>> control over the equipment.
>>
>> 3.) Once the special hardware support for PTP is not available, PTP
>> suffers from the same limitations as NTP, i.e. the unknown latencies
>> mentioned above.
>> Undere these conditions NTP can even yield better results due to the
>> statistical methods it uses.
>>
>> The other way round, we have made tests with NTP using the same
>> hardware timestamping methods, and we could see that NTP can yield the
>
>> same accuracy as PTP if the basic conditions are similar.
>>
>> The problem here is that the current specification of the NTP protocol
>
>> does not provide a method to send a follow-up message to the client in
>
>> order to let the client know when the original packet really made its
>> way onto the wire. If you add such a method then it breaks
>> compatibility with existing implementations of NTP.
>>
>> It's an advantage for the PTP protocol that it could be implemented in
>
>> a way which adds that option missing for NTP.
>>
>>
>> Best regards,
>>
>> Martin
>
>
> _______________________________________________
> ntpwg mailing list
> ntpwg at lists.ntp.org
> https://lists.ntp.org/mailman/listinfo/ntpwg
> _______________________________________________
> ntpwg mailing list
> ntpwg at lists.ntp.org
> https://lists.ntp.org/mailman/listinfo/ntpwg