[ntpwg] Testing NTP performance

Fri May 2 16:57:03 UTC 2008

Gee Greg - I didn't think you spoke to me anymore...

NTP is BOTH a package and a physical data-exchange protocol but I thought 
that was self-evident to anyone on this list. The NTP transaction process 
protocol - designed by Mills and his teams originally was implemented in a 
SW model to prove its functionality but the term NTP is generally used to 
refer to the NTP Daemon and the protocol it runs.

My assertion here is that NTP has become sorta like the term XEROX which 
pertains to a process and the actual copy itself. Notice that when any other 
part of the package is discussed (SNTP or NTPDATE for instance) these are 
clearly enumerated.

As to the testing you talk about its good its in papers but those need to 
come with the product itself as part of the performance specifications for 
the unit and not as a part of a whitepaper per se IMHO.

Todd

----- Original Message ----- 
From: "Greg Dowd" <GDowd at symmetricom.com>
To: "TS Glassey" <tglassey at earthlink.net>; "STUART VENTERS" 
<stuart.venters at adtran.com>; <ntpwg at lists.ntp.org>
Sent: Friday, May 02, 2008 9:24 AM
Subject: RE: [ntpwg] Testing NTP performance

--- SNIP ---

>
> Remember that NTP is a SW product as it is distributed today
> from the land
> of Oz and that SW is by definition 'an emulation of
> something' - an idea we
> seem to have forgotten across our culture today.
>
NTP is a protocol, not software.  It is defined by RFC1305 or 4330.  The
implementation is something different to me.  Closely aligned but not
the same.  ntpd is software and I agree with you that we often confuse
the two.
> >
> > For my testing, I'm usually testing the egress port of a server for
> > accuracy with respect to the server clock and jitter.  For
> that, I use
> > custom designed ntp "probes".  These probes are pseudo clients,
> > basically boxes with hardware timestamping synchronized to
> gps or house
> > pps and then pointed at a single server and querying at 1 packet per
> > second.
>
> Greg does Symmetricom publish the results of this testing?
Yes we do publish the results.  That's more a marketing function but the
results show up in datasheets, white papers, presentations, etc.  We do
joint papers with various partners as well.  But, as I mentioned, we're
mostly testing our egress performance so it is not germane to most
people.

>
> > The probes record the statistics using peerstats and rawstats
> > which I can then load into a custom statistics package we
> have called
> > TimeMonitor Analyzer.  Phase plots, histograms, ADEV, TDEV, minTDEV,
> > MTIE at your fingertips.  After getting a baseline, I can
> start loading
> > up ntp traffic, network traffic, etc and continue to sample.
> >
> > I am not a big fan of just testing pps signals as they
> don't tell all
> > the story.  They give you the end to end performance but
> don't let you
> > break down the error into discrete components to analyze.
> But, it can
> > be useful.  It isn't too hard to hack linux these days to capture
> > timestamps or toggle serial lines.
> >
> > I have also tested the standard performance of linux clients running
> > ntpd against our servers.  In that case, one great way is
> to plug a gps
> > receiver into the slave, with appropriate refclock, and run
> the query
> > against the server notrust.  Log the stats and analyze.
> >
> > If you can't do that, put a known good ntp server right on
> the switch
> > port next to your box and then test the server under test
> with notrust.
> >
> > If you are testing the client, the easy way is to turn on stats if
> > possible.  Not absolutely accurate, but they give good data.
> >
> > When testing networks, another thing I like to use are
> Ethernet taps.
> > These guys let you "sample" the flow at any point when you
> are testing
> > loaded networks (a la g.8261).  Then you can map the packet delay
> > variation, and its subsequent impact on the client servo.
>
> Yes but this is only true if the local network is set up for
> portspanning
> between the TAP Port and the TEST Port. Most people today use
> switches for
> their timing service networks and that means that the ports isolate
> same-subnet traffic to the specific ports it is flowing
> to/from on that
> switch. So unless portspanning is used to turn the listener
> port on properly
> or a Hub is used instead of a switch, this wont necessaryily work.
>
Most of my stuff has moved into the Gigabit domain now and, while the
standard supports it, I don't think anyone makes a GBE hub :-)  I'm
talking about a standalone tap (www.networktaps.com).  They work
anywhere.  Handy things to have around the lab.

> >
> > I have not been overly impressed with the current
> generation of network
> > testers (e.g., spirent or ixia) for their ability to
> generate packets
> > with timestamps embedded or to manipulate timestamp fields.
>  But I think
> > that is getting better.  You really, really have to watch
> out for how
> > the testers are configured when setting up test nets.  It
> is very easy
> > to get standing wave patterns in your network that trash the test.
> > Really easy.  Too easy.  And very time-consuming :-)
> >
> >
> > Greg Dowd
> > gdowd at symmetricom dot com (antispam format)
> > Symmetricom, Inc.
> > www.symmetricom.com
> > "Everything should be made as simple as possible, but no
> simpler" Albert
> > Einstein
> >
> >
> >> -----Original Message-----
> >> From: ntpwg-bounces+gdowd=symmetricom.com at lists.ntp.org
> >> [mailto:ntpwg-bounces+gdowd=symmetricom.com at lists.ntp.org] On
> >> Behalf Of STUART VENTERS
> >> Sent: Friday, May 02, 2008 7:52 AM
> >> To: ntpwg at lists.ntp.org
> >> Subject: Re: [ntpwg] Testing NTP performance
> >>
> >>
> >> >>Not to get too philosophical, but isn't this a description of the
> >> >>precise problem that NTP itself is designed to solve?
> >>
> >> >>Rob Seaman
> >> >>NOAO
> >> >>--
> >>
> >>
> >> Rob,
> >>
> >> Guilty as charged, there is circular logic here.
> >>    I'm using NTP's ability to transport time to measure same.
> >>
> >> But perhaps I can get out on a technicality.
> >>    I'm proposing to use the time transport ability over easy links
> >>        to measure the time transport ability over a more
> >> difficult link.
> >>
> >> I'd be happy to say that the only unit under test is A
> >>   and so B and C can be 'Golden units'.
> >>
> >> I need to test down to sub-millisecond accuracy.
> >>   There are servers available for B with 100nS accuracy.
> >>   There are packet bert testers that could pinch hit as a
> >> client C with similar accuracy.
> >>   These should be sufficient to get us to uS accuracy.
> >>
> >> Again, this seems like a complicated plan, there ought to be
> >> an easier way.
> >>
> >> Asked another way, folks here have claimed that NTP runs
> >> with particular accuracies much tighter
> >>    than a human can check by printing out the time on a console.
> >>
> >> How do they know?
> >>
> >>
> >> Regards,
> >>
> >> Stuart
> >>
> >>
> >>
> >>
> >>
> >> > Greetings,
> >> >
> >> > I'm puzzling over how to test embedded NTP implementations.
> >> >
> >> > Given a client and server connected through some test network,
> >> >   how do you accurately measure the difference between the
> >> server and
> >> > client clocks.
> >> >
> >> > If there was a 1PPS test wire coming out of each
> implementation, it
> >> > would be easy.
> >> >   But unless you can add code to the client, there is
> usually not a
> >> > wire,
> >> >
> >> > I need some way to peek into the client and server to see their
> >> > internal clocks.
> >> >
> >> > I'm thinking the server functionality embedded with the
> >> client might
> >> > help.
> >> >
> >> >
> >> > Consider the following system with a third NTP box:
> >> >
> >> >
> >> >                        (C)
> >> >                       CLIENT
> >> >                    /         \
> >> >                  /             \
> >> >                /                 \
> >> >              /                     \
> >> >            /                         \
> >> >          /
> >> >      SERVER ---------- TEST  ------ CLIENT/SERVER
> >> >        (B)            NETWORK           (A)
> >> >
> >> >
> >> >
> >> > We are trying to measure the effects of the test network on
> >> the client
> >> > in A.
> >> >
> >> > Server B provides a stable timing reference for the test.
> >> >
> >> > Client/Server A is locked to Server B through the test network.
> >> >
> >> > Client C is connected to servers A and B.
> >> >
> >> >
> >> > If the links connecting Client C are simple point to point
> >> links, then
> >> > hopefully most of the errors in time transport
> >> >   are due to the test network affecting A.
> >> >
> >> > C provides an offset between B&C and also between A&C.
> >> >
> >> > It seems like the difference between these offsets  should be the
> >> > offset between A&B.
> >> >
> >> > This seems like a complicated plan, is there a better way?
> >> >
> >> >
> >> > Regards,
> >> >
> >> > Stuart
> >> >
> >> >
> >> > ps: My apologies for an NTP 101 question.
> >> >     Perhaps someone could point me to where this has already been
> >> > sorted out.
> >>
> >>
> >> _______________________________________________
> >> ntpwg mailing list
> >> ntpwg at lists.ntp.org
> >> https://lists.ntp.org/mailman/listinfo/ntpwg
> >>
> > _______________________________________________
> > ntpwg mailing list
> > ntpwg at lists.ntp.org
> > https://lists.ntp.org/mailman/listinfo/ntpwg
>
> 