[ntpwg] Testing NTP performance
Greg Dowd
GDowd at symmetricom.com
Fri May 2 16:24:43 UTC 2008
Greg Dowd
gdowd at symmetricom dot com (antispam format)
Symmetricom, Inc.
www.symmetricom.com
"Everything should be made as simple as possible, but no simpler" Albert
Einstein
> -----Original Message-----
> From: TS Glassey [mailto:tglassey at earthlink.net]
> Sent: Friday, May 02, 2008 8:56 AM
> To: Greg Dowd; STUART VENTERS; ntpwg at lists.ntp.org
> Subject: Re: [ntpwg] Testing NTP performance
>
> Greg -
>
> ----- Original Message -----
> From: "Greg Dowd" <GDowd at symmetricom.com>
> To: "STUART VENTERS" <stuart.venters at adtran.com>;
> <ntpwg at lists.ntp.org>
> Sent: Friday, May 02, 2008 8:31 AM
> Subject: Re: [ntpwg] Testing NTP performance
>
>
> >I think the testing methodology is quite dependent on what you are
> > trying to observe. Are you looking for the absolute accuracy of a
> > client with respect to UTC, accuracy wrt server, relative accuracy
> > between 2 clients, or a pool of clients, etc. Or, are you testing a
> > server's ability to put time on a network? Are you looking for p-p
> > error, convergence time, rms. Are you testing some
> application which
> > would add its own error on top?
>
> Remember that NTP is a SW product as it is distributed today
> from the land
> of Oz and that SW is by definition 'an emulation of
> something' - an idea we
> seem to have forgotten across our culture today.
>
NTP is a protocol, not software. It is defined by RFC1305 or 4330. The
implementation is something different to me. Closely aligned but not
the same. ntpd is software and I agree with you that we often confuse
the two.
> >
> > For my testing, I'm usually testing the egress port of a server for
> > accuracy with respect to the server clock and jitter. For
> that, I use
> > custom designed ntp "probes". These probes are pseudo clients,
> > basically boxes with hardware timestamping synchronized to
> gps or house
> > pps and then pointed at a single server and querying at 1 packet per
> > second.
>
> Greg does Symmetricom publish the results of this testing?
Yes we do publish the results. That's more a marketing function but the
results show up in datasheets, white papers, presentations, etc. We do
joint papers with various partners as well. But, as I mentioned, we're
mostly testing our egress performance so it is not germane to most
people.
>
> > The probes record the statistics using peerstats and rawstats
> > which I can then load into a custom statistics package we
> have called
> > TimeMonitor Analyzer. Phase plots, histograms, ADEV, TDEV, minTDEV,
> > MTIE at your fingertips. After getting a baseline, I can
> start loading
> > up ntp traffic, network traffic, etc and continue to sample.
> >
> > I am not a big fan of just testing pps signals as they
> don't tell all
> > the story. They give you the end to end performance but
> don't let you
> > break down the error into discrete components to analyze.
> But, it can
> > be useful. It isn't too hard to hack linux these days to capture
> > timestamps or toggle serial lines.
> >
> > I have also tested the standard performance of linux clients running
> > ntpd against our servers. In that case, one great way is
> to plug a gps
> > receiver into the slave, with appropriate refclock, and run
> the query
> > against the server notrust. Log the stats and analyze.
> >
> > If you can't do that, put a known good ntp server right on
> the switch
> > port next to your box and then test the server under test
> with notrust.
> >
> > If you are testing the client, the easy way is to turn on stats if
> > possible. Not absolutely accurate, but they give good data.
> >
> > When testing networks, another thing I like to use are
> Ethernet taps.
> > These guys let you "sample" the flow at any point when you
> are testing
> > loaded networks (a la g.8261). Then you can map the packet delay
> > variation, and its subsequent impact on the client servo.
>
> Yes but this is only true if the local network is set up for
> portspanning
> between the TAP Port and the TEST Port. Most people today use
> switches for
> their timing service networks and that means that the ports isolate
> same-subnet traffic to the specific ports it is flowing
> to/from on that
> switch. So unless portspanning is used to turn the listener
> port on properly
> or a Hub is used instead of a switch, this wont necessaryily work.
>
Most of my stuff has moved into the Gigabit domain now and, while the
standard supports it, I don't think anyone makes a GBE hub :-) I'm
talking about a standalone tap (www.networktaps.com). They work
anywhere. Handy things to have around the lab.
> >
> > I have not been overly impressed with the current
> generation of network
> > testers (e.g., spirent or ixia) for their ability to
> generate packets
> > with timestamps embedded or to manipulate timestamp fields.
> But I think
> > that is getting better. You really, really have to watch
> out for how
> > the testers are configured when setting up test nets. It
> is very easy
> > to get standing wave patterns in your network that trash the test.
> > Really easy. Too easy. And very time-consuming :-)
> >
> >
> > Greg Dowd
> > gdowd at symmetricom dot com (antispam format)
> > Symmetricom, Inc.
> > www.symmetricom.com
> > "Everything should be made as simple as possible, but no
> simpler" Albert
> > Einstein
> >
> >
> >> -----Original Message-----
> >> From: ntpwg-bounces+gdowd=symmetricom.com at lists.ntp.org
> >> [mailto:ntpwg-bounces+gdowd=symmetricom.com at lists.ntp.org] On
> >> Behalf Of STUART VENTERS
> >> Sent: Friday, May 02, 2008 7:52 AM
> >> To: ntpwg at lists.ntp.org
> >> Subject: Re: [ntpwg] Testing NTP performance
> >>
> >>
> >> >>Not to get too philosophical, but isn't this a description of the
> >> >>precise problem that NTP itself is designed to solve?
> >>
> >> >>Rob Seaman
> >> >>NOAO
> >> >>--
> >>
> >>
> >> Rob,
> >>
> >> Guilty as charged, there is circular logic here.
> >> I'm using NTP's ability to transport time to measure same.
> >>
> >> But perhaps I can get out on a technicality.
> >> I'm proposing to use the time transport ability over easy links
> >> to measure the time transport ability over a more
> >> difficult link.
> >>
> >> I'd be happy to say that the only unit under test is A
> >> and so B and C can be 'Golden units'.
> >>
> >> I need to test down to sub-millisecond accuracy.
> >> There are servers available for B with 100nS accuracy.
> >> There are packet bert testers that could pinch hit as a
> >> client C with similar accuracy.
> >> These should be sufficient to get us to uS accuracy.
> >>
> >> Again, this seems like a complicated plan, there ought to be
> >> an easier way.
> >>
> >> Asked another way, folks here have claimed that NTP runs
> >> with particular accuracies much tighter
> >> than a human can check by printing out the time on a console.
> >>
> >> How do they know?
> >>
> >>
> >> Regards,
> >>
> >> Stuart
> >>
> >>
> >>
> >>
> >>
> >> > Greetings,
> >> >
> >> > I'm puzzling over how to test embedded NTP implementations.
> >> >
> >> > Given a client and server connected through some test network,
> >> > how do you accurately measure the difference between the
> >> server and
> >> > client clocks.
> >> >
> >> > If there was a 1PPS test wire coming out of each
> implementation, it
> >> > would be easy.
> >> > But unless you can add code to the client, there is
> usually not a
> >> > wire,
> >> >
> >> > I need some way to peek into the client and server to see their
> >> > internal clocks.
> >> >
> >> > I'm thinking the server functionality embedded with the
> >> client might
> >> > help.
> >> >
> >> >
> >> > Consider the following system with a third NTP box:
> >> >
> >> >
> >> > (C)
> >> > CLIENT
> >> > / \
> >> > / \
> >> > / \
> >> > / \
> >> > / \
> >> > /
> >> > SERVER ---------- TEST ------ CLIENT/SERVER
> >> > (B) NETWORK (A)
> >> >
> >> >
> >> >
> >> > We are trying to measure the effects of the test network on
> >> the client
> >> > in A.
> >> >
> >> > Server B provides a stable timing reference for the test.
> >> >
> >> > Client/Server A is locked to Server B through the test network.
> >> >
> >> > Client C is connected to servers A and B.
> >> >
> >> >
> >> > If the links connecting Client C are simple point to point
> >> links, then
> >> > hopefully most of the errors in time transport
> >> > are due to the test network affecting A.
> >> >
> >> > C provides an offset between B&C and also between A&C.
> >> >
> >> > It seems like the difference between these offsets should be the
> >> > offset between A&B.
> >> >
> >> > This seems like a complicated plan, is there a better way?
> >> >
> >> >
> >> > Regards,
> >> >
> >> > Stuart
> >> >
> >> >
> >> > ps: My apologies for an NTP 101 question.
> >> > Perhaps someone could point me to where this has already been
> >> > sorted out.
> >>
> >>
> >> _______________________________________________
> >> ntpwg mailing list
> >> ntpwg at lists.ntp.org
> >> https://lists.ntp.org/mailman/listinfo/ntpwg
> >>
> > _______________________________________________
> > ntpwg mailing list
> > ntpwg at lists.ntp.org
> > https://lists.ntp.org/mailman/listinfo/ntpwg
>
>
More information about the ntpwg
mailing list