[ntpwg] Testing NTP performance

TS Glassey tglassey at earthlink.net
Fri May 2 15:56:07 UTC 2008 

Greg -

----- Original Message ----- 
From: "Greg Dowd" <GDowd at symmetricom.com>
To: "STUART VENTERS" <stuart.venters at adtran.com>; <ntpwg at lists.ntp.org>
Sent: Friday, May 02, 2008 8:31 AM
Subject: Re: [ntpwg] Testing NTP performance


>I think the testing methodology is quite dependent on what you are
> trying to observe.  Are you looking for the absolute accuracy of a
> client with respect to UTC, accuracy wrt server, relative accuracy
> between 2 clients, or a pool of clients, etc.  Or, are you testing a
> server's ability to put time on a network?  Are you looking for p-p
> error, convergence time, rms.  Are you testing some application which
> would add its own error on top?

Remember that NTP is a SW product as it is distributed today from the land 
of Oz and that SW is by definition 'an emulation of something' - an idea we 
seem to have forgotten across our culture today.

>
> For my testing, I'm usually testing the egress port of a server for
> accuracy with respect to the server clock and jitter.  For that, I use
> custom designed ntp "probes".  These probes are pseudo clients,
> basically boxes with hardware timestamping synchronized to gps or house
> pps and then pointed at a single server and querying at 1 packet per
> second.

Greg does Symmetricom publish the results of this testing?

> The probes record the statistics using peerstats and rawstats
> which I can then load into a custom statistics package we have called
> TimeMonitor Analyzer.  Phase plots, histograms, ADEV, TDEV, minTDEV,
> MTIE at your fingertips.  After getting a baseline, I can start loading
> up ntp traffic, network traffic, etc and continue to sample.
>
> I am not a big fan of just testing pps signals as they don't tell all
> the story.  They give you the end to end performance but don't let you
> break down the error into discrete components to analyze.  But, it can
> be useful.  It isn't too hard to hack linux these days to capture
> timestamps or toggle serial lines.
>
> I have also tested the standard performance of linux clients running
> ntpd against our servers.  In that case, one great way is to plug a gps
> receiver into the slave, with appropriate refclock, and run the query
> against the server notrust.  Log the stats and analyze.
>
> If you can't do that, put a known good ntp server right on the switch
> port next to your box and then test the server under test with notrust.
>
> If you are testing the client, the easy way is to turn on stats if
> possible.  Not absolutely accurate, but they give good data.
>
> When testing networks, another thing I like to use are Ethernet taps.
> These guys let you "sample" the flow at any point when you are testing
> loaded networks (a la g.8261).  Then you can map the packet delay
> variation, and its subsequent impact on the client servo.

Yes but this is only true if the local network is set up for portspanning 
between the TAP Port and the TEST Port. Most people today use switches for 
their timing service networks and that means that the ports isolate 
same-subnet traffic to the specific ports it is flowing to/from on that 
switch. So unless portspanning is used to turn the listener port on properly 
or a Hub is used instead of a switch, this wont necessaryily work.

>
> I have not been overly impressed with the current generation of network
> testers (e.g., spirent or ixia) for their ability to generate packets
> with timestamps embedded or to manipulate timestamp fields.  But I think
> that is getting better.  You really, really have to watch out for how
> the testers are configured when setting up test nets.  It is very easy
> to get standing wave patterns in your network that trash the test.
> Really easy.  Too easy.  And very time-consuming :-)
>
>
> Greg Dowd
> gdowd at symmetricom dot com (antispam format)
> Symmetricom, Inc.
> www.symmetricom.com
> "Everything should be made as simple as possible, but no simpler" Albert
> Einstein
>
>
>> -----Original Message-----
>> From: ntpwg-bounces+gdowd=symmetricom.com at lists.ntp.org
>> [mailto:ntpwg-bounces+gdowd=symmetricom.com at lists.ntp.org] On
>> Behalf Of STUART VENTERS
>> Sent: Friday, May 02, 2008 7:52 AM
>> To: ntpwg at lists.ntp.org
>> Subject: Re: [ntpwg] Testing NTP performance
>>
>>
>> >>Not to get too philosophical, but isn't this a description of the
>> >>precise problem that NTP itself is designed to solve?
>>
>> >>Rob Seaman
>> >>NOAO
>> >>--
>>
>>
>> Rob,
>>
>> Guilty as charged, there is circular logic here.
>>    I'm using NTP's ability to transport time to measure same.
>>
>> But perhaps I can get out on a technicality.
>>    I'm proposing to use the time transport ability over easy links
>>        to measure the time transport ability over a more
>> difficult link.
>>
>> I'd be happy to say that the only unit under test is A
>>   and so B and C can be 'Golden units'.
>>
>> I need to test down to sub-millisecond accuracy.
>>   There are servers available for B with 100nS accuracy.
>>   There are packet bert testers that could pinch hit as a
>> client C with similar accuracy.
>>   These should be sufficient to get us to uS accuracy.
>>
>> Again, this seems like a complicated plan, there ought to be
>> an easier way.
>>
>> Asked another way, folks here have claimed that NTP runs
>> with particular accuracies much tighter
>>    than a human can check by printing out the time on a console.
>>
>> How do they know?
>>
>>
>> Regards,
>>
>> Stuart
>>
>>
>>
>>
>>
>> > Greetings,
>> >
>> > I'm puzzling over how to test embedded NTP implementations.
>> >
>> > Given a client and server connected through some test network,
>> >   how do you accurately measure the difference between the
>> server and
>> > client clocks.
>> >
>> > If there was a 1PPS test wire coming out of each implementation, it
>> > would be easy.
>> >   But unless you can add code to the client, there is usually not a
>> > wire,
>> >
>> > I need some way to peek into the client and server to see their
>> > internal clocks.
>> >
>> > I'm thinking the server functionality embedded with the
>> client might
>> > help.
>> >
>> >
>> > Consider the following system with a third NTP box:
>> >
>> >
>> >                        (C)
>> >                       CLIENT
>> >                    /         \
>> >                  /             \
>> >                /                 \
>> >              /                     \
>> >            /                         \
>> >          /
>> >      SERVER ---------- TEST  ------ CLIENT/SERVER
>> >        (B)            NETWORK           (A)
>> >
>> >
>> >
>> > We are trying to measure the effects of the test network on
>> the client
>> > in A.
>> >
>> > Server B provides a stable timing reference for the test.
>> >
>> > Client/Server A is locked to Server B through the test network.
>> >
>> > Client C is connected to servers A and B.
>> >
>> >
>> > If the links connecting Client C are simple point to point
>> links, then
>> > hopefully most of the errors in time transport
>> >   are due to the test network affecting A.
>> >
>> > C provides an offset between B&C and also between A&C.
>> >
>> > It seems like the difference between these offsets  should be the
>> > offset between A&B.
>> >
>> > This seems like a complicated plan, is there a better way?
>> >
>> >
>> > Regards,
>> >
>> > Stuart
>> >
>> >
>> > ps: My apologies for an NTP 101 question.
>> >     Perhaps someone could point me to where this has already been
>> > sorted out.
>>
>>
>> _______________________________________________
>> ntpwg mailing list
>> ntpwg at lists.ntp.org
>> https://lists.ntp.org/mailman/listinfo/ntpwg
>>
> _______________________________________________
> ntpwg mailing list
> ntpwg at lists.ntp.org
> https://lists.ntp.org/mailman/listinfo/ntpwg

More information about the ntpwg mailing list