Wed Nov 12 12:39:41 EST 2008

More about timing with Python

Ok, I'm learning more about getting precise calendar timing in python so I thought I would document it a bit.

Python has the commonly used time module time. time.time() returns seconds since the epoch (Jan 1, 1970). On Mac/Linux you get values with meaningful fractional seconds to 1 microsecond precision. I think, on these OSs the gettimeofday() C function is used to implement time.time().

On Windows, the gettimeofday() call is not available to time.time(). It is not exactly clear to me at this point what function time.time() calls on Windows, however, the result gives values reported to 1 microsecond, but only values to 1 millisecond are meaningful (the latter three digits are typically 0’s). Moreover, although the time.time() call on windows produces values that change with 1 millisecond precision, these values are not actually updated every millisecond. It seems the rate at which they are updated is hardware dependent, many report an update rate between 10 and 20 ms.

In addition to time.time(), there is time.clock(). On Linux/Mac systems this call is not helpful, as it produces a time value in seconds of CPU time. However on Windows time.clock() gives the seconds since the process was started and these are reported to the precision of the operating system. On Windows, time.clock() uses the QueryPerformanceCounter() method which measures time based on CPU clock cycles. To use time.clock() as a wall-clock (i.e. calendar time), one must then initialize it with call when the process executes, coinciding with a time.time() call to get the wall-clock date-time stamp. Then subsequent time stamps can be measured accurately by subsequent calls to time.clock() which then gives the offset to the initial time stamp. This general method is implemented in code posted in this thread.

This is not the end of the story however. For a really good read on these issues, have a look at When Microseconds Count. Here the folks at IBM tackle the problem much the same way, in a C library, that handles a few of the other ugly bits we've overlooked. The biggest issues, as I can see, in the python code snippet posted at the link above, that’s addressed in this IBM article directly, is the fact that time.clock() returns a 64 bit value which will eventually wrap for long-running processes. Unfortunately, for whatever reason, the code and executables once accompanying this article have been removed and are listed as retired. I'm not sure what that means, perhaps there is now a better way.

Finally, depending on how QueryPerformanceCounter is called, on a multi-processor (and maybe multi-core?) machine, you may get varying results. There is a mention of this in [this documentation.] (http://msdn.microsoft.com/en-us/library/ms644904.aspx)

Finally as an aside note, the python module timeit uses time.clock() on Windows and time.time() on everything else.


Posted by vschmidt | Permanent link | File under: mac, windows, linux, python