Introduction
Even if Microsoft Windows is not a real-time operating system and cannot, by design, guarantee real-time constraints, it is sometimes necessary for software to provide a cyclic behavior with the best possible frequency accuracy. This article aims to compare some solutions provided by Microsoft Windows to manage time, time precision, and the impact of CPU overload on the frequency accuracy.
Terminology
In this article, each cycle has two parts: the active and the passive part. The active part represents the time spent doing some real computing in the application. The time spent waiting for the next cycle to begin is in the passive part. CPU resources should be freed by the current thread in the passive part to allow other processes to use it.
The period will be the duration between two beginning of cycles, including both the active part and passive part, and the frequency is defined as the reverse of the period.
For success rate calculation, we will consider as a success a cycle that lasts 100ms ± 1ms.
Environment
The experiment code is in C#, targeting .NET Core 3.1.
The laptop used is a Dell Precision M4700. The processor is an IntelCore i7-3820QM with a fixed frequency of 2.68Hz without turbo and hyperthreading.
Experiments description
Experiments are done on only one processor core by modifying the current process processor affinity at the beginning of the program (see ProcessorAffinity). The testing program has two threads, one running the expected cyclic behavior and the other simulating a punctual processor overload.
In each experiment, the period is 100ms, which means a frequency of 10 Hz. We recorded 600 cycles in which the processor overload is simulated from the 200th to the 400th cycle. By doing this, we can evaluate the frequency accuracy of our cyclic behavior. On the charts provided thereafter, this processor overload will be represented between vertical blue lines.
In the case of an external timer, we used a semaphore. Each time the timer raised an event, it gives a token to the working thread to allow it to do one cycle. The working thread executes its active part, and then wait for one semaphore resource, which represents its passive part. This behavior is shown in the next figure.
Remark
Measures are taken by the millisecond because it is not useful to be more accurate on a .NET program running under Windows OS, which is not a real-time OS.
Test cases
In this article, we will compare a few solutions:
- Measure the active part duration of the cycle. Use a Thread.Sleep to suspend the current thread for the rest of the period.
- The same manipulation as in 1. but setting thread priority to “AboveNormal” in the initialization.
- The same manipulation as in 1. but setting thread priority to “Highest” in the initialization.
- Use an external cyclic System.Timers.Timer provided by a C# library to trigger the start of each cycle. The timer thread is running in Normal priority.
- Use an external cyclic System.Timers.Timer provided by a C# library to trigger the start of each cycle. The timer thread is running as the Highest priority.
- Use the old Windows Multimedia Timer API to make an external cyclic timer with a resolution of 1ms.
- Use the old Windows Multimedia Timer API to make an external cyclic timer with a resolution of 20ms.
- Use ThreadPoolLegacyApiSet to make an external cyclic timer with a resolution of 1ms.
- Use ThreadPoolLegacyApiSet to make an external cyclic timer with a resolution of 20ms with a Normal priority thread.
- Use ThreadPoolLegacyApiSet to make an external cyclic timer with a resolution of 20ms with a Highest priority thread.
Results and Interpretation
Thread.Sleep with normal priority thread
We can distinguish two cases,
- When the processor is overloaded, most cycles last 140ms.
- When the processor is not overloaded, most of the cycles last exactly 100ms, which is the expected duration. However, there are many spikes for durations between 110 and 120ms, and a spike at 200ms.
We obtain a success rate of 52% with 1ms tolerance. The average period is 111.44ms and the standard deviation is 16.77ms, which are bad results that are even worse when the processor is overloaded.
|
Success rate (%)
|
Average period (ms)
|
Standard deviation (ms)
|
|
51.50
|
111.44
|
16.77
|
Thread.Sleep with AboveNormal priority
By using an above priority thread, 76.5% of cycles last 100ms ± 1ms. This is a great result. However, we can see many punctual spikes between 101 and 120ms, which represent 22.8% of cycles. The average is 102.6ms and the standard deviation is 5.32ms.
A surprising fact is that we obtain more accurate results when the processor is overloaded.
|
Success rate (%)
|
Average period (ms)
|
Standard deviation (ms)
|
|
76.50
|
102.60
|
5.32
|
Thread.Sleep with Highest priority
We obtain similar results as in part 3., but with a better accuracy: 81.5% of the cycles last 100ms ± 1ms. The average is 102.02ms and the standard deviation is 4.83ms, which are a bit better than with a thread priority set to AboveNormal.
We also see a better accuracy when the processor is overloaded:
|
Success rate (%)
|
Average period (ms)
|
Standard deviation (ms)
|
|
81.50
|
102.02
|
4.83
|
Use the classic C# external timer with a Normal priority for the timer thread
By using the classic C# external timer, we see an important difference when the processor is overloaded. The cycle duration repartition shows that only 41.5% of the cycles effectively last 100ms (± 1ms).
However, by looking at errors, we can observe that a delay is often followed or preceded by a “compensating” advance. Therefore, the total average is 101.12ms which is great when thinking about 58.5% of errors on cycle duration. Inversely, the standard deviation is 10.34ms, which is an important number compared to the previous cycle management (using a Thread.Sleep).
|
Success rate (%)
|
Average period (ms)
|
Standard deviation (ms)
|
|
41.50
|
101.12
|
10.34
|
Use the classic C# external timer with a Highest priority for the timer thread
When reading these results, the first observation is that this timer is a little less efficient with the highest priority thread than with a thread with a normal priority. That is surprising, and even more when looking at the chart where we could see that the timer is less performant when the processor is overloaded.
|
Success rate (%)
|
Average period (ms)
|
Standard deviation (ms)
|
|
37.83
|
100.79
|
12.04
|
Multimedia Timer API with a 1ms resolution
By using Windows Multimedia Timer API, the accuracy is exemplary. In most cases, each cycle lasts 99ms with regular spikes at 100ms. A thread overload has no impact on cycle duration, except for the first cycle where it implies an advance of 44ms. By considering 99ms as a valid value for the period, we have a success rate of 99%.
However, these results are obtained with a 1ms resolution. We can nuance them by looking at the article “
Why are the Multimedia Timer APIs (timeSetEvent) not as accurate as I would expect?”. The main default of this timer is related to its working principle. Setting the resolution to 1ms reduces the period between two calls to the OS scheduler. Consequently, the proportion of time where CPU is allocated to the scheduler increases, and other processes have less time to be executed. In brief, increasing timer accuracy implies a global system slowing down. In case of OS saturation (that means when processes ask for more CPU time than what is available), the timer will become useless due to high response time.
Following this article, we can also see the advantage of a timer that works on fixed dates. Indeed, the next cycle will come faster if the timer is late for one event. In the worst case, many events might be raised at the same time. This is a good point for monitoring applications where cumulative lateness is worse than punctual delays compensated by an advance on the next cycle.
This article also shows that there will regularly be delays until 20ms. It is a Windows constraint. Whatever timer we use, running a program under this OS makes it impossible to guarantee an accuracy better than 20ms.
Nonetheless, Microsoft does not support this timer anymore, recommending to use the ThreadPoolLegacyApiSet instead. We will come back on this API further in this document.
Note
The timer thread priority is Highest by default for Multimedia Timers.
|
Success rate (%)
|
Average period (ms)
|
Standard deviation (ms)
|
|
99.00
|
99.04
|
1.49
|
Multimedia Timer API with a 20ms resolution
By using this timer with a 20ms resolution, which is 20 times less demanding for the OS, we obtain a satisfying global accuracy. 80.33% of the cycles last 100ms ± 1ms. 99% of bad cycle durations have an error which value is lower than 20ms, which is a nice value when looking at the article referenced in part 5. All delays of more than 20ms are compensated in the following cycle. The standard deviation is 4.63ms, which is an admissible value.
|
Success rate (%)
|
Average period (ms)
|
Standard deviation (ms)
|
|
80.33
|
99.62
|
4.63
|
Use ThreadPoolLegacyApiSet with a 1ms resolution
We can easily distinguish the part where the processor is overloaded of the one where it is not. In this last case, cycle duration is quasi constant with a duration of 99ms, like its predecessor the Multimedia Timer API. However, 75% of cycles last around 95ms when the processor is overloaded, and the remaining 25% cycles have punctual delays that compensate exactly this cumulated advance. Therefore, the mean is around 99ms, and the standard deviation is 6.37ms.
Note
The timer thread priority is Normal.
|
Success rate (%)
|
Average period (ms)
|
Standard deviation (ms)
|
|
66.50
|
99.04
|
6.37
|
Using ThreadPollingLegacyApiSet with a 20ms resolution and Normal thread priority
When looking at the chart, we can see that the cycle duration is not the same for each cycle and the variation is important. The mean duration is 99.7ms. It is nearly the expected duration of 100ms but the standard deviation is 6.64ms, which is quite elevated.
Nonetheless, these results are unsatisfying compared to other results. Even the Multimedia Timer which is no longer supported has better results in terms of success rate and standard deviation, but this is normal due to timer thread priority. Indeed, the Multimedia timer uses the Highest priority thread to manage its timer, while the ThreadPoolLegacyApiSet only uses a Normal priority one. So, we must increase the ThreadPoolLegacyApiSet thread priority to have comparable results.
|
Success rate (%)
|
Average period (ms)
|
Standard deviation (ms)
|
|
67.83
|
99.69
|
6.64
|
Using ThreadPollingLegacyApiSet with a 20ms resolution and Highest thread priority
On these results, we can see that the timer thread priority has a huge impact on performances. Unlike on Normal priority, executing in Highest priority significantly increases the success rate and the standard deviation from (respectively) 67.8% to 99.72% and from 6.6ms to 3.36ms.
However, there is no way to increase properly the thread priority from Normal to Highest because it is hidden beside the imported API. To make this test, we changed the currently executing thread priority to Highest when the timer event is received, like on the line of code above.
Thread.CurrentThread.Priority =ThreadPriority.Highest;
|
Success rate (%)
|
Average period (ms)
|
Standard deviation (ms)
|
|
87.83
|
99.72
|
3.36
|
Experience results in one table:
| |
Success rate (%)
|
Average period (ms)
|
Standard deviation (ms)
|
Advantages
|
Disadvantages
|
|
1: Thread.Sleep, Normal thread priority
|
51.50
|
111.44
|
16.77
|
Easy to develop
|
Fluctuating period
Incorrect average
Not resilient to concurrency
|
|
2: Thread.Sleep, AboveNormal thread priority
|
76.50
|
102.60
|
5.32
|
Easy to develop
Resilient to concurrency
|
Fluctuating period
Incorrect average
|
|
3: Thread.Sleep, Highest thread priority
|
81.50
|
102.02
|
4.83
|
Easy to develop
Almost stable period
Resilient to concurrency
|
Incorrect average
|
|
4: Using C# external timer with Normal priority for the timer thread
|
41.50
|
101.12
|
10.34
|
Easy to develop
Almost correct average
|
Fluctuating period
Not resilient to concurrency
|
|
5: Using C# external timer with Highest priority for the timer thread
|
37.83
|
100.79
|
12.04
|
Moderate development
Correct average
|
Fluctuating period
Not resilient to concurrency
|
|
6: Using external Multimedia Timer with 1ms resolution
|
99.00
|
99.04
|
1.49
|
Stable period
Correct average
Resilient to thread concurrency
|
Difficult to develop
May cause a global system slowing down or freezing
|
|
7: Using external Multimedia Timer with 20ms resolution
|
80.33
|
99.62
|
4.63
|
Almost stable period
Correct average
Resilient to concurrency
|
Difficult to develop
|
|
8: Using a timer based on ThreadPoolLegacyApiSet with 1ms resolution
|
66.50
|
99.04
|
6.37
|
Without concurrency:
Stable period
Correct average
|
Difficult to develop
Not resilient to concurrency
May cause a global system slowing down or freezing
|
|
9: Using a timer based on ThreadPoolLegacyApiSet with 20ms resolution and Normal thread priority
|
67.83
|
99.69
|
6.64
|
Without concurrency:
Almost stable period
Correct average
|
Difficult to develop
Not resilient to concurrency
|
|
10: Using a timer based on ThreadPoolLegacyApiSet with 20ms resolution and Highest thread priority
|
87.83
|
99.72
|
3.36
|
Almost stable period
Correct average
Resilient to thread concurrency
|
Difficult to develop
|
Conclusion
Microsoft provides several APIs to manage cyclic aspects of an application developed within Windows OS. However, when looking at the accuracy, we can see that depending on our application the choice may be important. In fact, for managing time to wait to provide a cyclic aspect, the more instinctive choice to do is to use an external timer.
Timers provide effectively the right average period, but the standard deviation could be pretty much heavy than the one provided by simply using a call to Thread.Sleep at the end of each cycle for waiting during the remaining time. The Thread.Sleep implies a greater average that will be a little over the expected one, but its standard deviation is remarkably lower when increasing the thread priority.
Without manually changing thread priority, the Multimedia Timer has better results than its successor, the ThreadPoolLegacyApiSet, but this is due to the default timer thread priority. The Multimedia Timer has a timer thread with Highest priority while the ThreadPoolLegacyApiSet has a normal priority timer thread. By manually changing the thread priority on this last timer to Highest, we obtain the best results of the experiment. However, the only way to do that is to set the “Thread.CurrentThread.Priority” when the timer is initializing, which could be seen as a bad practice.
One other bad aspect of Multimedia Timer and the timer based on ThreadPoolLegacyApiSet is that overriding the resolution to have better results implies a global system slowing down.
To conclude, there are our recommendations by use case,
- For those who want a cyclic behavior with high accuracy on the frequency and which task has a high priority and want a solution easy to develop and to maintain: use a Thread.Sleep and increase the thread priority to AboveNormal or Highest.
- For those who want a cyclic behavior with high accuracy on the frequency and which task has a high priority and want the best possible results: use the ThreadPoolLegacyApiSet with a resolution over 20ms by setting the thread priority to Highest at timer initialization.
- For those who want a cyclic behavior for a normal or low priority task for which frequency accuracy is not important: use the C# provided Timer (System.Timers.Timer) which is the simplest to use and maintain.
- For those who want a cyclic behavior for a normal or low priority task for which frequency accuracy is important, with limited CPU resource and high process concurrency: use the ThreadPoolLegacyApiSet with a resolution over 20ms.
- For those who want a cyclic behavior for a normal or low priority task for which frequency accuracy is important and for which CPU resource is not a limitation: use the ThreadPoolLegacyApiSet with a resolution bellow 20ms.
Acknowledgment
This work has been done within Agileo Automation for an Industrial Internet of Things (IIoT) application. The aim was to analyze the best possible choice to implement cyclic data collection plans for industrial equipment data acquisition within
Agileo Automation Equipment Controller Framework (A²ECF).