Every .NET/.NET Core application is associated with a thread pool and is a collection of threads that effectively executes asynchronous calls of an application. These threads on the thread pool are known as Worker threads.
These are worker threads are further divided logically into two types, one is worker threads which are used to process CPU-intensive logic i.e., it mostly used CPU to execute its logic like any calculations, etc.., and another is I/O threads, these threads are used to perform I/O requests or time-consuming requests such as read/write to file system, calling Database or calling any third party API/request.
Every .NET application when initialized, .NET runtime will spawn a minimum number of worker threads and keep those on an associated thread pool to be available for the next task/requests. The number of threads to spawn is equal to the number of logical processors available on the system. For example, on 4 core processors, can be found, 8 logical processors. In this case, the runtime will spawn 8 worker threads and make them available on the thread pool. The max allowed worker threads are 32,767 by default.
We can update this minimum count with our value at our application startup as follows,
ThreadPool.SetMinThreads(100, 100);
Here the first parameter accepts the minimum threads that should be spawned and the second parameter refers to Completion Port Thread count. Next will discuss more on this completion port and how it is important for all Asynchronous calls.
We need to be careful while setting minimum worker thread count as too many worker threads means more memory usage and CPU context switching and leads to a halt in the application and restart is required. I recommend leaving default values for min and max thread counts for better performance. I will discuss more on the use cases where we need to set these values in my next article, this article will concentrate more on understanding the I/O completion port (IOCP).
What is IOCP and how it is related to .NET runtime?
Well IOCP stands for Input/Output Completion Port and it provides an elegant solution to the problem of writing scalable applications that use multithreading and asynchronous I/O.
Modern server application development should always consider scalability and here we will be having a couple of major challenges, one is, work should be distributed across the threads to leverage modern multiprocessor hosts. Second, handling I/O operations must be scheduled effectively to maximize responsiveness and throughput. The I/O Completion ports will effectively address both of these challenges.
IOCP will effectively handle multiple asynchronous IO requests. Every time an application initialized a .NET runtime will create a Completion Port object using the CreateIoCompletionPort function as follows.
HANDLE CreateIoCompletionPort(
HANDLE FileHandle,
HANDLE ExistingCompletionPort,
ULONG_PTR CompletionKey,
DWORD NumberOfConcurrentThreads
);
Every completion port will be associated with multiple file handles or file descriptors. Here filehandle represents either read/write to file on the disk or any network endpoint or TCP socket.
Once the completion port is created, OS will create and associate with a queue to maintain all I/O completion request packets so that the application worker thread can be allotted and take this request further to process.
Let, for example, we are performing some I/O operations asynchronously in our application. The .NET runtime will allocate an I/O thread on the completion port asynchronously and wait for the callback event to happen and returns to the main thread to make the application responsive and handle other functionality in parallel. Once this I/O request is done, the I/O thread will call the PostQueuedCompletionStatus function on IOCP to move the completion request packet to the associated queue (first-in-first-out (FIFO)) and back to the available thread pool.
Once the application worker thread is available will call the GetQueuedCompletionStatus function to get the completion packet from the IOCP queue and continue the further actions. NULL will be returned If there is no item on the queue.
The variable NumberOfConcurrentThreads will decide the number of asynchronous I/O threads that should concurrently execute on IOCP. The default value will be a number of processors associated with the host or we can set this value as shown above by providing via the second parameter to the SetMinThreads method.
If there are no more asynchronous I/O threads are running that are no file handles the IOCP will be released by the system. So, we need to make sure to properly close all the I/O operation resources for better performance.
Hope this article makes you understand of IOCP architecture concept and the way it handling for all our asynchronous I/O calls in our server applications.
Happy Coding :)