A Minimal IOCP Server on Windows

The short version: IOCP is the completion port model on Windows. The shape is: create the listening socket with WSASocketW and WSA_FLAG_OVERLAPPED, create the completion port, associate each connection socket with it, submit a WSARecv, then pick up completions in the event threads with GetQueuedCompletionStatus.

Why IOCP

IOCP is the completion port model Microsoft recommends on Windows. Any network server that needs to perform well on Windows runs into it.

It differs from Linux epoll and FreeBSD kqueue in when it fires:

• epoll and kqueue usually fire when a buffer has content. epoll defaults to level-triggered, and EPOLLET switches it to edge-triggered.
• IOCP submits an overlapped I/O request and fires a completion event once the I/O finishes, meaning a certain number of bytes were read or written.

Correction: an earlier version of this post described epoll as edge-triggered only. That was wrong. epoll supports both level-triggered (the default) and edge-triggered.

There is also a trap around reusing TCP connections under IOCP, which this post does not cover.

The core API

IOCP needs overlapped I/O, so the calls involved are these.

WSASocketW

SOCKET WSAAPI WSASocketW(
  [in] int                 af,
  [in] int                 type,
  [in] int                 protocol,
  [in] LPWSAPROTOCOL_INFOW  lpProtocolInfo,
  [in] GROUP               g,
  [in] DWORD               dwFlags
);

You create the listening socket with this, and the argument that matters is the last one. Set dwFlags to WSA_FLAG_OVERLAPPED to mark the socket as taking part in overlapped I/O.

Correction: an earlier version said overlapped I/O could be understood as an identifier or GUID for a TCP connection. That was wrong. WSA_FLAG_OVERLAPPED enables overlapped I/O. It is not a connection identifier.

ioctlsocket

int ioctlsocket(
  [in]      SOCKET s,
  [in]      long   cmd,
  [in, out] u_long *argp
);

Used to set non-blocking I/O. Not much more to say here.

CreateIoCompletionPort

HANDLE WINAPI CreateIoCompletionPort(
  _In_     HANDLE    FileHandle,
  _In_opt_ HANDLE    ExistingCompletionPort,
  _In_     ULONG_PTR CompletionKey,
  _In_     DWORD     NumberOfConcurrentThreads
);

Creates the completion port kernel object and returns its handle.

• FileHandle: the file or socket handle to associate. Pass INVALID_HANDLE_VALUE the first time, to create the port.
• ExistingCompletionPort: an existing port handle. Pass nullptr the first time.
• CompletionKey: the completion key, which this post does not use.
• NumberOfConcurrentThreads: the ceiling on threads processing completions at once, not the size of a thread pool.

PostQueuedCompletionStatus

BOOL WINAPI PostQueuedCompletionStatus(
  _In_     HANDLE       CompletionPort,
  _In_     DWORD        dwNumberOfBytesTransferred,
  _In_     ULONG_PTR    dwCompletionKey,
  _In_opt_ LPOVERLAPPED lpOverlapped
);

Posts an event to the worker threads. This example uses it as a shutdown signal by passing -1 as the second argument.

GetQueuedCompletionStatus

BOOL GetQueuedCompletionStatus(
  [in]  HANDLE       CompletionPort,
        LPDWORD      lpNumberOfBytesTransferred,
  [out] PULONG_PTR   lpCompletionKey,
  [out] LPOVERLAPPED *lpOverlapped,
  [in]  DWORD        dwMilliseconds
);

Receives the completions posted by PostQueuedCompletionStatus, WSARecv, and WSASend. It takes one more argument than the Post version: a timeout.

WSARecv / WSASend

int WSAAPI WSARecv(
  [in]      SOCKET                             s,
  [in, out] LPWSABUF                           lpBuffers,
  [in]      DWORD                              dwBufferCount,
  [out]     LPDWORD                            lpNumberOfBytesRecvd,
  [in, out] LPDWORD                            lpFlags,
  [in]      LPWSAOVERLAPPED                    lpOverlapped,
  [in]      LPWSAOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine
);

int WSAAPI WSASend(
  [in]  SOCKET                             s,
  [in]  LPWSABUF                           lpBuffers,
  [in]  DWORD                              dwBufferCount,
  [out] LPDWORD                            lpNumberOfBytesSent,
  [in]  DWORD                              dwFlags,
  [in]  LPWSAOVERLAPPED                    lpOverlapped,
  [in]  LPWSAOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine
);

The Winsock versions of recv and send, with a few extra arguments.

The two custom types

enum class IOType {
    Read,
    Write
};

IOType identifies the operation. This example only handles Read.

#define MaxBufferSize 1024

struct IOContext {
    OVERLAPPED overlapped{};
    WSABUF wsaBuf{MaxBufferSize, buffer};
    CHAR buffer[MaxBufferSize]{};
    IOType type{};
    SOCKET socket = INVALID_SOCKET;
    DWORD nBytes = 0;
};

OVERLAPPED must be the first member of IOContext, because GetQueuedCompletionStatus returns a pointer you cast to OVERLAPPED*. That same pointer then reaches the whole context.

The flow

1. WSAStartup to initialize Winsock.
2. Create the listening socket with WSASocketW and WSA_FLAG_OVERLAPPED, then bind and listen.
3. Create the completion port with CreateIoCompletionPort.
4. Start AcceptWorkerThread, which accepts connections, associates them with the port, and submits the first WSARecv.
5. Start NumberOfThreads instances of EventWorkerThread, which call GetQueuedCompletionStatus and handle completions.
6. Signal shutdown with PostQueuedCompletionStatus(hIOCP, -1, ...) and join the threads.

The full example, with the bugs annotated

The logic is unchanged from the original. // [bug] marks the places that are wrong. The GitHub repository is the source of truth.

GitHub - SHIINASAMA/IOCP

/// \author kaoru
/// \file main.cpp
/// \version 0.2
/// \date 2022.7.21
/// \brief IOCP example

#include <WinSock2.h>
#pragma comment(lib, "ws2_32.lib")
#include <WS2tcpip.h>
#include <atomic>
#include <cstdint>
#include <thread>
#include <vector>

// [bug] The original wrote 1024 * 0.5 = 512 here, which contradicts
//       the #define MaxBufferSize 1024 above it.
constexpr static size_t MaxBufferSize = 1024 * 0.5;
constexpr static size_t NumberOfThreads = 1;

HANDLE hIOCP = INVALID_HANDLE_VALUE;
SOCKET serverSocket = INVALID_SOCKET;
std::vector<std::thread> threadGroup;
std::atomic_bool isShutdown{false};

// Accepts connections in a loop and posts one Read event for each
void AcceptWorkerThread();
// Handles the events that AcceptWorkerThread posts
void EventWorkerThread();

// Identifies the event type
enum class IOType {
    Read,
    Write
};

struct IOContext {
    OVERLAPPED overlapped{};
    WSABUF wsaBuf{MaxBufferSize, buffer};
    CHAR buffer[MaxBufferSize]{};
    IOType type{};
    SOCKET socket = INVALID_SOCKET;
    DWORD nBytes = 0;
};

int main() {
    // Initialize Winsock
    WSAData data{};
    WSAStartup(MAKEWORD(2, 2), &data);

    struct sockaddr_in address {};
    address.sin_family = AF_INET;
    address.sin_port = htons(8080);
    inet_pton(AF_INET, "127.0.0.1", &address.sin_addr);

    // Initialize the socket
    unsigned long ul = 1;
    serverSocket = WSASocketW(AF_INET, SOCK_STREAM, 0, nullptr, 0, WSA_FLAG_OVERLAPPED);
    if (INVALID_SOCKET == serverSocket) {
        // [bug] perror reads errno. Winsock errors need WSAGetLastError() / FormatMessage.
        perror("FAILED TO CREATE SERVER SOCKET");
        closesocket(serverSocket);
        exit(-1);
    }
    // [bug] Making the listening socket non-blocking makes accept spin in
    //       AcceptWorkerThread below. See the note there.
    if (SOCKET_ERROR == ioctlsocket(serverSocket, FIONBIO, &ul)) {
        perror("FAILED TO SET NONBLOCKING SOCKET");
        closesocket(serverSocket);
        exit(-2);
    }
    if (SOCKET_ERROR == bind(serverSocket, (const struct sockaddr *) &address, sizeof(address))) {
        perror("FAILED TO BIND ADDRESS");
        closesocket(serverSocket);
        exit(-3);
    }
    if (SOCKET_ERROR == listen(serverSocket, SOMAXCONN)) {
        perror("FAILED TO LISTEN SOCKET");
        closesocket(serverSocket);
        exit(-4);
    }

    // Initialize the completion port
    hIOCP = CreateIoCompletionPort(INVALID_HANDLE_VALUE, nullptr, 0, NumberOfThreads);
    if (INVALID_HANDLE_VALUE == hIOCP) {
        perror("FAILED TO CREATE IOCP HANDLE");
        closesocket(serverSocket);
        exit(-5);
    }

    // Start the worker threads
    for (size_t i = 0; i < NumberOfThreads; i++) {
        threadGroup.emplace_back(std::thread(EventWorkerThread));
    }

    void *lpCompletionKey = nullptr;
    auto acceptThread = std::thread(AcceptWorkerThread);
    getchar();
    // Any key begins shutdown
    isShutdown = true;
    // One post per thread, so every worker receives an event and exits on its own
    for (size_t i = 0; i < NumberOfThreads; i++) {
        PostQueuedCompletionStatus(hIOCP, -1, (ULONG_PTR) lpCompletionKey, nullptr);
    }
    acceptThread.join();
    for (auto &thread: threadGroup) {
        thread.join();
    }

    WSACleanup();
    return 0;
}

void AcceptWorkerThread() {
    while (!isShutdown) {
        // [bug] The listening socket is non-blocking, so accept returns
        //       WSAEWOULDBLOCK immediately when there is no connection and this
        //       continue spins the thread at full speed. A listening socket does
        //       not need to be non-blocking.
        SOCKET clientSocket = accept(serverSocket, nullptr, nullptr);
        if (INVALID_SOCKET == clientSocket) continue;

        unsigned long ul = 1;
        if (SOCKET_ERROR == ioctlsocket(clientSocket, FIONBIO, &ul)) {
            shutdown(clientSocket, SD_BOTH);
            closesocket(clientSocket);
            continue;
        }

        if (nullptr == CreateIoCompletionPort((HANDLE) clientSocket, hIOCP, 0, 0)) {
            shutdown(clientSocket, SD_BOTH);
            closesocket(clientSocket);
            continue;
        }

        DWORD nBytes = MaxBufferSize;
        DWORD dwFlags = 0;
        auto ioContext = new IOContext;
        ioContext->socket = clientSocket;
        ioContext->type = IOType::Read;
        auto rt = WSARecv(clientSocket, &ioContext->wsaBuf, 1, &nBytes, &dwFlags, &ioContext->overlapped, nullptr);
        auto err = WSAGetLastError();
        if (SOCKET_ERROR == rt && ERROR_IO_PENDING != err) {
            // An error other than ERROR_IO_PENDING
            shutdown(clientSocket, SD_BOTH);
            closesocket(clientSocket);
            delete ioContext;
        }
    }
}

void EventWorkerThread() {
    IOContext *ioContext = nullptr;
    DWORD lpNumberOfBytesTransferred = 0;
    void *lpCompletionKey = nullptr;

    DWORD dwFlags = 0;
    DWORD nBytes = MaxBufferSize;

    while (true) {
        BOOL bRt = GetQueuedCompletionStatus(
                hIOCP,
                &lpNumberOfBytesTransferred,
                (PULONG_PTR) &lpCompletionKey,
                (LPOVERLAPPED *) &ioContext,
                INFINITE);

        if (!bRt) continue;

        // The shutdown instruction posted by PostQueuedCompletionStatus
        if (lpNumberOfBytesTransferred == -1) break;

        // [bug] WSARecv completes with 0 bytes when the client disconnects.
        //       This continue neither closes the socket nor deletes ioContext,
        //       so both the connection and the memory leak.
        if (lpNumberOfBytesTransferred == 0) continue;

        // Bytes read, or bytes written in total
        ioContext->nBytes = lpNumberOfBytesTransferred;
        // Handle the event
        switch (ioContext->type) {
            case IOType::Read: {
                // Print what was read
                setbuf(stdout, nullptr);
                puts(ioContext->buffer);
                // fflush(stdout);
                // closesocket(ioContext->socket);
                // delete ioContext;
                // ioContext = nullptr;
                if (lpNumberOfBytesTransferred == MaxBufferSize) {
                    puts("post read request again");
                    int nRt = WSARecv(
                            ioContext->socket,
                            &ioContext->wsaBuf,
                            1,
                            &nBytes,
                            &dwFlags,
                            &(ioContext->overlapped),
                            nullptr);
                    auto e = WSAGetLastError();
                    // [bug] e != WSAGetLastError() is always false, because no
                    //       other Winsock call happens in between. The intent was
                    //       e != ERROR_IO_PENDING.
                    if (SOCKET_ERROR == nRt && e != WSAGetLastError()) {
                        // The read failed
                        closesocket(ioContext->socket);
                        delete ioContext;
                        ioContext = nullptr;
                    }
                }
            }
            // [bug] A missing break here drops the Read branch into Write.
            //       Write does nothing, so it is harmless, but it is a trap.
            case IOType::Write: {
                // This project had no use for it, so it is not handled
                break;
            }
        }
    }
}

The bugs

• perror: it reads the C errno, and Winsock errors need WSAGetLastError() with FormatMessage. Every perror here prints nothing useful.
• e != WSAGetLastError(): no Winsock call happens between the two, so the condition is always false and the error cleanup never runs. The intent was e != ERROR_IO_PENDING.
• Non-blocking listening socket with a spinning accept: FIONBIO makes the listening socket non-blocking, so accept returns WSAEWOULDBLOCK immediately when no connection is waiting, and the continue spins the thread at full speed.
• The 0-byte disconnect leak: a client disconnect completes with 0 bytes and the code continues straight past it, without closing the socket or deleting the context. Both the connection and the memory leak.
• The switch fallthrough: the Read branch has no break, so it falls into Write. Harmless because Write does nothing, but a trap.
• The inconsistent MaxBufferSize: the prose says #define MaxBufferSize 1024 and the full listing has 1024 * 0.5, which is 512.

Summary

Using IOCP comes down to a few steps:

• Initialize the socket and the completion port
• Start the accept thread
• Start the event threads
• Design a shutdown path you trust

The accept thread is optional, and the main thread can do the job instead, though that is not usually a good idea. Think of the accept thread as the event producer: it accepts a connection and posts the first read. The event threads handle what the completion port hands them.

Related reading

• Teaching CMake to copy dependent DLLs on Windows, another Windows and C++ note
• See all technical notes
• Shiina Labs open source