CreateFile for Pipe Client fails with Error 231 ERROR_PIPE

I am trying to have two DLLs communicate with each other. One serving the purpose of a Pipe Server which stays active as long as the program runs, and the other as a Pipe Client which can connect and disconnect at given circumstances while the Pipe Server is active. Consider these functions respectfully:

DLL1 (Pipe server)

void DLL1_Begin() // This gets called only once, when the program starts
{
    g_hPipeServer = CreateNamedPipe(TEXT("\\\\.\\pipe\\MyPipe123"),
            PIPE_ACCESS_DUPLEX,
            PIPE_TYPE_BYTE | PIPE_READMODE_BYTE | PIPE_NOWAIT,
            1,
            4,
            4,
            NMPWAIT_USE_DEFAULT_WAIT,
            NULL);

    if (g_hPipeServer == INVALID_HANDLE_VALUE)
    {
        char error[256];
        sprintf_s(error, "Server pipe error %lu", GetLastError());
        MessageBox(NULL, error, "Fatal error", MB_OK | MB_ICONERROR);
        exit(1);
    }

    ConnectNamedPipe(g_hPipeServer, NULL);
}

void DLL1_End() // This gets called only once, when the program closes
{
    DisconnectNamedPipe(g_hPipeServer);
    CloseHandle(g_hPipeServer);
}

DLL2 (Pipe client)

// Like DLL1, DLL2 gets initialized once and is always attached to the program.
// These functions are called occasionally and are meant solely for the Pipe.

void DLL2_InitPipeClient()
{
    g_hClientPipe = CreateFile(TEXT("\\\\.\\pipe\\MyPipe123"),
        GENERIC_READ | GENERIC_WRITE,
        0,
        NULL,
        OPEN_EXISTING,
        0,
        NULL);

    if (g_hClientPipe == INVALID_HANDLE_VALUE) // This scope is of interest
    {
        char error[256];
        sprintf_s(error, "Client pipe error %lu", GetLastError());
        MessageBox(NULL, error, "Fatal error", MB_OK | MB_ICONERROR);
        exit(1);
    }
}

void DLL2_EndPipeClient()
{
    CloseHandle(g_hClientPipe);
    g_hClientPipe = NULL;
}

These are the steps:

Program starts, DLL1 gets attached and DLL1_Begin gets called
At any time, user interacts with button, DLL2 gets attached and DLL2_InitPipeClient gets called. DLL2 remains attached to the program as long as it's running
At any time, user interacts with another button, DLL2_EndPipeClient gets called, then DLL2_InitPipeClient gets called immediately after.
Client pipe fails to initialize with CreateFile for a second time, giving error 231

The user needs to be able to immediately "restart" the Client pipe whenever they want. Why would the server pipe be busy? CloseHandle always returns true, but after that, calling CreateFile returns INVALID_HANDLE_VALUE with error 231.

Note that I can only have one pipe client at a time. I tried setting maxinstances parameter for CreateNamedPipe to a higher value just in case, but the error doesn't go away and I can't initialize the Client pipe... How do I fix this?

By the way, communication works back and forth. I just can't get the the Client pipe to close and start again, and I can't have it active all the time, which is beyond me.

CodePudding user response：

According to [MS.Docs]: ConnectNamedPipe function (namedpipeapi.h) - Remarks (emphasis is mine):

A named pipe server process can use ConnectNamedPipe with a newly created pipe instance. It can also be used with an instance that was previously connected to another client process; in this case, the server process must first call the DisconnectNamedPipe function to disconnect the handle from the previous client before the handle can be reconnected to a new client. Otherwise, ConnectNamedPipe returns zero, and GetLastError returns ERROR_NO_DATA if the previous client has closed its handle or ERROR_PIPE_CONNECTED if it has not closed its handle.

I modified your sources to have a working example.

inc.h:

#pragma once

// Dumped everything here for convenience.
// In real world, each .dll would have its own header file(s).

#include <stdio.h>
#include <Windows.h>

#define PIPE_NAME "\\\\.\\pipe\\DemoPipe1234"

#if defined(_BUILD_NO_DLLS)
#  define DLL_EXPORT_API
#else
#  if defined(DLL_EXPORTS)
#    define DLL_EXPORT_API __declspec(dllexport)
#  else
#    define DLL_EXPORT_API __declspec(dllimport)
#  endif
#endif

#if defined(__cpluscplus)
extern "C" {
#endif

DLL_EXPORT_API void DLL1_Begin();
DLL_EXPORT_API void ReconnectServerEndpoint();
DLL_EXPORT_API void DLL1_End();

DLL_EXPORT_API void DLL2_InitPipeClient();
DLL_EXPORT_API void DLL2_EndPipeClient();

#if defined(__cpluscplus)
}
#endif

srv.c:

#define DLL_EXPORTS
#include "inc.h"


HANDLE g_hPipeServer = INVALID_HANDLE_VALUE;


void DLL1_Begin()
{
    //printf("%s - 0x6llX\n", __FUNCTION__, g_hPipeServer);
    g_hPipeServer = CreateNamedPipe(PIPE_NAME,
            PIPE_ACCESS_DUPLEX,
            PIPE_TYPE_BYTE | PIPE_READMODE_BYTE | PIPE_NOWAIT,
            1,
            4,
            4,
            NMPWAIT_USE_DEFAULT_WAIT,
            NULL);

    if (g_hPipeServer == INVALID_HANDLE_VALUE) {
        printf("Server pipe creation error %lu", GetLastError());
    }
}


void ReconnectServerEndpoint()
{
    if (g_hPipeServer == INVALID_HANDLE_VALUE) {
        printf("Can't reconnect invalid pipe\n");
        return;
    }
    if (!DisconnectNamedPipe(g_hPipeServer)) {
        printf("Server pipe disconnect error %lu\n", GetLastError());
    }
    if (!ConnectNamedPipe(g_hPipeServer, NULL)) {
        DWORD gle = GetLastError();
        if (gle != ERROR_PIPE_LISTENING) {
            printf("Server pipe connect error %lu\n", gle);
        }
    }
}


void DLL1_End()
{
    //printf("%s - 0x6llX\n", __FUNCTION__, g_hPipeServer);
    if (g_hPipeServer != INVALID_HANDLE_VALUE) {
        if (!DisconnectNamedPipe(g_hPipeServer)) {
            printf("Server pipe disconnect error %lu\n", GetLastError());
        }
        CloseHandle(g_hPipeServer);
        g_hPipeServer = INVALID_HANDLE_VALUE;
    }
}

cli.c:

// Like DLL1, DLL2 gets initialized once and is always attached to the program.
// These functions are called occasionally and are meant solely for the Pipe.

#define DLL_EXPORTS
#include "inc.h"


HANDLE g_hClientPipe = INVALID_HANDLE_VALUE;


void DLL2_InitPipeClient()
{
    //printf("%s - 0x6llX\n", __FUNCTION__, g_hClientPipe);
    g_hClientPipe = CreateFile(PIPE_NAME,
        GENERIC_READ | GENERIC_WRITE,
        0,
        NULL,
        OPEN_EXISTING,
        0,
        NULL);

    if (g_hClientPipe == INVALID_HANDLE_VALUE) {
        printf("Client pipe creation error %lu\n", GetLastError());
    }
}


void DLL2_EndPipeClient()
{
    //printf("%s - 0x6llX\n", __FUNCTION__, g_hClientPipe);
    if (g_hClientPipe != INVALID_HANDLE_VALUE) {
        CloseHandle(g_hClientPipe);
        g_hClientPipe = INVALID_HANDLE_VALUE;
    }
}

main00.c:

#include "inc.h"
#include <conio.h>

#if defined(_BUILD_NO_DLLS)
#  include "cli.c"
#  include "srv.c"
#endif


void loop()
{
    char c = 0;
    while (c != 0x1B) {
        //printf("\n************ MENU ************\nPress:\n- c to connect client\n- d to disconnect client\n- ESC to quit\n- Anything else to do nothing\n******************************\n");
        printf("\nPress: c to connect client, d to disconnect client, ESC to exit:\n");
        c = getch();
        switch (c) {
            case 0x1B: {
                printf("Exit\n");
                break;
            }
            case 0x43:
            case 0x63: {
                printf("Connect client\n");
                DLL2_InitPipeClient();
                break;
            }
            case 0x44:
            case 0x64: {
                printf("Disonnect client\n");
                DLL2_EndPipeClient();
                ReconnectServerEndpoint();
                break;
            }
            default: printf("Do nothing\n");
        }
    Sleep(500);
    }
}


int main()
{
    DLL1_Begin();

    loop();

    DLL1_End();
    printf("\nDone.\n\n");
    return 0;
}

Notes:

I added ReconnectServerEndpoint that should be called when a client disconnects
I linked the executable to the 2 .dlls. I don't know how are you supposed to do things maybe dynamically load the .dlls, but it shouldn't make any difference

Output:

[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q074753431]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###

[prompt]> "c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\VC\Auxiliary\Build\vcvarsall.bat" x64 > nul

[prompt]> dir /b
cli.c
inc.h
main00.c
srv.c

[prompt]> cl /nologo /MD /DDLL srv.c  /link /NOLOGO /DLL /OUT:dll1.dll
srv.c
   Creating library dll1.lib and object dll1.exp

[prompt]> cl /nologo /MD /DDLL cli.c  /link /NOLOGO /DLL /OUT:dll2.dll
cli.c
   Creating library dll2.lib and object dll2.exp

[prompt]> cl /nologo /MD /W0 main00.c  /link /NOLOGO /OUT:main_dll.exe dll1.lib dll2.lib
main00.c

[prompt]> dir /b
cli.c
cli.obj
dll1.dll
dll1.exp
dll1.lib
dll2.dll
dll2.exp
dll2.lib
inc.h
main00.c
main00.obj
main_dll.exe
srv.c
srv.obj

[prompt]>
[prompt]> main_dll.exe

Press: c to connect client, d to disconnect client, ESC to exit:
Connect client

Press: c to connect client, d to disconnect client, ESC to exit:
Disonnect client

Press: c to connect client, d to disconnect client, ESC to exit:
Connect client

Press: c to connect client, d to disconnect client, ESC to exit:
Disonnect client

Press: c to connect client, d to disconnect client, ESC to exit:
Connect client

Press: c to connect client, d to disconnect client, ESC to exit:
Disonnect client

Press: c to connect client, d to disconnect client, ESC to exit:
Connect client

Press: c to connect client, d to disconnect client, ESC to exit:
Connect client
Client pipe creation error 231

Press: c to connect client, d to disconnect client, ESC to exit:
Disonnect client

Press: c to connect client, d to disconnect client, ESC to exit:
Disonnect client

Press: c to connect client, d to disconnect client, ESC to exit:
Exit

Done.

In the above output, it's visible that ERROR_PIPE_BUSY only occurs when the client tries to connect twice (there's already a connected client).

For more details, you could take a look at: