TimeoutException: A timeout occured after 30000ms selecting a server using CompositeServerSelector-CodePudding

I want to deploy an ASP.NET application with MongoDB on Kubernetes locally. Currently the application is working fine with the docker-compose.

In the case of Kubernetes YAML file, I have created a StatefulSet for the MongoDB and a Deployment for the ASP.NET app and also, I have created their respected Services and ConfigMap too. I have attached the complete code below.

Even I can see all the data are loaded into the Mongo database when I use the Mongo-Express Deployment. So, I am sure that the MongoDB StatefulSet is working fine. Now the only concern is the .NET App is throwing an exception called "Resource temporarily unavailable".

About the issue: the build is working fine while performing docker-compose up. But in the case of Kubernetes cluster deployment its throwing this exception:

fail:
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]

An unhandled exception has occurred while executing the request.

System.TimeoutException: A timeout occured after 30000ms selecting a server using CompositeServerSelector{ Selectors = MongoDB.Driver.MongoClient AreSessionsSupportedServerSelector, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 } }. Client view of cluster state is { ClusterId : "1", ConnectionMode : "Automatic", Type : "Unknown", State : "Disconnected", Servers : [{ ServerId: "{ ClusterId : 1, EndPoint : "Unspecified/mongo:27017" }", EndPoint: "Unspecified/mongo:27017", ReasonChanged: "Heartbeat", State: "Disconnected", ServerVersion: , TopologyVersion: , Type: "Unknown", HeartbeatException: "MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server.

System.Net.Internals.SocketExceptionFactory ExtendedSocketException (00000001, 11): Resource temporarily unavailable

at System.Net.Dns.InternalGetHostByName(String hostName)
at System.Net.Dns.ResolveCallback(Object context)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Exception source)
at System.Net.Dns.HostResolutionEndHelper(IAsyncResult asyncResult)
at System.Net.Dns.EndGetHostAddresses(IAsyncResult asyncResult)
at System.Net.Dns.<>c.b__25_1(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at MongoDB.Driver.Core.Connections.TcpStreamFactory.ResolveEndPointsAsync(EndPoint initial)
at MongoDB.Driver.Core.Connections.TcpStreamFactory.CreateStreamAsync(EndPoint endPoint, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Connections.BinaryConnection.OpenHelperAsync(CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at MongoDB.Driver.Core.Connections.BinaryConnection.OpenHelperAsync(CancellationToken cancellationToken)
at MongoDB.Driver.Core.Servers.ServerMonitor.InitializeConnectionAsync(CancellationToken cancellationToken) at MongoDB.Driver.Core.Servers.ServerMonitor.HeartbeatAsync(CancellationToken cancellationToken)", LastHeartbeatTimestamp: "2022-06-26T16:04:05.7393346Z", LastUpdateTimestamp: "2022-06-26T16:04:05.7393356Z" }] }. at MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException(IServerSelector selector, ClusterDescription description) at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChangedHelper.HandleCompletedTask(Task completedTask) at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChanged(IServerSelector selector, ClusterDescription description, Task descriptionChangedTask, TimeSpan timeout, CancellationToken cancellationToken) at MongoDB.Driver.Core.Clusters.Cluster.SelectServer(IServerSelector selector, CancellationToken cancellationToken) at MongoDB.Driver.MongoClient.AreSessionsSupportedAfterServerSelection(CancellationToken cancellationToken) at MongoDB.Driver.MongoClient.AreSessionsSupported(CancellationToken cancellationToken) at MongoDB.Driver.MongoClient.StartImplicitSession(CancellationToken cancellationToken) at MongoDB.Driver.OperationExecutor.StartImplicitSession(CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionImpl1.UsingImplicitSession[TResult](Func2 func, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionImpl1.FindSync[TProjection](FilterDefinition1 filter, FindOptions2 options, CancellationToken cancellationToken) at MongoDB.Driver.FindFluent2.ToCursor(CancellationToken cancellationToken) at MongoDB.Driver.IAsyncCursorSourceExtensions.FirstOrDefault[TDocument](IAsyncCursorSource1 source, CancellationToken cancellationToken) at MongoDB.Driver.IFindFluentExtensions.FirstOrDefault[TDocument,TProjection](IFindFluent2 find, CancellationToken cancellationToken) at HostSimNetwork.Controllers.ConcordController.Index() in /src/HostSimNetwork/Controllers/ConcordController.cs:line 48 at lambda_method(Closure , Object , Object[] ) at Microsoft.Extensions.Internal.ObjectMethodExecutor.Execute(Object target, Object[] parameters) at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync() at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync() --- End of stack trace from previous location where exception was thrown --- at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync() --- End of stack trace from previous location where exception was thrown --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|24_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync() --- End of stack trace from previous location where exception was thrown --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger) at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

YAML FILE:

apiVersion: v1
kind: ConfigMap
metadata:
  name: mongodb-configmap
data:
  database_url: mongodb-service
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mongodb-statefulset
spec:
  serviceName: "mongodb-service"
  selector:
    matchLabels:
      app: mongodb
  template:
    metadata:
      labels:
        app: mongodb
    spec:
      containers:
      - name: mongodb
        image: mongo
        ports:
        - containerPort: 27017
        env:
          - name: MONGO_INITDB_ROOT_USERNAME
            value: "admin"
          - name: MONGO_INITDB_ROOT_PASSWORD
            value: "password"
        volumeMounts:
        - name: mongo-initdb
          mountPath: /docker-entrypoint-initdb.d
      restartPolicy: Always
      volumes:
      - name: mongo-initdb
        configMap:
          name: mongo-initdb

---
apiVersion: v1
kind: Service
metadata:
  name: mongodb-service
spec:
  clusterIP: None
  selector:
    app: mongodb
  ports:
    - port: 27017
      targetPort: 27017
      protocol: TCP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: networksim
  labels:
    app: networksim
spec:
  replicas: 1
  selector:
    matchLabels:
      app: networksim
  template:
    metadata:
      labels:
        app: networksim
    spec:
      containers:
      - name: networksim-web
        image: srideepkar/sim:lotus
        ports:
        - containerPort: 80
        - containerPort: 8085
        env:
        - name: ConnectionStrings__MongoDB
          value: "mongodb://admin:password@mongo:27017"
        - name: MongoDbSettings__User
          value: "admin"
        - name: MongoDbSettings__Password
          value: "password"
        - name: MongoDbSettings__Host
          valueFrom: 
            configMapKeyRef:
              name: mongodb-configmap
              key: database_url
        
---
apiVersion: v1
kind: Service
metadata:
  name: networksim-service
spec:
  selector:
    app: networksim   
  type: LoadBalancer
  ports:
    - port: 5000
      targetPort: 80
      protocol: TCP
      name: port2
    - port: 8085
      targetPort: 8085
      protocol: TCP
      name: port3

AppSetting.JSON FILE

{
  "ConnectionStrings": {
    "MongoDB": "mongodb://mongo:27017",
    "DatabaseName": "NetworkSim",
    "CardtypesCollectionName": "cardtypes"
  },
  "Logging": {
    "LogLevel": {
      "Default": "Warning"
    }
  },
  "AllowedHosts": "*",
  "MongoDbSettings":{
    "Host": "localhost",
    "Port": "27017",
    "User": "admin"
  }
}

Docker-Compose file for the reference:

version: '3.8'
services:
  mongo:
    image: mongo
    restart: always
    ports:
      - 27017:27017
    volumes:
      - ./init-mongo.js:/docker-entrypoint-initdb.d/init-mongo.js:ro
  mongo-express:
    image: mongo-express
    restart: always
    ports:
        - 8081:8081
    depends_on:
        - mongo
  web:
    build: .
    ports:
    - 5000:80
    - 8085:8085

CodePudding user response：

Pretty sure you need the change the name of mongo db connectionstring. Check out the DNS for services section: https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/

In your case it would be mongodb-service.default.svc.cluster.local if you've deployed everything into default namespace.