I'm trying build a worker service on Core 5.0. My tree is basically like that =>
1 -) Program.cs 2-) Worker.cs 3-) MyStartUp.cs 4-) Client.cs

In MyStartUp.cs I am getting a list and calling Client class some servers according to list.
In the Client class, I connect to the devices and write the data I read to the database.
Device count nearly 1200, server way is TCP/IP.
What is your best suggestion for write a worker service like that? How can I use threads in it best form?

Below is my first try. This form is working but it's so slow for 1000 different client because there is so much reads in client.

public class Worker : BackgroundService
{
    private readonly ILogger<Worker> _logger;

    public Worker(ILogger<Worker> logger)
    {
        _logger = logger;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        StartUp startUp = new StartUp();
    }
}

public class StartUp
{
    public StartUp()
    {
        //... get client datas and initialize client object
        StartClients();
    }
    public void StartClients()
    {
        foreach (var item in ClientList)
        {
            new Thread(item.Run).Start();
        }
    }
}

public class Client
{
    System.Timers.Timer timer ;
    public Client()
    {
        timer = new Timer();
        timer.Interval = 100;
        timer.Elapsed  = Timer_Elapsed;
        
        //... initialize client connection and database
    }
    public void Run()
    {
        timer.Start();
    }
    private void Timer_Elapsed(object sender, ElapsedEventArgs e)
    {
        //... write values of read client to database
    }
}

CodePudding user response：

Say that you have 1k timers that run every 100ms, and say that each timer tick takes 50ms to execute. That means each timer needs 500ms/s, or 50% of one core, and you would need 500 cores to keep up with the work. You probably do not have that many cores, nor IO to process the requests, and that means the work will start piling up and your computer will more or less freeze since it does not have time to update the UI.

50ms might be an overestimation of the time used, but even at 5ms you would probably have issues unless you are running this on a monster server.

The solution would be to decrease the polling frequency to something more reasonable, say every 100s instead of every 100ms. Or to have one or more threads that polls your devices as fast as they can. For example something like:

private BlockingCollection<MyClient> clients = new ();
private List<Task> workers = new ();
public void StartWorker(){
    workers.Add(Task.Run(Run));
    void Run(){
        foreach(var client in clients.GetConsumingEnumerable()){
           // Do processing
           clients.Add(client); // re-add client to queue
      }
   } 
}
public void CloseAllWorkers(){
    clients.CompleteAdding();
    Task.WhenAll(workers).Wait();
}

I would note that usages of Thread is mostly superseded by tasks. And that creating a thread just to start a System.Timers.Timer is completely useless since the timer will run the tick event on the threadpool, regardless of the thread that started it. At least unless a synchronization object was specified.