reqwest post request freezes after random amount of time-CodePudding

I started learning rust 2 weeks ago, and has been making this application that watches a log file, and sends a bulk of the information to an elasticsearch DB.

The problem is that after certain amount of time, it freezes (using 100% CPU) and I don't understand why.

I've cut down on a lot of code to try to figure out the issue, but it still keeps freezing on this line according to clion debugger

let _response = reqwest::Client::new()
    .post("http://127.0.0.1/test.php")
    .header("Content-Type", "application/json")
    .body("{\"test\": true}")
    .timeout(Duration::from_secs(30))
    .send() // <-- Exactly here
    .await;

It freezes and doesn't return any error message.

This is the code in context:

use std::{env};
use std::io::{stdout, Write};
use std::path::Path;
use std::time::Duration;
use logwatcher::{LogWatcher, LogWatcherAction};
use serde_json::{json, Value};
use serde_json::Value::Null;
use tokio;

#[tokio::main]
async fn main() {
    let mut log_watcher = LogWatcher::register("/var/log/test.log").unwrap();
    let mut counter = 0;
    let BULK_SIZE = 500;

    log_watcher.watch(&mut move |line: String| { // This triggers each time a new line is appended to /var/log/test.log
        counter  = 1;

        if counter >= BULK_SIZE {
            futures::executor::block_on(async { // This has to be async because log_watcher is not async

                let _response = reqwest::Client::new()
                    .post("http://127.0.0.1/test.php") // <-- This is just for testing, it fails towards the DB too
                    .header("Content-Type", "application/json")
                    .body("{\"test\": true}")
                    .timeout(Duration::from_secs(30))
                    .send() // <-- Freezes here
                    .await;

                if _response.is_ok(){
                    println!("Ok");
                }
            });
            counter = 0;
        }
        LogWatcherAction::None
    });
}

The log file gets about 625 new lines every minute. The crash happends after about ~5500 - ~25000 lines has gone through, or it seems a bit random in general.

I'm suspecting the issue is either something to do with LogWatcher, reqwest, the block_on or the mix of async.

Does anyone have any clue why it randomly freezes?

CodePudding user response：

The problem was indeed because of a mix of async with tokio and block_on, NOT directly reqwest.

The problem was solved when changing main to be non-async, and using tokio as the block_on for async calls instead of futures::executor::block_on.

fn main() {
    let mut log_watcher = LogWatcher::register("/var/log/test.log").unwrap();
    let mut counter = 0;
    let BULK_SIZE = 500;

    log_watcher.watch(&mut move |line: String| {
        counter  = 1;

        if counter >= BULK_SIZE {
            tokio::runtime::Builder::new_multi_thread()
                .enable_all()
                .build()
                .unwrap()
                .block_on(async {

                    let _response = reqwest::Client::new()
                        .post("http://127.0.0.1/test.php")
                        .header("Content-Type", "application/json")
                        .body("{\"test\": true}")
                        .timeout(Duration::from_secs(30))
                        .send()
                        .await;

                    if _response.is_ok(){
                        println!("Ok");
                    }
            });
            counter = 0;
        }
        LogWatcherAction::None
    });
}