await makes http request return status code 0-CodePudding

Well, I'm building web parsing app and having some troubles making it async.

I have a method which creates async tasks, and decorator for RestSharp so I can do requests via proxy. Basically in code it just does 5 tries of requesting the webpage.

Task returns RestResponse and it's status code is always 0. And this is the problem, because if I do the same synchronously, it works.

private static async Task<HtmlNode> GetTableAsync(int page)
{
    ProxyClient client = new ProxyClient((name) =>ProxyProvider.GetCoreNoCD(name),
    serviceName, 10000, 10000);
    var task = client.TryGetAsync(new Uri(GetPageUrl(page)), (res) =>
    { 
       return res.IsSuccessStatusCode && res.IsSuccessful;
    },5);
            
    HtmlDocument doc = new HtmlDocument();
    doc.LoadHtml((await task).Content);
    return doc.DocumentNode.SelectSingleNode("//div[@class=\"table_block\"]/table");
}

And this works as expected, but synchronously.

private static async Task<HtmlNode> GetTableAsync(int page)
{
    ProxyClient client = new ProxyClient((name) =>ProxyProvider.GetCoreNoCD(name),
    serviceName, 10000, 10000);
    var task = client.TryGetAsync(new Uri(GetPageUrl(page)), (res) =>
            
    { 
        return res.IsSuccessStatusCode && res.IsSuccessful; 
    },5);
    task.Wait();
    HtmlDocument doc = new HtmlDocument();
   
    doc.LoadHtml(task.Result.Content);
    return doc.DocumentNode.SelectSingleNode("//div[@class=\"table_block\"]/table");
}

ProxyClient's insides:

public async Task<RestResponse?> TryGetAsync(Uri uri, 
            Predicate<RestResponse> condition, int tryCount = 15,
            List<KeyValuePair<string, string>> query = null,
            List<KeyValuePair<string, string>> headers = null, 
            Method method = Method.Get, string body = null)
        {
            WebClient? client = null;
            RestResponse? res = null;
            for(int i = 0; i < tryCount; i  )
            {
                try
                {
                    client = new WebClient(source.Invoke(serviceName), serviceName, timeout);
                    res = await client.GetResponseAsync(uri, query, headers, method, body);
                    if (condition(res))
                        return res;
                }
                catch(Exception)
                {
                    ///TODO:add log maybe?
                }
                finally
                {
                    if (client != null)
                    {
                        client.SetCDToProxy(new TimeSpan(cd));
                        client.Dispose();
                    }
                }
            }
            return res;

        }

I have no idea how to make it work with async and don't understand why it doesn't work as expected.

CodePudding user response：

I think it might have to do with the Task.Wait() I would consider changing to await like this.

private static async Task<HtmlNode> GetTableAsync(int page)
{
    ProxyClient client = new ProxyClient((name) =>ProxyProvider.GetCoreNoCD(name),
    serviceName, 10000, 10000);
    var statusOk = false;

    var result = await client.GetAsync(new Uri(GetPageUrl(page));  
    statusOk = result.IsSuccessStatusCode &&
                    result.StatusCode == HttpStatusCode.OK;

    //do what you want based on statusOk
    
    HtmlDocument doc = new HtmlDocument();
   
    doc.LoadHtml(result.Content);
    return doc.DocumentNode.SelectSingleNode("//div[@class=\"table_block\"]/table");
}

CodePudding user response：

Just decided to try different solutions, and seems like it works only if I return task result

Like this:

ProxyClient client = new ProxyClient((name) => ProxyProvider.GetCoreNoCD(name),
                    serviceName, 10000, 10000);
                return await client.TryGetAsync(new Uri(GetPageUrl(page)), (res) =>
                { return res.IsSuccessStatusCode && res.IsSuccessful; });

I thought it could be some kind of misunderstanding of async/await, but seems like no. Maybe some kind of RestSharp bug.