I am trying to test fetching data from a remote API. I setup the HttpClient as follows:
HttpClient httpClient = SetupHttpClient((HttpRequestMessage request) =>
{
FileStream file = new FileStream("API_Data.json"), FileMode.Open, FileAccess.Read);
StreamReader sr = new StreamReader(file, true);
var response = request.CreateResponse(HttpStatusCode.OK, sr.ReadToEnd());
response.Content.Headers.ContentEncoding.Add("UTF-8");
return Task.FromResult(response);
});
The SetupHttpClient is not relevant here - what matters is the Response that is delivered, which as you can see is created by creating a StreamReader from a FileStream and reading that Stream into the Response.
Using the Text Visualizer I can see that the file has been successfully read into the Response Stream and all special chars such as new lines, tabs and double quotes appear correctly, as per this screenshot:
On the other end, I fetch the content from the HttpResponseMessage as follows:
Stream responseStream = await response.Content.ReadAsStreamAsync();
StreamReader responseReader = null;
if (response.Content.Headers.ContentEncoding.Count > 0)
responseReader = new StreamReader(responseStream, System.Text.Encoding.GetEncoding(response.Content.Headers.ContentEncoding.First()));
else
responseReader = new StreamReader(responseStream, true);
string content = await responseReader.ReadToEndAsync();
return content;
Hover debugging the response again at this point shows that the data is still OK:
The Text Visualizer shows exactly the same as in the 1st screenshot above. Here comes the problem - even though the response content is a string, I can't access the Value property and all the retrieval mechanisms that response.Content offers are via Streams. OK fine, so I get the content via a Stream, however after having been through the Stream, all special chars are now double escaped as you can see here:
This means that I now have to un-escape all these special chars in order to be able to use the returned string as json - if I don't un-escape it then the JsonDeserializer chokes when I try to deserialize it. The StreamReader also adds a (single-escaped) double-quote as the first and last char for good measure.
Googling this all I can find are references to using the correct encoding. As such, I ensured that I saved the source file as UTF-8, that I sent 'UTF-8' as the encoding with the HttpResponseMessage (response.Content.Headers.ContentEncoding.Add("UTF-8");), and that when decoding the Response 'UTF-8' was again used as the encoding (responseReader = new StreamReader(responseStream, System.Text.Encoding.GetEncoding(response.Content.Headers.ContentEncoding.First()));) - as you can see, this has not achieved the desired effect of obtaining a string which is not double-escaped.
I don't want to have to do a 'manual' un-escaping of all special chars when I get the response string from the Stream - this is a terrible hack, however it feels like the only option at the minute - either that or use reflection to get the content of the response.Content.Value property if I detect that response.Content is a String - again another hack that I don't want to do.
How can I ensure that when fetching the response.Content value via a StreamReader that I don't get double-escaped special chars?
EDIT: For clarity, here is the SetupHttpClient method:
public HttpClient SetupHttpClient(Func<HttpRequestMessage, Task<HttpResponseMessage>> response)
{
var configuration = new HttpConfiguration();
var clientHandlerStub = new HttpDelegatingHandlerStub((request, cancellationToken) =>
{
request.SetConfiguration(configuration);
return response(request);
});
HttpClient httpClient = new HttpClient(clientHandlerStub);
mockHttpClientFactory.Setup(_ => _.CreateClient(It.IsAny<string>())).Returns(httpClient);
return httpClient;
}
and the HttpDelegatingHandlerStub
public class HttpDelegatingHandlerStub : DelegatingHandler
{
private readonly Func<HttpRequestMessage, CancellationToken, Task<HttpResponseMessage>> _handlerFunc;
public HttpDelegatingHandlerStub()
{
_handlerFunc = (request, cancellationToken) => Task.FromResult(request.CreateResponse(HttpStatusCode.OK));
}
public HttpDelegatingHandlerStub(Func<HttpRequestMessage, CancellationToken, Task<HttpResponseMessage>> handlerFunc)
{
_handlerFunc = handlerFunc;
}
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
return _handlerFunc(request, cancellationToken);
}
}
EDIT2: A minimal, reproducible example - this requires the following packages - Microsoft.AspNet.WebApi.Core, Microsoft.Extensions.Http, Moq:
using System;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using System.Web;
using System.Web.Http;
using Moq;
namespace StreamReaderEncoding
{
internal class Program
{
static Mock<IHttpClientFactory> mockHttpClientFactory;
static void Main(string[] args)
{
MainAsync().Wait();
}
static async Task MainAsync()
{
mockHttpClientFactory = new Mock<IHttpClientFactory>();
string content = @"{
""test"": ""true""
}";
Console.WriteLine($"content before: {content}");
HttpClient httpClient = SetupHttpClient((HttpRequestMessage request) =>
{
var stream = new MemoryStream();
var writer = new StreamWriter(stream);
writer.Write(content);
writer.Flush();
stream.Position = 0;
StreamReader sr = new StreamReader(stream, true);
var response = request.CreateResponse(HttpStatusCode.OK, sr.ReadToEnd());
response.Content.Headers.ContentEncoding.Add("UTF-8");
return Task.FromResult(response);
});
HttpResponseMessage response = await httpClient.GetAsync("https://www.test.com");
Stream responseStream = await response.Content.ReadAsStreamAsync();
StreamReader responseReader = null;
if (response.Content.Headers.ContentEncoding.Count > 0)
responseReader = new StreamReader(responseStream, System.Text.Encoding.GetEncoding(response.Content.Headers.ContentEncoding.First()));
else
responseReader = new StreamReader(responseStream, true);
content = await responseReader.ReadToEndAsync();
Console.WriteLine($"content after: {content}");
}
static HttpClient SetupHttpClient(Func<HttpRequestMessage, Task<HttpResponseMessage>> response)
{
var configuration = new HttpConfiguration();
var clientHandlerStub = new HttpDelegatingHandlerStub((request, cancellationToken) =>
{
request.SetConfiguration(configuration);
return response(request);
});
HttpClient httpClient = new HttpClient(clientHandlerStub);
mockHttpClientFactory.Setup(_ => _.CreateClient(It.IsAny<string>())).Returns(httpClient);
return httpClient;
}
}
internal class HttpDelegatingHandlerStub : DelegatingHandler
{
private readonly Func<HttpRequestMessage, CancellationToken, Task<HttpResponseMessage>> _handlerFunc;
public HttpDelegatingHandlerStub()
{
_handlerFunc = (request, cancellationToken) => Task.FromResult(request.CreateResponse(HttpStatusCode.OK));
}
public HttpDelegatingHandlerStub(Func<HttpRequestMessage, CancellationToken, Task<HttpResponseMessage>> handlerFunc)
{
_handlerFunc = handlerFunc;
}
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
return _handlerFunc(request, cancellationToken);
}
}
}
Output from the example:
content before: {
"test": "true"
}
content after: "{\r\n \"test\": \"true\"\r\n}"
CodePudding user response：
This has nothing to do with Moq... it's all about HttpRequestMessageExtensions.CreateResponse(), which is taking your string and encoding it as JSON.
Here's a rather more minimal example (as a .NET 6 console app; you can either live with it complaining about restoring a target for net461, or retarget it for .NET 4.7.1 or similar and add some project options and using directives; I think it's simpler to leave it targeting .NET 6.)
using System.Web.Http;
var request = new HttpRequestMessage();
request.SetConfiguration(new HttpConfiguration());
string json = "{ \"test\": \"true\" }";
Console.WriteLine($"Before: {json}");
var response = request.CreateResponse(json);
string text = await response.Content.ReadAsStringAsync();
Console.WriteLine($"After: {text}");
Output:
Before: { "test": "true" }
After: "{ \"test\": \"true\" }"
The thing that I believe is confusing you is that in the debugger you were looking at the response.Content, seeing the Value in the ObjectContent<string> as the string as you wanted it, and assumed that that's the data that's going to be written to the response. It's not. That's the data before it's formatted.
The simplest way of fixing this is to provide the content of the response as a StringContent instead. As that point you don't need any dependencies - the code below is a minimal example which prints the same text for "before" and "after":
var request = new HttpRequestMessage();
string json = "{ \"test\": \"true\" }";
Console.WriteLine($"Before: {json}");
var response = new HttpResponseMessage { Content = new StringContent(json) };
string text = await response.Content.ReadAsStringAsync();
Console.WriteLine($"After: {text}");
Of course you may well want to set some other headers on the response, but I believe this proves that it really is the CreateResponse method (and its use of ObjectContent) that's causing the problem.