When reading data from a .NET Core 3.0 API is much harder than it should be…
Towards the end of last year I fired up a new ASP.NET Core 3.0 project for a client of mine as my first foray into building a real full .NET Core business application from end to end rather than simply producing prototypes and making changes to v2.2 projects.
This particular piece of work consists of a .NET Core MVC Website and Web API, and so as part of this it will need to serialise/deserialise data to and from the API side of things.
Whereas traditionally with the .NET Framework and older versions of .NET Core we’d reach for our favourite 3rd party serialisation library, with version 3 we no longer have to, as more and more functionality is being baked into the product. With the introduction of ASP.NET Core 3.0 the default JSON serialiser has been changed from Newtonsoft.Json to the native System.Text.Json serialiser. It therefore made sense for to me to employ that instead of installing JSON.NET, especially as the native API reportedly outperforms in most circumstances.
This sounded like a great idea and I set about building out the comms code between the API and the client. However I immediately encountered a problem with a simple call to one of the API endpoints from my Web site:
The data was being sent back from the API but my model class was empty when deserialised, and since my web page was expecting the model to be populated, the site was erroring. This was very strange since my Models were identical on both the API and MVC Web site ends.
The MVC web site consistently failed to deserialise the API response using a model that was identical on both the API side and the Web app side.
The problem code
Lets demonstrate the issue I had by looking at some code that exhibits the problem:
The Model POCO class (identical in both API and MVC projects):
public class Client()
{
public Guid ClientId { get; set; }
public string ClientName { get; set; }
public string ClientAddress { get; set; }
}API code:
public async Task<ActionResult> GetClient(Guid clientId)
{
var client = await _repository.GetClient(clientId);
return Ok(client);
}MVC code:
// Call API to get Client info
var response = await _httpClient.GetAsync($"Clients/{id}"))
using var responseStream = await response.Content.ReadAsStreamAsync();
var client = await JsonSerializer.DeserializeAsync<Client>(responseStream);In the example above the model will fail to deserialise correctly and all properties will simply be set to null:
This was puzzling until I took a closer look at the serialised JSON…
{"clientId":"ea4e2b39-fb3f-4d7a-9329-00fe658e9dca","clientName":"Sherlock Holmes","clientAddress":"221B Baker Street"}It seems the Web API was returning JSON in Camel case format but the the MVC Web application was attempting to deserialise it in Pascal case (or more accurately the same case as my model). This struck me as odd since I had set no explicit configuration of the serialiser in either project so it should have been using the default settings.
If you’re using Pascal case models, out of the box in .NET Core 3, your Web API will return your model as Camel case JSON, and the JsonSerializer will fail to deserialise it into an instance of the exact same model.
Why is this happening?
Lets look at the problem in detail, firstly from the (API) serialisation end as it returns data and secondly from the (MVC) Client end as it deserialises the API response:
1) API serialisation with Camel case settings
When setting up a new Web API project in .NET Core we typically set up our services in the Startup.cs class and call services.AddControllers(). Amongst other things this sets up the FormatterMappings needed to return data from controllers to their consumers including SystemTextJsonOutputFormatter, which is needed to return JSON string data.
In our earlier example, when we return the Ok() ActionResult from the API Controller Action Method, the SystemTextJsonOutputFormatter is employed to call the JsonSerializer and write out the data to the response. By default in MVC the JsonSerializer uses the Microsoft.AspNetCore.Mvc.JsonOptions class for deciding how to serialise the data back to the client.
The default JsonSerializerOptions in ASP.NET Core MVC 3.0 are set to the following:
PropertyNameCaseInsensitive = true; PropertyNamingPolicy = JsonNamingPolicy.CamelCase;
Hence when our Controller endpoint returns data, we receive camel case JSON data back from the API.
2. Client deserialisation with case sensitive settings
Now when it comes to deserialising a response from a Web API in MVC, we might typically use an instance of the HttpClient class to make an HTTP GET call to an API endpoint. We would then call the new JsonSerializer.Deserialize() method to convert the response into an instance of our strongly typed Model.
However in .NET Core 3.0, when calling the JsonSerializer manually, by default it is initialised with case sensitive property deserialisation. You can see this for yourself by delving into the System.Text.Json source code. The result of this means that any JSON in the API response has to exactly match the casing of your target model, which is typically Pascal case, or your properties will not be deserialised.
How do we resolve this?
We have a few potential options to tackle this:
1. Use custom JsonSerializerOptions when deserialising
We can pass in our own JsonSerializerOptions to the JsonSerialiser in the MVC client whenever we carry out deserialisation, to force the routine to recognise Camel case:
var options = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase };
var json = "{\"firstName\":\"John\",\"lastName\":\"Smith\"}";
var user = JsonSerializer.Deserialize<User>(json, options);Alternatively in a similar manner, we can tell the JsonSerialiser that instead of a specific property naming policy, we just want to ignore case completely by passing in options with PropertyNameCaseInsensitive = true:
var options = new JsonSerializerOptions { PropertyNameCaseInsensitive = true };
var json = "{\"firstName\":\"John\",\"lastName\":\"Smith\"}";
var user = JsonSerializer.Deserialize<User>(json, options); This approach is obviously a little tedious and error prone as it means you end up having to remember to specify your custom options every time you wish to wish to deserialise some JSON. Not ideal.
You might be thinking “How can I globally set the default options for the JsonSerializer?“
It would be nice if we could set our preferred approach globally for the MVC consumer project. You’d think we could just do something like this in our Startup.cs class ConfigureServices() method:
services.AddControllers()
.AddJsonOptions(options => {
options.JsonSerializerOptions.PropertyNamingPolicy = JsonNamingPolicy.CamelCase)
});
OR
services.AddControllers()
.AddJsonOptions(options => {
options.JsonSerializerOptions.PropertyNameCaseInsensitive = true
});But unfortunately not – this code only configures JsonSerializer options for MVC and Web API controllers. This will not configure JSON settings application wide or for any of your own code that calls the JsonSerializer methods.
Since System.Text.Json.JsonSerializer is a static class it carries no state and therefore there is no way to set this on a global project basis (unless you want to get dirty and use reflection, which I don’t :P)
2. Set attributes on your model classes
You can decorate you model properties on the client with the JsonPropertyNameAttribute. e.g. :
[JsonPropertyName("clientName")]
public string ClientName { get; set; } However rather you than me if you have a large number of Model classes.
3. Change the casing of your models from Pascal case to Camel case
Just no.
4. Use reflection
Double no.
But if you really have to, this code should sort you out.
5. Employ an extension method to deserialize the JSON with your custom settings
Using an extension method to deserialize the JSON with your custom settings is an option but again, this has the problem of having to enforce usage of the extension method which on larger projects with many developers might prove tricky.
6. Abstract the (de)serialiser behind an interface
We could wrap the JsonSerialiser behind an interface and ask the IoC container for an IJsonSerialiser. Our concrete representation could then set the appropriate JsonSerializerOptions and deserialise the data correctly.
This is not a bad option and it does mean we could more easily change the serialiser should we want to swap out System.Text.Json in future without breaking the consumers. Again this is all good as long as all developers are aware of the interface and don’t go off piste deserialising via JsonSerialiser directly.
7. Replace the Web API defaults for the OutputFormatter
We could replace the default options in the Web API. To do this add the following to your WebAPI Startup class in the ConfigureServices method:
services.AddControllers() .AddJsonOptions(o => o.JsonSerializerOptions.PropertyNamingPolicy = null);
This will override the defaults and set them back to the .Net Core default of serialising out the JSON with the same casing as your model. This is by far the easiest option as long as you don’t have a requirement for Camel case JSON to be returned from your API.
8. Sledgehammer approach
Go back to Newtonsoft’s JSON.NET
Surely there’s got to be a better way?
There has been some robust discussion around the subject here and whether the ability to set the JsonSerialiser defaults on an application wide basis should be part of .NET Core going forward. I would personally like to see this feature but as with everything it has trade-offs with performance that Microsoft are keen to stay on top of with their flagship language.
Notwithstanding the performance question, the ability to set the options globally for the JsonSerialiser has reportedly been added to the .NET 5.0 roadmap which is expected to be released in November 2020 but I’m yet to see any details on how this change might be implemented .
I hope this goes some way to clearing up why things can go wrong with the new JsonSerializer in .NET Core 3.0. This is a small issue but if this helps one person avoid the time I wasted on it, then it was worth posting ?
Useful reading
- Microsoft has comprehensive documentation on the new JsonSerializer and it’s configuration options here.
- What Those Benchmarks Of System.Text.Json Don’t Mention