Home > database >  GroupBy using date in EF Core
GroupBy using date in EF Core

Time:11-17

I am working on an application where I am getting JSON results set from third-party API. I have deserialized it into my custom view model.

There is a list of records that contains records with repeating records I want to get the list with the average user on that date.

Here is my code and JSON. My point is to use the dt_txt date column in a single record and get the average temperature, wind speed, and humidity. It counts 2 from the below JSON to get the average of the above mentions fields.

public async Task<WeatherViewModel?> GetForecast(string param)
{
    var response = JsonConvert.DeserializeObject<WeatherReponseModel>(await httpClientApiService.GetRequestAsync(param, string.Format(WeatherEndPointsUrls.forecast, param)));

    if (response?.list == null)
        return null;

    // Here I want to rearrange the records. 
    // var orderList = response.list.GroupBy(x => Convert.ToDateTime(x.dt_txt)).ToList();
    
    return new WeatherViewModel();
}

Resulting JSON:

"message":0,
   "cnt":40,
   "list":[
      {
         "dt":1668632400,
         "main":{
            "temp":5.98,
            "feels_like":2.53,
            "temp_min":5.23,
            "temp_max":5.98,
            "pressure":1000,
            "sea_level":1000,
            "grnd_level":1002,
            "humidity":81,
            "temp_kf":0.75
         },
         "weather":[
            {
               "id":803,
               "main":"Clouds",
               "description":"broken clouds",
               "icon":"04n"
            }
         ],
         "clouds":{
            "all":83
         },
         "wind":{
            "speed":5.06,
            "deg":91,
            "gust":10.53
         },
         "visibility":10000,
         "pop":0.29,
         "sys":{
            "pod":"n"
         },
         "dt_txt":"2022-11-16 21:00:00"
      },
      {
         "dt":1668643200,
         "main":{
            "temp":4.69,
            "feels_like":0.63,
            "temp_min":3.86,
            "temp_max":4.69,
            "pressure":1003,
            "sea_level":1003,
            "grnd_level":1002,
            "humidity":78,
            "temp_kf":0.83
         },
         "weather":[
            {
               "id":803,
               "main":"Clouds",
               "description":"broken clouds",
               "icon":"04n"
            }
         ],
         "clouds":{
            "all":84
         },
         "wind":{
            "speed":5.69,
            "deg":94,
            "gust":12.3
         },
         "visibility":10000,
         "pop":0.27,
         "sys":{
            "pod":"n"
         },
         "dt_txt":"2022-11-17 00:00:00"
      },
      {
         "dt":1668654000,
         "main":{
            "temp":3.12,
            "feels_like":-1.59,
            "temp_min":3.12,
            "temp_max":3.12,
            "pressure":1006,
            "sea_level":1006,
            "grnd_level":1001,
            "humidity":73,
            "temp_kf":0
         },
         "weather":[
            {
               "id":804,
               "main":"Clouds",
               "description":"overcast clouds",
               "icon":"04n"
            }
         ],
         "clouds":{
            "all":97
         },
         "wind":{
            "speed":6.22,
            "deg":93,
            "gust":12.73
         },
         "visibility":10000,
         "pop":0,
         "sys":{
            "pod":"n"
         },
         "dt_txt":"2022-11-17 03:00:00"
      },
      {
         "dt":1668664800,
         "main":{
            "temp":3.11,
            "feels_like":-1.81,
            "temp_min":3.11,
            "temp_max":3.11,
            "pressure":1006,
            "sea_level":1006,
            "grnd_level":1000,
            "humidity":74,
            "temp_kf":0
         },
         "weather":[
            {
               "id":804,
               "main":"Clouds",
               "description":"overcast clouds",
               "icon":"04n"
            }
         ],
         "clouds":{
            "all":89
         },
         "wind":{
            "speed":6.72,
            "deg":100,
            "gust":12.45
         },
         "visibility":10000,
         "pop":0,
         "sys":{
            "pod":"n"
         },
         "dt_txt":"2022-11-17 06:00:00"
      },
      {
         "dt":1668675600,
         "main":{
            "temp":5.35,
            "feels_like":0.76,
            "temp_min":5.35,
            "temp_max":5.35,
            "pressure":1006,
            "sea_level":1006,
            "grnd_level":1000,
            "humidity":59,
            "temp_kf":0
         },
         "weather":[
            {
               "id":804,
               "main":"Clouds",
               "description":"overcast clouds",
               "icon":"04d"
            }
         ],
         "clouds":{
            "all":100
         },
         "wind":{
            "speed":7.59,
            "deg":111,
            "gust":12.21
         },
         "visibility":10000,
         "pop":0,
         "sys":{
            "pod":"d"
         },
         "dt_txt":"2022-11-17 09:00:00"
      },
      {
         "dt":1668686400,
         "main":{
            "temp":5.65,
            "feels_like":1.15,
            "temp_min":5.65,
            "temp_max":5.65,
            "pressure":1004,
            "sea_level":1004,
            "grnd_level":999,
            "humidity":66,
            "temp_kf":0
         },
         "weather":[
            {
               "id":804,
               "main":"Clouds",
               "description":"overcast clouds",
               "icon":"04d"
            }
         ],
         "clouds":{
            "all":100
         },
         "wind":{
            "speed":7.6,
            "deg":113,
            "gust":13.15
         },
         "visibility":10000,
         "pop":0,
         "sys":{
            "pod":"d"
         },
         "dt_txt":"2022-11-17 12:00:00"
      }
   ],
   "city":{
      "id":2950159,
      "name":"Berlin",
      "coord":{
         "lat":52.5244,
         "lon":13.4105
      },
      "country":"DE",
      "population":1000000,
      "timezone":3600,
      "sunrise":1668580182,
      "sunset":1668611564
   }
}

Model classes:

public class WeatherReponseModel
{
        public string? cod { get; set; }
        public dynamic? message { get; set; }
        public int cnt { get; set; }
        public List<ListViewModel>? list { get; set; }
        public CityViewModel? city { get; set; }
}

public class Main
{
        public double temp { get; set; }
        public double feels_like { get; set; }
        public double temp_min { get; set; }
        public double temp_max { get; set; }
        public int pressure { get; set; }
        public int sea_level { get; set; }
        public int grnd_level { get; set; }
        public int humidity { get; set; }
        public double temp_kf { get; set; }
}

public class CityViewModel
{
        public int id { get; set; }
        public string? name { get; set; }
        public Coord? coord { get; set; }
        public string? country { get; set; }
        public int population { get; set; }
        public int timezone { get; set; }
        public int sunrise { get; set; }
        public int sunset { get; set; }
}

public class Clouds
{
        public int all { get; set; }
}

public class Coord
{
        public double lat { get; set; }
        public double lon { get; set; }
}

public class ListViewModel
{
        public ListViewModel()
        {
            this.weather = new List<Weather>();
        }

        public int dt { get; set; }
        public Main main { get; set; }
        public List<Weather> weather { get; set; }
        public Clouds clouds { get; set; }
        public Wind wind { get; set; }
        public int visibility { get; set; }
        public double pop { get; set; }
        public Sys sys { get; set; }
        public DateTime dt_txt { get; set; }
        public Rain rain { get; set; }
        public Snow snow { get; set; }
}

public class Rain
{
        [JsonProperty("3h")]
        public double _3h { get; set; }
}

public class Root
{
}

public class Snow
{
        [JsonProperty("3h")]
        public double _3h { get; set; }
}

public class Sys
{
        public string? pod { get; set; }
}

public class Weather
{
        public int id { get; set; }
        public string? main { get; set; }
        public string? description { get; set; }
        public string? icon { get; set; }
}

public class Wind
{
        public double speed { get; set; }
        public int deg { get; set; }
        public double gust { get; set; }
}

CodePudding user response:

Here's a pretty straightforward answer, and I'm going to explain it below:

var result = response.list
    .GroupBy(listViewModelElement => DateOnly.FromDateTime(new DateTime(listViewModelElement.dt_txt)))
    .Select(dateGroup => new {
            dt = dateGroup.Key,
            TemperatureAverage = dateGroup.Average(x => x.main.temp),
            // not sure how "Wind" class looks, so replace "SPEED" with whatever the correct prop name is
            WindSpeedAverage = dateGroup.Average(x => x.wind.SPEED),
            HumidityAverage = dateGroup.Average(x => x.main.humidity),
    });

Note: not sure about the new DateTime(listViewModelElement.dt_txt) syntax, maybe you need to massage the string for it to work.

Note 2: Be aware of potential performance issues with this simplified approach. If your input (number of elements in response.list) is very large it's not the most optimal solution. If performance becomes a real concern more manual stuff has to be added instead of relying on LINQ.

So what does this code do?

First, as the question states, we're grouping the contents of the respose.list by DateTime constructed out of dt_txt property value, with time completely stripped. That's the .GroupBy(listViewModelElement => DateOnly.FromDateTime(new DateTime(listViewModelElement.dt_txt))) piece. DateOnly.FromDateTime does the "remove the time bit" part.

This GroupBy creates a "group" of sorts. Think about it as key-value collection, one-to-many. In this case a "key" for each group is the DateTime value, and the "value" for each group is a collection (IEnumerable) of ListViewModel's.

After that, for each resulting "group" (meaning for each distinct date in all dt_txt values here) we create a new anonymous object out of the results. That's the piece inside the Select lambda:

new {
            dt_datetime = dateGroup.Key,
            TemperatureAverage = dateGroup.Average(x => x.main.temp),
            WindSpeedAverage = dateGroup.Average(x => x.wind.SPEED),
            HumidityAverage = dateGroup.Average(x => x.main.humidity),
    }

You can see that dt_datetime is set to the Key prop value (which is present on the "grouping" created by the GroupBy), and other values are calculated from dateGroup values.

Final note: if you plan to reuse the values across your code alot I'd suggest creating a proper class to hold the results. Something like that:

class WeatherAveragesResult {
    public dt_datetime DateTime;
    public TemperatureAverage float;
    public WindSpeedAverage float;
    public HumidityAverage float;

    public WeatherAveragesResult(datdt_datetime DateTime, tempAvg float, windSpeedAvg float, humidityAvg float) { this.dt = date; ... }
}

And maybe creating a public method on ListViewModel class to remove the boilerplate DateOnly.FromDateTime(new DateTime(listViewModelElement.dt_txt)):

public class ListViewModel {
   // ...

   public DateTime GetDateValue() {
       return DateOnly.FromDateTime(new DateTime(this.dt_txt));
   }
   
   // ...
}

And do

var result = response.list
    .GroupBy(listViewModelElement => listViewModelElement.GetDateValue())
    .Select(dateGroup => new WeatherAveragesResult (
        dateGroup.Key,
        dateGroup.Average(x => x.main.temp),
        dateGroup.Average(x => x.wind.SPEED),
        dateGroup.Average(x => x.main.humidity))
    );
  • Related