I am reading from a google sheet of 100,000 records but I want to load only the records after a certain date. ( so applying filter ) . But I haven't been able to accomplish that as I do not know how to access the column name without using foreach.

Here is what the data looks like from the googlesheet

So basically, I would like to filter the records something like this.

Timestamp ge '10/13/2021' so it will only return records for 10/13/2021 and 10/14/2021... etc.

Is this possible to do that? If not, what is the best recommended way to approach this issue as i just wanted to load daily records to sql db in the next step.

CodePudding user response：

The Get rows action of the Google Sheets connector doesn't support filtering. You can only specify how many rows you want returned, and how many rows you want to skip. E.g. if you have 100,000 rows in your sheet, you can easily get the rows between 90,001 and 90,200, should you wish to do so.

While you can't use this connector to retrieve filtered data from Google Sheets, you can use the Filter array action to filter the retrieved data as you wish.

You might still need to use the For each loop to retrieve and filter data in chunks.