I am gathering some data for my supervisor using google cloud platform. Right now we give the instance name and few other keywords.

For example If the Instance is meat. We have different vendors. (It has like 1000s of vendors) I am trying to gather which vendors we used in certain months. Some months we have used the same vendor multiple times. As of now what I do is I gather the data and import it to excel and then I delete the duplicates. I was wondering if there is a way if I can filter/ ignore the duplicates as I'm running the query in the first place?

Thanks for any advice

I’m not sure what key words to use I tried googling but couldn’t actually figure it out

CodePudding user response：

No, not using the Cloud Logging service (generally via gcloud logging or the Console's Logs Explorer).

Cloud Logging is oriented around log entries; log entries are the "records". In order to remove duplicates, post-processing (or more specific filtering¹) is required.

It would be useful to be able to ask the Cloud Logging backend to remove duplicates (as you would not then need to do it) but it doesn't provide this functionality.

The consequence is that you must post-process logs (e.g. remove duplicates) yourself and e.g. using a spreadsheet, is a fine solution.

Alternatively, you can use e.g. Linux tools (e.g. awk, uniq) to remove duplicates for you.

¹ I think this doesn't apply in your case but, one (obvious) way to remove "duplicates" is to provide a more specific query.