Compile .proto files on every directory-CodePudding

I have a directory structure like this:

Makefile
README.md
go.mod
google/
  -> Protobuf dependencies (ignore)
gopb/
  -> Compiled Go protobuf files (ignore)
protoc-gen-openapiv2/
  -> Protobuf OpenAPI depdencies (ignore)
providers/
  - polygon/
    - tickers.proto
    - query.proto

I'd like to setup a single command that can find all directories with .proto files (except the ones indicated above) and run protoc on each of these, with output going to a directory of the same name under the gopb directory. So far, I have been able to create a find command that returns the directory names:

$ find -not \( -path "*.git*" -or -path "*google*" -or -path "*protoc*" -or -path . -or -path "*gopb*" \) -type d -print

./providers
./providers/polygon

Obviously, this is not entirely accurate and I'd like to fix that, but the main issue is that, when I use the -exec flag to call protoc:

$ find -not \( -path "*.git*" -or -path "*google*" -or -path "*protoc*" -or -path . -or -path "*gopb*" \) -type d \
> -exec protoc --go_out=./gopb/ --go_opt=paths=source_relative {}/*.proto \;

This prints two errors:

Could not make proto path relative: ./providers/*.proto: No such file or directory
Could not make proto path relative: ./providers/polygon/*.proto: No such file or directory

I'm not sure what I'm doing wrong here because, if I call protoc --go_out=./gopb/ --go_opt=paths=source_relative ./providers/polygon/*.proto, it creates the files as expected. What am I doing wrong here?

CodePudding user response：

There are various ways to tackle the problem, but for *.proto to be treated as a glob you need to interpret it in a shell.

Here's how you could do it with find and bash:

find . -not \( ...paths... \) \
       -type d \
       -exec bash -c '
           shopt -s nullglob
           for d
           do
               f=( "$d"/*.proto )
               (( ${#f[@]} > 0 )) || continue
               protoc --go_out=./gopb/ --go_opt=paths=source_relative "${f[@]}"
           done
' _ {}

CodePudding user response：

With help from @Fravadona, I realized that the method I was using to compile my .proto files wasn't going to work. So, I moved all my protobuf files that I wanted to compile into a new directory called protos, which modified my directory structure to:

Makefile
README.md
go.mod
google/
  -> Protobuf dependencies (ignore)
gopb/
  -> Compiled Go protobuf files (ignore)
protoc-gen-openapiv2/
  -> Protobuf OpenAPI depdencies (ignore)
protos/
  - providers/
    - polygon/
      - tickers.proto
      - query.proto

and then I modified the command to be this:

find ./protos -type f -name "*.proto" -exec protoc --go_out=../../../ {} \;

This new command finds all .proto files in the /protos directory and calls protoc on each. I removed the source-relative option which also required some code changes in my imports as well. But, this works for my needs. Note I had to use ../../../ for my output directory because my protobuf repo exists inside of github.com/myrepo/protobuf and I want the files written to github.com/myrepo/protobuf/gopb.