I am using Dataflow by connecting MariaDB in AWS' RDS. Recently, our security policy of MariaDB was updated so that only certain ip ranges can access it. And I registered the GCP's Default VPC ip ranges at the AWS RDS security policy. However, it is not accessible from GCP's Dataflow to AWS RDS with Default VPC's ip ranges.

Can specify the IP number of GCP's Dataflow?

The version of apache.beam is '2.25.0'(beam-runners-google-cloud-dataflow-java)

CodePudding user response：

I don't think there is a way for you to explicitly assign ip addresses to Dataflow workers.

Does your default VPC network offer an external IP range so that you can configure it on AWS? If you configure the internal IP range provided by the Google Cloud VPC on AWS, it wouldn't work even if you specify the network for Dataflow and enable the public ips on Dataflow workers.

CodePudding user response：

I solve this.

1. add dataflow param those : '--usePublicIps=false', '--network=something', '--subnetwork=https://www.googleapis.com/compute/v1/projects/rba-distributed-prd/regions/us-central1/subnetworks/[my subnet name in regions]'

2. make cloud nat

3. make vpc private google access (must do this)