I was trying to export data from firestore to google cloud storage using this code snippet

const functions = require('firebase-functions');
const firestore = require('@google-cloud/firestore');
const client = new firestore.v1.FirestoreAdminClient();

const bucket = 'gs://BUCKET_NAME';

exports.scheduledFirestoreExport = functions.pubsub.schedule('every 24 hours').onRun(async() => {

  const projectId = process.env.GCP_PROJECT || process.env.GCLOUD_PROJECT;
  const databaseName = client.databasePath(projectId, '(default)');

  const response = await client.exportDocuments({
        name: databaseName,
        outputUriPrefix: bucket,
        collectionIds: [],
      });
 console.log(`Backup Successful :${response}`, {response});
//here I am trying to import the data to bigquery
});

The problem I am facing is the client.exportDocuments completes a few milliseconds before the time of file creation in Google Cloud Storage Bucket. So when I am trying to access it for importing it says no such file exist.URL is wrong.

Any suggestions on this?

CodePudding user response：

Here's the underlying method databases.export documents.

The response is an Operation which is a potentially long-running process on GCP.

You'll need to poll (I think there's no way to subscribe to) the Operation endpoint until the job succeeds or fails.

If it completes, you could then begin the BigQuery job.

See: Managing Export and Import operations

However, this will likely exceed the timeout on the Cloud Function and should probably not be attempted during a single function invocation.

You may want to consider creating another process that's triggered once the export completes. I've not done this. It's possible that you could create a background function that's triggered by the GCS event. I don't know.