Home > other >  Write to bigquery using java cloud function
Write to bigquery using java cloud function

Time:08-31

I want to create a table then write to bigquery using a cloud function, but I dont want to duplicate the data in the tables so I deleted the table first then create the table each time I call the function.

    public static void runCreateTable() {
        Schema schema =
                Schema.of(
                        Field.of("id", StandardSQLTypeName.INT64),
                        Field.of("name", StandardSQLTypeName.STRING));
        createTable(DATASET, TABLE_NAME, schema);
    }
public static void createTable(String datasetName, String tableName, Schema schema) {
        try {
            BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
            TableId tableId = TableId.of(datasetName, tableName);
            TableDefinition tableDefinition = StandardTableDefinition.of(schema);
            TableInfo tableInfo = TableInfo.newBuilder(tableId, tableDefinition).build();
            bigquery.create(tableInfo);
            System.out.println("Table created successfully");
        } catch (BigQueryException e) {
            System.out.println("Table was not created. \n"   e.toString());
        }
    }
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();

            if (bigquery.getTable(TableId.of(DATASET, TABLE_NAME)).delete()) {
                runCreateTable();
                TableRow row = new TableRow();

                for (Map.Entry<String, Object> entry : campaign.entrySet()) {
                    row.set("id", entry.getKey()).set("name", entry.getValue());
                    bigquery.insertAll(InsertAllRequest.newBuilder(bigquery.getTable(TableId.of(DATASET, TABLE_NAME))).addRow(row).build());
                }
            }

So the error is when I deleted the table first the insertAll cannot find the table when it is recreated to write to I got this error: Table abc.abc_names not found

CodePudding user response:

I suggest you avoid recreation of the table instead you can delete its content:

DELETE FROM TABLE_NAME WHERE 1=1

Also, you can check the below answer:

"Not found: Table" for new bigquery table

  • Related