Ad

Google BigQuery: Loading Data From A Local CSV File Using The Java API With Custom Field Delimiter

- 1 answer

I am trying to load the rows of a local CSV file into BigQuery, using the Java API.

The CSV file is pipe-delimited.

The official documentation of BigQuery suggests the following solution, implemented with a POST request:

TableId tableId = TableId.of(datasetName, tableName);
WriteChannelConfiguration writeChannelConfiguration =
WriteChannelConfiguration.newBuilder(tableId)
    .setFormatOptions(FormatOptions.csv())
    .build();
TableDataWriteChannel writer = bigquery.writer(writeChannelConfiguration);
// Write data to writer
try (OutputStream stream = Channels.newOutputStream(writer)) {
 Files.copy(csvPath, stream);
}
// Get load job
Job job = writer.getJob();
job = job.waitFor();
LoadStatistics stats = job.getStatistics();
return stats.getOutputRows();

This solution works only if the CSV is comma-delimeted.

I thought that it was possible to specify the delimiter with the class FormatOptions, but this class does not have methods that allow to do this.

Any suggestions?

Ad

Answer

I finally managed to solve my problem by substituting .setFormatOptions(FormatOptions.csv()) with .setFormatOptions(com.google.cloud.bigquery.CsvOptions.newBuilder().setFieldDelimiter("|").build())

Ad
source: stackoverflow.com
Ad