I am trying to modify this example as follows:
@BigQueryType.fromQuery("SELECT weight_pounds FROM [bigquery-public-data:samples.natality]")
class Row
@BigQueryType.toTable
case class Result(weight_pounds: Double)
sc.typedBigQuery[Row]()
.map(r => r.weight_pounds.getOrElse(0.0)) // return 0 if weight_pounds is None
.top(100)
.map(x => Result(x))
// Convert elements from Result to TableRow and save output to BigQuery.
.saveAsTypedBigQueryTable(
Table.Spec(args("output")),
writeDisposition = WRITE_TRUNCATE,
createDisposition = CREATE_IF_NEEDED
)
Above gives me following error:
[error] /Users/me/scio/scio-examples/src/main/scala/com/spotify/scio/examples/foo.scala:66:24: type mismatch;
[error] found : Iterable[Double]
[error] required: Double
[error] .map(x => Result(x))
[error] ^
How can I fix this?
CodePudding user response:
The trick is to add a flatMap(x => x)
right after top(100)