Apache Beam - BigQueryIO read Projection

Robin Trietsch :

I have a Dataflow pipeline that reads from a BigQuery table. However, when reading the data, there is no other option than to read all records with the read(SerializableFunction) or the readTableRows() methods. I was wondering, when using these methods, is there an option to provide a column-like projection (similar to the HBaseIO Scan addColumn() filter).

I know that there is the option of performing a query when loading data from BigQuery (using the readTableRows().fromQuery()), but I'm wondering if there is a similar option to do projection like possible in HBaseIO.

jkff :

As of right now, fromQuery() is the only option, but this may change in the future.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related