I have a Dataflow pipeline that reads from a BigQuery table. However, when reading the data, there is no other option than to read all records with the read(SerializableFunction) or the readTableRows() methods. I was wondering, when using these methods, is there an option to provide a column-like projection (similar to the HBaseIO Scan addColumn() filter).
I know that there is the option of performing a query when loading data from BigQuery (using the readTableRows().fromQuery()), but I'm wondering if there is a similar option to do projection like possible in HBaseIO.
As of right now, fromQuery() is the only option, but this may change in the future.
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments