Writeback issue to GCP dataset

Ask the Community
,
1

Hi have dataset in foundry which i need to sync back to gcp dataset.
So i created new pipeline which takes this dataset as input and created new output dataset with expected schema.
Created new data connection task which then exports this to gcp.
It ran with SNAPSHOT configuration but failing when using REQUIRE_INCREMENTAL

task yaml as below -
type: magritte-bigquery-export-task
config:
table:
dataset: gcp-dataset
table: gcp-table
project: gcp-project
incrementalType: REQUIRE_INCREMENTAL

2

I would recommend to use Virtual Tables as they provide a much better performance and also allow you to have more control by using code. Export tasks are deprecated as well.