I have a pipeline that uses a history table and a current table as inputs. In the pipeline, I want to compare the records in the current table to the records in the history table. If certain records repeat in the current table, I set a repeat flag to indicate the records then repeat. I want to append the current table to the history table. When I try to do this using Output dataset write mode of append only new records, I get an error that says: Some output datasets already exist. The output dataset should be unique in the output folder".
How can I fix this error?