Is there a way I could pull in the input dataset name or path in a transform in Builder. I have a pipeline where I’m unioning a lot of datasets, and I want a column that indicates the source dataset name before I do the union.
Something similar is available in media set transforms where I can get the path. That would be helpful for normal datasets as well.
DISCLAIMER: This is not what you’re looking for nor I understand why it works.
You can ask Pipeline builder AIP integration this prompt:
For reasons I don’t understand that works:
l
and the results is what you’re expecting:
However (there is a big however here), there is NO TRANSFORMATION. When you get into it you see this:
Clearly AIP is accessing a function I don’t have access to (I guess).
This transformation can’t be connected to another dataset, as the result doesn’t change.
The only benefit of this approach is to automatically generate all of the names for your input datasets using AIP.
This does in fact work, but I’m not sure why either. Thank you.
