Hi Community,
I have the following:
def compute():
transforms = []
entitys_list = var.entity_list
for entity in entitys_list:
def create_transform_function(entity):
@transform(
outputTbl=Output(path + '{entity}'.format(entity=entity_name)),
sourceTbl=Input("ri.foundry.main.dataset.xxxxxxxxxxxxx"),
)
def write_entitys(sourceTbl, outputTbl):
filter the input_data based on entity
data transformation based on entity
return write_entitys
transforms.append(create_transform_function(entity))
return transforms
TRANSFORMS = compute()
this is working with a fixed entity list, but I would like for the entitys_list be based on a value from another dataset.
I understand that due to Continuous integration (CI) checks we cannot have two transforms in this case. But I wonder if there is another way I am not thinking that I might be able to do this even outside repository.