Hello,
I have written a Python Transform within Code repo and successfully built it. But is it possible to add this Python Transform to Pipeline Builder?
I dont see any option of Custom written transform such as mine except already built-in ones.
Please help.
Hello,
Did you check the function options below:
I assume you are building a transformation script and not a dataset-building script, or else you would import the code repository’s result into the pipeline builder directly.
Let me know how this goes.
Did you ever find a solution to your issues? I have also created a custom Python transform (using MediaSets as inputs/outputs), built it, registered it via Pipeline(), but still it doesn’t show up in the Pipeline Builder as custom function.
It runs in the Code Repo and does its job given a fixed input/output set, but I was hoping to use it in Pipeline Builder as a custom transformation step where I can assign my own input and output.
No, it turns out Pipeline builder only accepts Python Functions not the Transform. I had to ditch the idea of using the Pipeline builder altogether. Maybe you can adapt yours to Python Functions and integrate it o Pipeline Builder. In my case, I had lots of other stuff going on in my Transforms, doing it on Python Functions was not feasible.
sorry for the tangent, but by any chance do you have an example/repo for your mediaset input/output? I’m having trouble with doing that myself despite following the docs…also my post is pending
Thanks for your reply!
Indeed, that is what I also figured out in the meantime. Python Transforms run in standalone mode and you need to hardcode source and destination in the code. The Build function, which I thought was building the code and exposing it to Pipeline Builder, actually “building” (aka running) the Python pipeline to generate the results of the transformation.
I have to say that the embedded support chatbot was very misleading in this case. It kept hinting to the fact I could expose the transform as a custom function to Pipeline Builder, but this is not the case. You need a UDF for that, and also in that case it doesn’t become a node but simply one of the Transforms you can use.
Also, I couldn’t create a Python UDF taking Mediasets as input/output so far. Need to go deeper into it.
What do you mean with a mediaset example? I was processing documents (risk reviews in PDF format) and I wanted to extract single pages then process them in a custom way. Eventually the Transform worked but I couldn’t expose that functionality to Pipeline Builder to generalize it and use it also with different Mediasets. Still need to figure how to create a UDF function which is visible in Pipeline Builder and takes Mediasets as input/output.
ok thank you for the response, i also was having trouble with mediaset output so i guess we are in the same boat
