I’ve created a dataset using Workspace. However, after checking the data lineage, I found that there is no connection between the input dataset and the output dataset created by the Jupyter Notebook. Is there a way to schedule and trigger the code created in Workspace with schedule?
Hi Jacob_SE,
Yes! You can register transforms from Jupyter Workspaces - this will register an ipynb file as a reproducible build that will run top-to-bottom. You can register inputs and outputs as part of your transforms spec. For documentation on this you can check out: https://www.palantir.com/docs/foundry/code-workspaces/jupyterlab/#jupyter-notebook-transforms
You are correct that if there is no transform registered between input and output datasets, no link will appear in Data Lineage. Once you register the transform, then a link should appear.
Caleb