Hi,
i use lightweight for executing model on pandas. i use semi supervised clustering model, and i need to access the previous version of the output, in order to inject it to the input.
while its working good when i work in regular spark mode, was not able to find a way to access previous on lightweight.
Any suggestions?
Thanks
You should be able to access previous in a similar way to non lightweight transform by using the incremental decorator
https://www.palantir.com/docs/foundry/transforms-python/lightweight-examples#incremental-workflows