Hi is there no more auto-submittal to modeling objectives? What is the canonical implementation? just manually add from either the model ui or modling objectives ui?
hello? still have this question
I agree, this would be useful. It would also be useful to allow access to a model objective to access the tagged versions.
I also need this feature. I want to build a CI/CD pipeline that allows automatic retraining of a model, comparison with the current production model, and finally deploy the new model if it is better than the current one.
Right now I can create new models in Model Studio just by scheduling a run through Data Lineage. But couldn’t find a way to automatically deploy it to Modeling Objectives (for instance, through an API).
Just coming back here to say that the modelling objective framework in Palantir Foundry is completely unusable, and i have finally given up (after years of trying to make it work) and am rolling my own solution to manage model deployment.
Hi all, coming from the modeling product team. I really appreciate the feedback on Modeling Objectives, we are aware of its shortcomings, and have been working towards addressing them.
Our strategy for the last little while has been to re-build the siloed, rigid features in Modeling Objectives into the model itself, enabling better integrations throughout the rest of the platform. Over the last little while we have been releasing new features to continue covering that gap:
- Models support running deployments directly on the model vs in modeling objectives
- These will auto upgrade as new versions are produced on a branch
- Models now support writing evaluations directly to the model vs running modeling objectives
- Models can be brought directly into Pipeline builder vs using batch deployments
- This will also auto upgrade as new versions are produced on a branch
We recognize this does not cover the full suite of tooling available in Modeling Objectives (such as the release management tooling, like staging and prod channels) which we are aiming to address through integrations with broader platform tooling like global branching and marketplace. The new modeling evaluations will play directly into global branching as a form of release management, where a merge can be blocked until evaluation scores meet some threshold. The end state we are aiming for is to be able to manage the entire model release management cycle through global branching/marketplace, including upgrading live deployments, functions, and downstream consumers.
These integrations are actively being working on, and we should have something to share within the next few months!