Can I trigger a dataset build from a function in Workshop?

I have a Python transform that I want to trigger by pressing a button in a Workshop.

The hacky solution would be to create a small object, materialize that to a dataset, and trigger the build on that dataset changing … but is there a cleaner solution?

Calling a Webhook from an Action, where the “external system” is Foundry’s API for building datasets, is in principle the right approach here. However, since endpoints for building datasets are not currently exposed via the public API, you would have to use private API endpoints, and this would potentially increase the long-term maintenance burden. Considering that, I would personally use the “hacky” approach that you selected if I were in your position (though it’s worth noting that creating an object and waiting for the materialization to update would be a higher-latency implementation than triggering a build directly, so that’s another aspect to consider).

5 Likes

Another hacky solution could be embedding a hidden Slate App: toggle a workshop variable to a certain value and back on button click, and pass this variable to Slate. And in Slate, on this variable being a certain value, trigger a query to the desired API endpoint.
Or embed a Slate button.

There is however the same maintenance issue that @sandpiper pointed out.

1 Like

Just wanted to flag that we are working on exposing an endpoint to trigger builds via the public API that will be available within the next few months. I’ll follow up here once this endpoint is available in beta.

5 Likes

I am also facing similar issues as above. when will the api endpoints be available like ETA?

In the meantime you can use the builds2 submitBuild API:

response = requests.request(
        "POST",
        "https://<your-stack>.com/build2/api/manager/submitBuild",
        headers={"authorization": "Bearer <token>"}
        json={
            "branch": "master",
            "branchFallbacks": {"branches": []},
            "buildParameters": {},
            "jobSpecSelections": [
                {
                    "type": "datasets",
                    "datasets": {
                        "datasetRids": ["dataset-rid"],
                        "isRequired": True,
                    },
                }
            ],
            "forceBuild": True,
            "inputFailureStrategies": [],
            "ignoreBuildPolicy": False,
        },
    )
1 Like

It should be available within the next few weeks. When it becomes available, you should see it appear on https://www.palantir.com/docs/foundry/api/ under a new “Builds [V2]” section.

1 Like

The API is now available and will be in the public API docs soon. You can also find some docs for this API in the Python SDK repo.

1 Like

thank you very much Smith!

Running into this issue now as well.

Is the best way still to write a function, use that function to call a webhook, and the webhook is configured to hit the public API that triggers a build? This still feels pretty hacky (and not easy to test).

The real issue with that solution is that the user token won’t be propagated unless you configure your Foundry stack as outbound application which means you need to generate another Third Party App.

Another solution is to use a slate iframe and a slate Query (if you have build2 or the new platform APIs available there). This will preserve the user token.

Both solutions feel not natural and calling foundry APIs from workshop with the user token has been always cumbersome… would love for PD to develop a more native way similar to the simplicity of the Slate Queries.

1 Like

We have this trouble at the moment, any news that you have seen? My developer assist told me that I could build a python function and create an action.. and I told it, no. I cannot see anything but ontology_edit functions. (still created the function anyway to just see if it was correct) It gave me this answer, has anyone heard of this feature? It seems to say there is a way to see more functions (that in my case triggers the build if I could just.. well. trigger the function :slight_smile: )

Based on your description, it sounds like the Ontology Manager UI is currently filtering only for “ontology edit functions” in the Action rules dialog, and you are unable to change this filter to see your published Python function UDF.
Possible causes and next steps:

  1. Feature Availability:

    • In some Foundry stacks, the ability to use Python function UDFs as Ontology Actions may be restricted by feature flags or permissions. If you only see ontology edit functions and cannot change the filter, it is possible that UDF Actions are not enabled for your stack or ontology.
  2. If this is the case, you will need to contact your Foundry administrator to request that “Function UDFs” or “User-defined Actions” be enabled for your environment. This is a common prerequisite for using Python functions in Actions 1 “

Triggering a dataset build should be possible today:

  1. Create a third party application with the confidential client workflow
    1. Include only the Foundry Platform SDK option
    2. Grant the resulting service user edit access on your dataset(s)
  2. Create a a Python serverless function that calls the Platform SDK at the endpoint below
    1. Use the credentials from (1)
  3. Wrap this function inside in Action
  4. Call from Workshop

https://github.com/palantir/foundry-platform-python/blob/develop/docs/v2/Orchestration/Build.md#create

1 Like

So the action mentioned in step #3 uses the ontology edit function rule? because thats the only kind of functions that are available in the action rules dropdown?

1 Like

The rule type isn’t important since in this setup the Action is a wrapper for the function. The “Ontology Edit” type could be a reasonable placeholder (even though no edits are being performed).

I tried this, I created a function that uses a third party application for authentication. But I cant use external OAuth from within the runtime environment of Foundry.

Sounds like a potential config issue? Let me know if you get your use case working.

There’s a community AIP example that describes some detail about creating an OAuth client for this SDK, which may be relevant here:
https://github.com/palantir/aip-community-registry/tree/develop/Platform%20Governance%20App%20with%20Platform%20SDK