Can I trigger a dataset build from a function in Workshop?

I have a Python transform that I want to trigger by pressing a button in a Workshop.

The hacky solution would be to create a small object, materialize that to a dataset, and trigger the build on that dataset changing … but is there a cleaner solution?

Calling a Webhook from an Action, where the “external system” is Foundry’s API for building datasets, is in principle the right approach here. However, since endpoints for building datasets are not currently exposed via the public API, you would have to use private API endpoints, and this would potentially increase the long-term maintenance burden. Considering that, I would personally use the “hacky” approach that you selected if I were in your position (though it’s worth noting that creating an object and waiting for the materialization to update would be a higher-latency implementation than triggering a build directly, so that’s another aspect to consider).

5 Likes

Another hacky solution could be embedding a hidden Slate App: toggle a workshop variable to a certain value and back on button click, and pass this variable to Slate. And in Slate, on this variable being a certain value, trigger a query to the desired API endpoint.
Or embed a Slate button.

There is however the same maintenance issue that @sandpiper pointed out.

1 Like

Just wanted to flag that we are working on exposing an endpoint to trigger builds via the public API that will be available within the next few months. I’ll follow up here once this endpoint is available in beta.

4 Likes

I am also facing similar issues as above. when will the api endpoints be available like ETA?

In the meantime you can use the builds2 submitBuild API:

response = requests.request(
        "POST",
        "https://<your-stack>.com/build2/api/manager/submitBuild",
        headers={"authorization": "Bearer <token>"}
        json={
            "branch": "master",
            "branchFallbacks": {"branches": []},
            "buildParameters": {},
            "jobSpecSelections": [
                {
                    "type": "datasets",
                    "datasets": {
                        "datasetRids": ["dataset-rid"],
                        "isRequired": True,
                    },
                }
            ],
            "forceBuild": True,
            "inputFailureStrategies": [],
            "ignoreBuildPolicy": False,
        },
    )
1 Like

It should be available within the next few weeks. When it becomes available, you should see it appear on https://www.palantir.com/docs/foundry/api/ under a new “Builds [V2]” section.

1 Like

The API is now available and will be in the public API docs soon. You can also find some docs for this API in the Python SDK repo.

1 Like

thank you very much Smith!

Running into this issue now as well.

Is the best way still to write a function, use that function to call a webhook, and the webhook is configured to hit the public API that triggers a build? This still feels pretty hacky (and not easy to test).

The real issue with that solution is that the user token won’t be propagated unless you configure your Foundry stack as outbound application which means you need to generate another Third Party App.

Another solution is to use a slate iframe and a slate Query (if you have build2 or the new platform APIs available there). This will preserve the user token.

Both solutions feel not natural and calling foundry APIs from workshop with the user token has been always cumbersome… would love for PD to develop a more native way similar to the simplicity of the Slate Queries.