Using foundry-dev-tools in Python Functions to create Objects on demand via SQL Queries

Ask the Community
, ,
1

Hello dear community!

  • I’m trying to use the foundry-dev-tools to:
    • Retrieve data from SQL queries
    • Create objects on demand via the OSDK
  • It works exactly as expected in Live Preview but fails to create the FoundryContext in the Published version.
  • Has anyone tried something similar and was successful?
  • Are there other options, such as:
    • Using direct access to the Foundry API to run the queries using client = source.get_https_connection().get_client() to make client requests with the requests library?
    • Get the context via function input (DISCLAIMER: It’s a Python function, not a Python transform) by any means?

This is my code snippet:

@function(edits=[myObjectType], sources=['FoundryAPI'])
def retrieve_dnb_data(search_items: str) -> list[OntologyEdit]:

source = get_source("FoundryAPI")
JWT = source.get_secret('additionalSecretJWT')
ctx = FoundryContext(token_provider=JWTTokenProvider(host='myinstance.palantirfoundry.com', jwt=JWT))
ontology_edits = FoundryClient().ontology.edits()

query = f"SELECT * FROM `ri.foundry.main.dataset.941c7165-5cc2-40a5-9821-3b071fdf8328` WHERE item_id IN ({search_items})"
 df = ctx.foundry_sql_server.query_foundry_sql(query, branch="master")

Thank you all for your attention!

2

What is the error message / stacktrace you are receiving? Did you deploy the function (I think egress only works in deployed functions)?

Here is some working code I tested a while ago:

from functions.api import function
from functions.sources import get_source
from foundry_dev_tools import FoundryContext, JWTTokenProvider
import json


@function(sources=["FoundryHostname"])
def user_info() -> str:
    source = get_source("FoundryHostname")

    fdt_context = FoundryContext(
        token_provider=JWTTokenProvider(
            host="stack.palantirfoundry.com",
            jwt=source.get_secret("additionalSecretTOKEN"),
        )
    )
    user_info = json.dumps(fdt_context.multipass.get_user_info())

    return user_info

On the actual workflow it looks a little bit like an anti pattern: why would you want to create objects dynamically and not use a pipeline to perform the sql query and use the resulting dataset as backing dataset for the object type?

3

Hey Nico!

Thank you for your answer!

I’m having https connectivity errors. To be able to use sources, I need to be able to pass the client to do the requests, I’m not sure if there is any way to do that.

Regarding the pattern, I have a long list of records that I don’t really them to be ontologized, only when in need for the usage, so I’m querying the list and I’m creating the objects on demand, via OSDK, by querying the dataset. The purpose is, not having objects without usage causing resource consumption when they are not needed and my list is huge, which would cause a lot of pollution to the object type.

DEBUG [2025-04-02T17:44:31.535974979Z] Starting new HTTPS connection (1): stack.palantirfoundry.com:443
DEBUG [2025-04-02T17:44:31.542951345Z] Exception thrown when attempting to run <function ContextHTTPClient.request at 0x7f4c1234a480>, attempt 0 of 3
DEBUG [2025-04-02T17:44:31.644458771Z] Starting new HTTPS connection (2): stack.palantirfoundry.com:443
DEBUG [2025-04-02T17:44:31.655462742Z] Exception thrown when attempting to run <function ContextHTTPClient.request at 0x7f4c1234a480>, attempt 1 of 3
DEBUG [2025-04-02T17:44:31.75727129Z] Starting new HTTPS connection (3): stack.palantirfoundry.com:443
DEBUG [2025-04-02T17:44:31.767488956Z] Exception thrown when attempting to run <function ContextHTTPClient.request at 0x7f4c1234a480>, attempt 2 of 3
DEBUG [2025-04-02T17:44:31.869091034Z] Starting new HTTPS connection (4): stack.palantirfoundry.com:443
4

I’ve just tried with the function you’ve sent and the problem persists:

5

I have seen this error when the function is not deployed. After you published, did you deploy it?

Regarding the client that is being passed - you don’t need to use it. Since the egress policy is defined calling the hostname will work from any library (exception: agent proxy use cases)

1 Like
6

Hey Nico!! It worked after I’ve deployed it!

Thank you for this tip!

1 Like
7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.