Hello~
We have two external systems to integrated with.
We’ve already finished developing python @function integrating with system A and used well from pipeline builder as a UDF. ( Enabled allow access to external systems from this repository option) Both, Preview and build works well.
The code should be looks like below.
from functions.api import function
from functions.sources import get_source
@function(sources=[“A”])
def my_function() → str:
source = get_source(“A”)
url = source.get_https_connection().url
client = source.get_https_connection().get_client()
response = client.get(url)
return response.text
Now. It’s a problem system.
The problem is system B.
We have to integrate with system B as same mechanism.
But, system B requires setting SSL Server Certificate.
-----BEGIN CERTIFICATE-----
MIIG6zCCBdOgAwIBAgIQAU/mbue6gDmNDupnZg+G7jANBgkqhkiG9w0BAQsFADBZ
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMTMwMQYDVQQDEypE
BlahBlahBlah
-----END CERTIFICATE-----
with verify option, we can preview well in python function.
response = client.post(url, headers=headers, json=body, verify=False)
But, In case of preview UDF from pipeline builder we got an error.
( Failed to load preview
Unable to fetch data)
I’m doubting strongly that it’s a certificate problem.
Also, I can see the error message that spark saying likes below.
Caused by: com.fasterxml.jackson.databind.exc.ValueInstantiationException: Cannot construct instance of `com.palantir.foundry.api.v2.omni.connectivity.ImmutableRestConnectionConfiguration`, problem: Cannot build RestConnectionConfiguration, some of required attributes are not set [additionalSecrets]
at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 260]
So I used this code but it’s useless.
filepath = “”
data = “”
with tempfile.NamedTemporaryFile() as tmp:
with open(tmp.name + “.pem”, “w”, encoding=“utf-8”) as tmp_f:
tmp_f.write(server_certificate)
filepath = tmp.name + “.pem”
response = client.post(url, headers=headers, json=body, verify=filepath)
Can you advise me how can I set SSL Server Certificate when we use python function?
(likes when we set on webhook,
)
If you need an addition information, I can provide.
Thanks regards.