Hi all! I am looking for the best way to connect to an external source. It is inside a network so I need to use an agent proxy. The issue is, all configuration options are https based, but the endpoint is http. I tried creating a generic connector using an agent proxy and adding in the certificates there, but there was no option to add a domain - I can only do that via direct connection. I am at a loss on how to move forward and don’t know what my options are.
Also, I tried running the agent proxy connection and manually adding the url in the code (I probably did this wrong) but from what I understand for the client to work you still need to add get_https_connection() in the script. Also when I ran it, I got the error
raise ValueError("Only single connection sources are supported.")
transforms.external.systems._redact_credentials_in_output.ValueError: Only single connection sources are supported.
Again my set up is all wrong so I don’t know where to start with troubleshooting
@external_systems(
source=Source("<rid>")
)
# @incremental(snapshot_inputs=['ids'], require_incremental=True)
@transform(
media_set_output=MediaSetOutput(
"<rid>"
)
)
def compute(ctx, source, media_set_output):
url = "http://pseudo.com"
client = source.get_https_connection().get_client
id = "10771574"
response = client.get(url + f"/{id}/AB")
if response.status_code == 200:
pdf_content = response.content
with tempfile.NamedTemporaryFile() as tmp:
tmp.write(pdf_content)
tmp.flush()
with open(tmp.name, "rb") as tmp_read:
media_set_output.put_media_item(tmp_read, path=id)
