Code Repo with Input & Output in Data Pipeline

[momo301]

I have a code repository that makes external REST API calls. I’d like for the API calls to be dependent on user input which requires the code repo to have an export control, as outlined here, since the transform is now using an input and external systems.

My problem is that I don’t have the necessary permissions to modify my code repo settings to enable dataset inputs, a defined in the linked API docs.

Any help or recommendations for a workaround are welcome!

[yix]

Is there someone who has the Information Security Officer permissions able to set up the repository to allow input datasets to external transforms per the docs?

[momo301]

I’m not necessarily sure how to check if I have the ISO role assigned to me, even after consulting the docs. I have a new dev account in which I am the only member of my own ‘organization’.

I’m able to check the ‘Allow use of Foundry inputs with external systems in this repository’ box, but I see no results under ‘Markings’. After combing through the docs, I’m also not sure how to go about creating markings and applying them to my input dataset.

[2c17e0fe21fe49c8714d]

Came across similar authorization issues in other parts of my workflow. Assuming this is a Foundry organization you have made for the Winter Fellowship, you should be able to change permissions yourself if you dive into settings.

[michaelt]

Hi @momo301 ,

You can configure this in Control Panel, available under Applications. Just load this up and search for Information Security Officer.

[momo301]

Ok, I’m now getting a new error after confirming that I have the necessary permissions and ISO role. Upon attempting to run my code repo I get a ‘permission denied’ error. I think this is due to the fact that I have not created and applied any markings to the input dataset that is trying to send data to external systems via a REST API call.

I found some relevant docs that cover creating a marker, but I’m not sure how to access the corresponding page on Foundry to do so. Anyone know where to find the ‘manage markings’ page?

[nicornk]

You don’t need to apply any markings on your input dataset but you might need to pass the „Org Marking“ to the ExportControl class.

Maybe someone else can chime in how to retrieve the Org Marking. On our stack it’s a static value.

Edit: Settings → Organizations → Select your org → Copy the Marking ID on the right hand side.

[momo301]

Not sure which settings page you’re referring to, any additional details would be appreciated! I found an Rid that I thought might solve the issue by going to:

Control Panel → Enrollment details → Show details → Rid

However, I still get the same permission denied error when passing in said Rid to ExportControl()

Also when I ask AIP Assist where to find org Rids, it tells me that orgs don’t have Rids associated with them; not sure If that’s a hallucination or not.

[nicornk]

I was referring to this page:

https://stack.foundry.com/workspace/settings/organizations

In addition you need to activate it in the repository settings:

https://stack.foundry.com/workspace/data-integration/code/repos//settings/repo_preferences

Sample code

from transforms.api import transform, Input, Output
from transforms.external.systems import use_external_systems, EgressPolicy, Credential, ExportControl
 
 
@use_external_systems(
    export_control=ExportControl(markings=["<org-marking-id"]),
    egress=EgressPolicy("<egress-policy>"),
    creds=Credential("<credential-rid>")
)
@transform(
    output=Output("ri.foundry.main.dataset.2345"),
    input=Input("ri.foundry.main.dataset.1234"),
)
def compute(ctx, export_control, egress, creds, output, input):
    username = creds.get("USERNAME")
    password = creds.get("PASSWORD")
    ...

Hope this helps?

[momo301]

Thank you! I was struggling to find the ‘organizations’ page, and the organizations ‘Marking ID’ was indeed that parameter that had to be passed into the ExportControl() function.