I would like to create a pipeline builder and its input dataset as a product in the Marketplace, and make it possible to install it in other organizations via the shared space.
At this time, I would like to have a system in place to prevent confidential data such as personal data from being accidentally installed in a development environment organization (i.e. an environment where confidential data should not be placed). How can this be achieved?
1 Like
Hey, this is already possible today although the docs may not make it clear. For context, we use DevOps and Marketplace very heavily so we’ve seen a lot of its ins and outs.
When you create a new Marketplace product via DevOps, you can simply add as a resource (i.e., Content) the Pipeline Builder file that contains all the transform logic you wish to package up. DevOps will be able to detect that the PB file you just added contains 1 (or N) input datasets, and in the Inputs tab, it will list the name of that dataset. You can rename the input to something else so that it’s clear it doesn’t need to be the same dataset. Then you can simply publish your product.
When someone wants to install the product to another organization, from the product install page, they’ll be provided a dropdown of Organizations on the Foundry instance that they want to install it in. In the Inputs page of the installation draft, they can then select any dataset in the target organization. Note that this assumes the input dataset already exists, and has an identical schema (column names and respective column data types; Marketplace will throw an error otherwise).
This will ultimately give the installer a bit more flexibility and an extra layer of security to prevent potential sensitive data from being installed in environments they were not meant for.
Thank you for your support.
Thank you for your support.
I understand that it’s possible to add just the Pipeline Builder to a Marketplace product, but since it’s possible to include input datasets if you want to, I’m looking into whether there’s a way to add additional checks to avoid accidentally installing datasets with sensitive information for other organizations.
I found out that expand access and downloader permissions are required to create a Marketplace product that includes a dataset under a shared space, so I think the only way is for users with these permissions to carefully add resources to the product.
Are there any other best practices, such as incorporating an approval process when adding resources to a product?
Understood! I’ve accidentally used the incorrect input dataset (i.e., from the wrong Organization) while installing a product to a new Organization before, so I think I know what you’re referring to.
In my experience, the only (manual) foolproof way to ensure you’re grabbing the correct input resource is to copy its RID in a separate browser tab and paste it into the Resource Locator modal that pops up when you’re selecting the input resource during installation. As far as I know, there aren’t any other first class features that act as a safeguard against this (yet), although I could be wrong.
1 Like
Thank you for your repeated responses!
I guess I just have to be careful when choosing my resources.
I’ll wait a little longer for advice from others.
Thank you so much.
Currently, there does not appear to be any functionality that would allow for an approval process when adding resources to a product, so this topic will be closed.