We have a data connection we are bringing encrypted data in from, and need to store a PGP key to do it. The type of data connection we are using doesn’t support storing arbitrary secrets. I can store the secret directly in the repo, but I get a message when turning on external transforms that this is no longer the recommended way. How should I be storing this secret if not in the repo/data connection?
Absolutely do not try to store your secret in the repository*. Store this as a custom secret in the source definition. See the docs here.
*Even if the source method does not work for you, there are many many better options that storing the secret in the plain text of the repository.
Sorry I didn’t mean in plain text, I meant in the credentials tab of the repo , as used to be the recommended way. But enabling the credentials tab seems to no longer be recommended in favour of importing sources directly into the repo.
Aha that did panic me a bit…
Yep importing the source is much smoother and generally super easy to set up in my experience. It also gives you a lot of consistency around the permissions of the source resource and is generally a lot cleaner.
Agreed, and in general that’s my preferred method. But the particular type of data connection I am using (SFTP) does not support storing arbitrary secrets on the connection itself (like a REST connection would), and therefore I would have to either
- store the details in the repo
- create a dummy data connection just to store the secret
I was wondering if there was a third way as neither of these seem ideal
Hi @ggarber
Adding an encrypted PGP property on the SFTP source that you can use in a code repository is something we’re actively working on.
Meanwhile, like you suggested, I’d recommend creating a stub REST API source that could store your egress policies, credentials and additional PGP key, to consume it in a code repo
Best
Barth
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.