Storing PGP Secrets

We have a data connection we are bringing encrypted data in from, and need to store a PGP key to do it. The type of data connection we are using doesn’t support storing arbitrary secrets. I can store the secret directly in the repo, but I get a message when turning on external transforms that this is no longer the recommended way. How should I be storing this secret if not in the repo/data connection?

Absolutely do not try to store your secret in the repository*. Store this as a custom secret in the source definition. See the docs here.


*Even if the source method does not work for you, there are many many better options that storing the secret in the plain text of the repository.

Sorry I didn’t mean in plain text, I meant in the credentials tab of the repo :laughing: , as used to be the recommended way. But enabling the credentials tab seems to no longer be recommended in favour of importing sources directly into the repo.

1 Like

Aha that did panic me a bit…

Yep importing the source is much smoother and generally super easy to set up in my experience. It also gives you a lot of consistency around the permissions of the source resource and is generally a lot cleaner.

Agreed, and in general that’s my preferred method. But the particular type of data connection I am using (SFTP) does not support storing arbitrary secrets on the connection itself (like a REST connection would), and therefore I would have to either

  • store the details in the repo
  • create a dummy data connection just to store the secret

I was wondering if there was a third way as neither of these seem ideal

Hi @ggarber

Adding an encrypted PGP property on the SFTP source that you can use in a code repository is something we’re actively working on.

Meanwhile, like you suggested, I’d recommend creating a stub REST API source that could store your egress policies, credentials and additional PGP key, to consume it in a code repo

Best
Barth

2 Likes

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.