It looks like on May 13th our Python transform builds started breaking with the following error:
transforms.external.systems._redact_credentials_in_output.JSONDecodeError: Invalid control character at: line 5 column 46 (char 184)
Traceback (most recent call last): File "/app/work-dir/__python_runtime_environment__/__SYMLINKS__/site-packages/transforms/_build.py", line 331, in run self._transform.compute(**kwargs, **parameters) File "/app/work-dir/__python_runtime_environment__/__SYMLINKS__/site-packages/transforms/api/_transform.py", line 318, in compute self(**kwargs) File "/app/work-dir/__python_runtime_environment__/__SYMLINKS__/site-packages/transforms/api/_transform.py", line 226, in __call__ return self._compute_func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/work-dir/__environment__/__SYMLINKS__/site-packages/transforms/external/systems/_redact_credentials_in_output.py", line 24, in wrapper raise _redact_exception_chain(e, secrets) File "/app/work-dir/__environment__/__SYMLINKS__/site-packages/transforms/external/systems/_redact_credentials_in_output.py", line 21, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/app/work-dir/__user_code_environment__/__SYMLINKS__/site-packages/myproject/datasets/transcripts_from_gdrive.py", line 222, in compute json.loads(secret.encode().decode("unicode_escape").strip('"')), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/work-dir/__python_runtime_environment__/__SYMLINKS__/python/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/work-dir/__python_runtime_environment__/__SYMLINKS__/python/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/work-dir/__python_runtime_environment__/__SYMLINKS__/python/json/decoder.py", line 353, in raw_decode obj, end = self.scan_once(s, idx) ^^^^^^^^^^^^^^^^^^^^^^transforms.external.systems._redact_credentials_in_output.JSONDecodeError: Invalid control character at: line 5 column 46 (char 184)
We have an API secrets encoded as a JSON string and a base64 encoded JSON string that, when attempting to decode, all of a sudden started blowing up with this error without any code changes.
And the same thing happened with TypeScript functions. We started seeing the same error attempting to access JSON secrets:
INFO [2025-05-24T15:24:18.126702Z] SyntaxError: Unexpected token ':'INFO [2025-05-24T15:24:18.126703Z] at internalCompileFunction (node:internal/vm:77:18)INFO [2025-05-24T15:24:18.126704Z] at wrapSafe (node:internal/modules/cjs/loader:1288:20)INFO [2025-05-24T15:24:18.126705Z] at Module._compile (node:internal/modules/cjs/loader:1340:27)INFO [2025-05-24T15:24:18.126706Z] at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)INFO [2025-05-24T15:24:18.126707Z] at Module.load (node:internal/modules/cjs/loader:1207:32)INFO [2025-05-24T15:24:18.126708Z] at Module._load (node:internal/modules/cjs/loader:1023:12)INFO [2025-05-24T15:24:18.126709Z] at Module.require (node:internal/modules/cjs/loader:1235:19)INFO [2025-05-24T15:24:18.126710Z] at require (node:internal/modules/helpers:176:18)INFO [2025-05-24T15:24:18.126711Z] at validateServiceAccount (/app/dist/src/core/index.js:128:29)INFO [2025-05-24T15:24:18.126713Z] at getGoogleAuth (/app/dist/src/core/index.js:161:35)INFO [2025-05-24T15:24:18.126713Z] }INFO [2025-05-24T15:24:18.126714Z]}
To fix the issue in the functions layer I removed the code that used the REST API source to retrieve the base64 encoded JSON secret and just hard coded loading it via the file system by including the file in my container (which I do not want to continue doing but have no choice):
const absolutePath = path.resolve(process.cwd(), 'service_account.json');
const fileContents = fs.readFileSync(absolutePath, 'utf-8');
const credentials = JSON.parse(fileContents);
validateCredentialsStructure(credentials);
console.log('✅ Service account file loaded successfully'); return credentials;
Unfortunately, I can not apply a similar fix to our Python transforms without checking in the secret file to the repo, which I am not willing to do. The secret in question is provided by Google to authenticate signed requests for service users.
Please see what may have changed to cause this error in both our Python transforms and our TypeScript functions. They both retrieve the secrets from the same REST API source, which is why I am blaming a change in that layer of Foundry.