Help with using AWS Javascript SDK with Typescript function

Hi,

Im not a seasoned developer, I know my way around Python. Now with the help of LLM’s I have been venturing into writing code in other languages.

I apologize if its a stupid question Im asking.

I have been trying to write a function that will use Claude models via bedrock from my account. The goal is to register the model and use it in AIP logic.

Here is my code

import { Function, Integer } from "@foundry/functions-api";
import { ExternalSystems, UserFacingError } from "@foundry/functions-api";
import { BedrockRuntime } from "@foundry/external-systems/sources";
import { Sources } from "@foundry/external-systems";
import { ChatCompletion } from "@palantir/languagemodelservice/contracts";
import {
    FunctionsGenericChatCompletionRequestMessages,
    GenericCompletionParams,
    FunctionsGenericChatCompletionResponse,
} from "@palantir/languagemodelservice/api";

import {
    BedrockRuntimeClient,
    ConverseCommand,
    ConversationRole,
    Message,
} from "@aws-sdk/client-bedrock-runtime";

// Import the Node.js crypto module
import { randomBytes } from 'crypto';
import { Sha256 } from "@aws-crypto/sha256-js";
import { HttpRequest } from "@aws-sdk/protocol-http";
import { SignatureV4 } from "@aws-sdk/signature-v4";

// Add a polyfill for crypto.getRandomValues
const cryptoPolyfill = {
    getRandomValues: function(buffer: Uint8Array): Uint8Array {
        const bytes = randomBytes(buffer.length);
        buffer.set(bytes);
        return buffer;
    }
};

// Set global crypto if it doesn't exist
if (typeof global.crypto === 'undefined') {
    (global as any).crypto = cryptoPolyfill;
}

export class BedrockLLMFunctions {
    @ExternalSystems({ sources: [BedrockRuntime] })
    @ChatCompletion()
    public async wueryBedrock(
        messages: FunctionsGenericChatCompletionRequestMessages,
        params?: GenericCompletionParams
    ): Promise<FunctionsGenericChatCompletionResponse> {
        console.log("Starting queryBedrock function");

        try {
            const SECRETKEY = Sources.BedrockRuntime.getSecret("additionalSecretSECRETKEY");
            const ACCESSKEY = Sources.BedrockRuntime.getSecret("additionalSecretACCESSKEY");
            const region = "us-east-1";

            console.log("Retrieved secrets for Bedrock Runtime");

            // Create a custom signer
            const signer = new SignatureV4({
                credentials: {
                    accessKeyId: ACCESSKEY,
                    secretAccessKey: SECRETKEY,
                },
                region: region,
                service: "bedrock-runtime",
                sha256: Sha256,
            });

            // Initialize the client with the custom signer
            const client = new BedrockRuntimeClient({
                region: region,
                credentials: {
                    accessKeyId: ACCESSKEY,
                    secretAccessKey: SECRETKEY,
                },
                signer,
                maxAttempts: 3,
            });

            console.log("Initialized BedrockRuntimeClient");

            const conversation: Message[] = messages.map((m) => ({
                role: m.role.toLowerCase() as ConversationRole,
                content: [{ text: m.content }],
            }));

            const command = new ConverseCommand({
                modelId: "anthropic.claude-3-haiku-20240307-v1:0",
                messages: conversation,
                inferenceConfig: {
                    maxTokens: 512,
                    temperature: 0.5,
                    topP: 0.9,
                },
            });

            console.log("Sending command to Bedrock");
            const response = await client.send(command);

            if (!response.output?.message?.content?.[0]?.text) {
                throw new UserFacingError("Invalid response format from Bedrock");
            }

            const completionText: string = response.output.message.content[0].text;

            return {
                completion: completionText,
                tokenUsage: {
                    promptTokens: response.usage?.inputTokens ?? 0,
                    completionTokens: response.usage?.outputTokens ?? 0,
                    maxTokens: 512,
                },
            };
        } catch (error: unknown) {
            console.error("Error during queryBedrock execution:", error);
            const errorMessage = error instanceof Error 
                ? `Error calling Bedrock: ${error.message}\nStack: ${error.stack}`
                : "An unknown error occurred while calling Bedrock";
            return Promise.reject(new UserFacingError(errorMessage));
        }
    }
}

When I publish the code and run the code with input

{
  "messages": [
    {
      "role": "user",
      "content": "why is the sky blue"
    },
    {
      "role": "system",
      "content": "why is the sky blue"
    }
  ],
  "params": {
    "temperature": 1,
    "maxTokens": 1234
  }
}

I get error

LOG [2024-11-21T16:03:18.789Z] Starting queryBedrock function
LOG [2024-11-21T16:03:18.791Z] Retrieved secrets for Bedrock Runtime
LOG [2024-11-21T16:03:18.8Z] Initialized BedrockRuntimeClient
LOG [2024-11-21T16:03:18.8Z] Sending command to Bedrock
ERROR [2024-11-21T16:03:18.816Z] Error during queryBedrock execution: {}
UserFacingError: Error calling Bedrock: Secure random number generation is not supported by this browser.
Use Chrome, Firefox or Internet Explorer 11
Stack: Error: Secure random number generation is not supported by this browser.
Use Chrome, Firefox or Internet Explorer 11
    at s.getRandomValues.t.exports (UserCode:2:886319)
    at Object.getRandomValues (UserCode:2:942920)
    at de (UserCode:2:38444)
    at me (UserCode:2:38634)
    at UserCode:2:88314
    at async UserCode:2:89980
    at async p.wueryBedrock (UserCode:2:943847)
    at async Ne (FunctionsIsolateRuntimePackage:2:1080191)
    at async ye.executeFunction (FunctionsIsolateRuntimePackage:2:1080698)
    at async userFunction (FunctionsInitialization:10:43)

I have tried several prompting strategies with several LLM’s which is how I got “polyfill” to bypass the crypto error, then now Im in this place where random generation is not supported by this browser.

I tried to reach to folks who work on AWS SDK and they say, its issue with the runtime not being compatible with Node.js.

Anyone tried using AWS SDK in a typescript function before ? any recommendation on how to make it work ?

1 Like