Skip to main content

Get started

Intellinode module provides various language models, including OpenAI's ChatGPT and Llama V2 model from Replicate or AWS SageMaker.

We will demonstrate the setup for OpenAI's ChatGPT, followed by the two methods of integrating the Llama V2 model - through Replicate's API or AWS SageMaker dedicated deployment. All the models are available with the unified chatbot interface with a minimum code change when switching between models.

ChatGPT Model​

  1. Import the necessary modules from IntelliNode. This will include the Chatbot, ChatGPTInput, and ChatGPTMessage classes.
const { Chatbot, ChatGPTInput, ChatGPTMessage } = require('intellinode');
  1. To use OpenAI, you'll need a valid API key. Create a Chatbot instance, providing the API key and 'openai' as the provider.
const chatbot = new Chatbot(OPENAI_API_KEY, 'openai');
  1. Construct a chat input instance and add user messages:
const system = 'You are a helpful assistant.';
const input = new ChatGPTInput(system);
input.addUserMessage('Explain the plot of the Inception movie in one line.');
  1. Use the chatbot instance to send chat input:
const responses = await chatbot.chat(input);

responses.forEach(response => console.log('- ', response));

ChatGPT Streaming​

To use the ChatGPT streaming with IntelliNodese, call chatbot.stream method to send the chat input and receive a stream of responses:

let response = '';
for await (const contentText of chatbot.stream(input)) {
  response += contentText;
  console.log('Received chunk:', contentText);
}

By using the chatbot.stream, you can receive a stream of responses from ChatGPT instead of waiting for the entire conversation to complete. The stream function supported for openai provider only.

Cohere Model​

Initiate the chatbot with cohere coral model and web search capabilities.

  1. Import the necessary modules.
const { Chatbot, CohereInput, SupportedChatModels } = require('intellinode');
  1. Initiate the chatbot object with a valid api key from (cohere.com).
const bot = new Chatbot(process.env.COHERE_API_KEY, SupportedChatModels.COHERE);
  1. Prepare the input with cohere web search extension:
const input = new CohereInput('You are a helpful computer programming assistant.', {web: true});
input.addUserMessage('What is the difference between Python and Java?');
  1. Call the chatbot and parse the responses.
const responses = await bot.chat(input);
responses.forEach((response) => console.log('- ' + response));

Mistral AI​

Mistral provide open source mixer of experts models.

  1. Import the Chatbot and MistralInput modules.
const { Chatbot, MistralInput, SupportedChatModels } = require('intellinode');
  1. Initiate the chatbot object with a valid api key from (mistral.ai).
const mistralBot = new Chatbot(apiKey, SupportedChatModels.MISTRAL);
  1. Prepare the input and select your preferred mistral model like mistral-tiny or mistral-medium.
const input = new MistralInput('You are an art expert.', {model: 'mistral-medium'});
input.addUserMessage('Who painted the Mona Lisa?');
  1. Call the chatbot and parse the responses.
const responses = await mistralBot.chat(input);

Anthropic​

Anthropic provide models with large context window like claude 3.

  1. Import the Chatbot and AnthropicInput modules.
const { Chatbot, AnthropicInput, SupportedChatModels } = require('intellinode');
  1. Initiate the chatbot object with a valid api key from (console.anthropic.com).
const bot = new Chatbot(apiKey, SupportedChatModels.ANTHROPIC);
  1. Prepare the input and select your preferred claude model.
const input = new AnthropicInput('You are an art expert.', {model: 'claude-3-sonnet-20240229'});
input.addUserMessage('Who painted the Mona Lisa?');
  1. Call the chatbot and parse the responses.
const responses = await bot.chat(input);

Llama V2 Model​

Integration with Llama V2 is attainable via two options, using:

  1. Replicate's API: simple integration.
  2. AWS SageMaker: hosted in your account for extra privacy and control (SageMaker steps).

Replicate's Llama Integration​

  1. Import the necessary classes.
const { Chatbot, LLamaReplicateInput, SupportedChatModels } = require('intellinode');
  1. You'll need a valid API key. This time, it should be for replicate.com.
const chatbot = new Chatbot(REPLICATE_API_KEY, SupportedChatModels.REPLICATE);
  1. Create the chat input with LLamaReplicateInput
const system = 'You are a helpful assistant.';
const input = new LLamaReplicateInput(system);
input.addUserMessage('Explain the plot of the Inception movie in one line.');
  1. Use the chatbot instance to send chat input:
const response = await chatbot.chat(input);

console.log('- ', response);

Advanced Settings

You can create the input with the desired model name:

// import the config loader
const {Config2} = require('intellinode');

// llama 13B model (default)
const input = new LLamaReplicateInput(system, {model: Config2.getInstance().getProperty('models.replicate.llama.13b')});
// llama 70B model 
const input = new LLamaReplicateInput(system, {model: Config2.getInstance().getProperty('models.replicate.llama.70b')});

AWS SageMaker Integration​

Integration with the Llama V2 model via AWS SageMaker, providing an additional layer of control, is achievable through IntelliNode.

IntelliNode Integration​
  1. Import the necessary classes:
const { Chatbot, LLamaSageInput, SupportedChatModels } = require('intellinode');
  1. With AWS SageMaker, you'll be providing the URL of your API gateway, the steps to deploy your model and get the URL in the Prerequisite section:
const chatbot = new Chatbot(null /*replace with the API key, or null if the API gateway key not used*/, 
                            SupportedChatModels.SAGEMAKER, 
                            {url: process.env.AWS_API_URL /*replace with your API gateway url*/});
  1. Create the chat input with LLamaSageInput:
const system = 'You are a helpful assistant.';
const input = new LLamaSageInput(system);
input.addUserMessage('Explain the plot of the Inception movie in one line.');
  1. Use the chatbot instance to send the chat input:
const response = await chatbot.chat(input);

console.log('Chatbot response:' + response);
Prerequisite to Integrate AWS SageMaker and IntelliNode​

The steps to leverage AWS SageMaker for hosting the Llama V2 model:

  1. Create a SageMaker Domain: Begin by setting up a domain on your AWS SageMaker. This step establishes a controlled space for your SageMaker operations.
  1. Deploy the Llama Model: Utilize SageMaker JumpStart to deploy the Llama model you plan to integrate.
  1. Copy the Endpoint Name: Once you have a model deployed, make sure to note the endpoint name, which is crucial for future steps.

  1. Create a Node.js Lambda Function: AWS Lambda allows running the back-end code without managing servers. Create a Node.js lambda function to use for integrating the deployed model.

  2. Set Up Environment Variable: Create an environment variable named llama_endpoint with the value of the SageMaker endpoint.

  3. Intellinode Lambda Import: You need to import the prepared Lambda zip file that establishes a connection to your SageMaker Llama deployment. This export is a zip file, and it can be found in the lambda_llama_sagemaker directory.

  4. API Gateway Configuration: Click on the "Add trigger" option on the Lambda function page, and select "API Gateway" from the list of available triggers.

  1. Lambda Function Settings: Update the lambda role to grant necessary permissions to access SageMaker endpoints. Additionally, the function's timeout period should be extended to accommodate the processing time. Make these adjustments in the "Configuration" tab of your Lambda function.

Once you complete these steps, your AWS SageMaker will be ready to host and run the Llama V2 model, and you can easily integrate it with IntelliNode.