How to build an AI chatbot widget powered by Llama 2

The age of LLMs and the application of Llama2

Following the rapid advancement of LLMs like OpenAI's ChatGPT, a multitude of integrations and plugins have been developed to leverage the capabilities of these models. As these developments unfold, data privacy remains a paramount concern, and measures have been taken to ensure that user information is protected and secure. Among the various ways to apply LLMs in business scenarios, there's a growing anticipation of using the Llama2 model as a stand-alone solution to establish a personalized AI engine.

If you’ve been waiting for an open alternative to closed-source chatbots, Llama 2-Chat is likely your best choice today!

In this tutorial, we will delve into how to create an efficient AI chatbot based on Llama2 using the Sendbird conversation platform.

LLM-based AI chatbots: Transitioning from a basic bot to a tailored AI assistant

AI chatbots, designed to offer comprehensive responses to user queries by learning from previous chat data, significantly improve user experiences. Such AI chatbots uplift applications or services by providing richer and more immersive user interactions.

With Sendbird, there's an opportunity to integrate a variety of LLM models, and now, there's a chance to directly utilize Llama2. By adopting Llama2, there's no longer a need to pass data to shared resources, ensuring data remains secure with Sendbird's high-level security measures. Furthermore, by leveraging Sendbird's technology, one can harness existing chat data for learning, paving the way for a more specialized chatbot experience.

The upcoming guide will illustrate how Sendbird's AI chatbot effectively harnesses the capabilities of Llama2.

Prerequisites to create a Llama2 AI ChatBot

Sendbird account: Go to the Sendbird Dashboard and create an account for a free trial. If you already have a Sendbird account, sign into your account.

Implementation: How to create a Llama2 AI chatBot

Step 1: Get Sendbird Application ID

1. Create a new Sendbird application by clicking `Create +` at the middle of your screen.

2. Choose a Product Type and enter a name for your application. Select the appropriate server region. Then, click Confirm.

3. Click the application you just created under Applications on the top-left corner of your screen.

4. On the Overview page, you can find the Application ID which you will need when initializing the Chat SDK. You will need this Application ID later in this tutorial!

Step 2: Set up your AI chatbot

1. Click on the `Create bot +` button to set up a new AI chatbot.

2. In the Bot Name field, enter a unique Bot Name and Bot ID of your choice. Store the ID because you will need it when creating a channel for the Bot. For the Bot AI engine, select `Meta AI Llama2`.

Bot profile

Default message

• Welcome message

  • Start message from Bot when first talking to Bot

• Suggested replies

  • Suggested answers to be displayed with Welcome messages

• Sample implementation image

  

Bot settings

• Model

  • LLAMA2 7B (default): Default model

  • Please contact the Sales Team for the Custom Size Custom Tune model.

• System prompt

  • `System prompt` is very good at telling Llama 2 who it should be or constraints for how it should respond.

    • ex) You are a customer service chatbot.

• Source

  • You can input data into the ChatBot's knowledge base. The ChatBot will generate appropriate answers based on the input data.

  • File
    

    

    • ex) Easy_recipes.pdf
      

  • URL
    

    

• Max tokens (1 ~ 2048)

  • Model generates text until the output length (excluding the input context length) reaches max_new_tokens. If specified, it must be a positive integer.

• Temperature (0.0 ~ 1.0)

  • Controls the randomness in the output. Higher temperature results in output sequence with low-probability words and lower temperature results in output sequence with high-probability words. If temperature -> 0, it results in greedy decoding. If specified, it must be a positive float.

• Top P (0.0 ~ 1.0)

  • In each step of text generation, sample from the smallest possible set of words with cumulative probability top_p. If specified, it must be a float between 0 and 1.

3. Click Create.

Step 3: Testing your AI chatbot

After select your AI chatbot is created, you can start a test conversation directly from the web interface.

Step 4: Fine-Tuning (Reach Out to the Sales Team)

The currently available Llama2 is based on the 7B model. By utilizing larger models and training with managed data, one can achieve a desired level of LLM ChatBot performance.

At Sendbird, we offer access to chat history and have the expertise to train and provide tailored models. One can conduct training by combining the fundamental training dataset with Sendbird's chat history.

If you're interested in a custom model, please contact our Sales team.

Run LLama2 with Chat Widget

The most widely known form of ChatBot service is the Web Widget. The Chat-AI-Widget provided by Sendbird can be easily built by simply entering the Application ID and Bot ID set in the Sendbird Dashboard.

If you are operating a Web Service using React:

1. You can install the Sendbird chat-ai-widget library

2. Add import ... and <ChatAiWidget/> Component to your Code.

Even if you don't use React, by setting up the following in your serving Html, refer to example.html

1. Incorporate React, ReactDOM, Chat-AI-Widget and babel Libraries

2. Set up the Chat AI Widget

Try Sendbird to build an AI chatbot powered by Llama 2

And that's it! You now know how to build an AI chatbot based on Meta's LLM, Llama 2. In this tutorial, we discussed the steps to creating an efficient AI chatbot based on Llama2. We recommend that you dive deeper into our tutorials to unlock the full potential of generative AI. Sendbird empowers you to shape customer conversations that resonate with your audience, making every interaction meaningful and memorable. Enhance customer satisfaction, streamline customer communication, and foster loyalty with Sendbird's conversational AI capabilities today! If you need help or have more questions, please start a discussion in the Sendbird Community or contact us. Our experts are always happy to help!

Happy AI chatbot building!🧑‍💻