LlamaIndex

LlamaIndex is a framework for building context-augmented LLM applications by applying LLMs to private or domain-specific data. It supports use cases like question-answering chatbots, document understanding and information extraction. LlamaIndex provides tools for data ingestion, processing, and implementing complex query workflows that combine data access with LLM prompting.

​

Installation and setup

We start by installing llama-index and premai-sdk. You can type the following command to install:

pip install premai llama-index

Before proceeding further, please make sure that you have made an account on PremAI and already created a project. If not, please refer to the quick start guide to get started with the PremAI platform. Create your first project and grab your API key.

from llama_index.llms.premai import PremAI
from llama_index.core.llms import ChatMessage

​

Setup PremAI client with LlamaIndex

Once we imported our required modules, let’s setup our client. For now let’s assume that our project_id is 8. But make sure you use your project-id, otherwise it will throw error.

In order to use llama-index with PremAI, you do not need to pass any model name or set any parameters with our chat-client. By default it will use the model name and parameters used in the LaunchPad.

import os
import getpass

if os.environ.get("PREMAI_API_KEY") is None:
    os.environ["PREMAI_API_KEY"] = getpass.getpass("PremAI API Key:")

prem_chat = PremAI(project_id=8)

​

Chat Completions

Now you are all set. We can now start with interacting with our application. Let’s start by building simple chat request and responses using llama-index.

messages = [
    ChatMessage(role="user", content="What is your name"),
    ChatMessage(
        role="user", content="Write an essay about your school in 500 words"
    ),
]

Please note that: You can provide system prompt in ChatMessage like this:

messages = [
    ChatMessage(role="system", content="Act like a pirate"),
    ChatMessage(role="user", content="What is your name"),
    ChatMessage(role="user", content="Where do you live, write an essay in 500 words"),
]

Additionally, you can also instantiate your client with system prompt like this:

chat = PremAI(project_id=8, system_prompt="Act like nemo fish")

Now let’s call the model

response = prem_chat.chat(messages)
print(response)

You can also convert your chat function to a completion function. Here’s how it works

completion = prem_chat.complete("Paul Graham is ")

​

Native RAG Support with Prem Repositories

Prem Repositories which allows users to upload documents (.txt, .pdf etc) and connect those repositories to the LLMs. You can think Prem repositories as native RAG, where each repository can be considered as a vector database. You can connect multiple repositories. You can learn more about repositories here.

Repositories are also supported in langchain premai. Here is how you can do it.

query = "what is the diameter of individual Galaxy"
repository_ids = [
    1991,
]
repositories = dict(ids=repository_ids, similarity_threshold=0.3, limit=3)

First we start by defining our repository with some repository ids. Make sure that the ids are valid repository ids. You can learn more about how to get the repository id here.

messages = [
    ChatMessage(role="user", content=query),
]

response = prem_chat.chat(messages, repositories=repositories)
print(response)

So, this also means that you do not need to make your own RAG pipeline when using the Prem Platform. Prem uses it’s own RAG technology to deliver best in class performance for Retrieval Augmented Generations.

​

Streaming

In this section, let’s see how we can stream tokens using llama-index and PremAI. It is very similar to above methods. Here’s how you do it.

streamed_response = prem_chat.stream_chat(messages)

for response_delta in streamed_response:
    print(response_delta.delta, end="")

This will stream tokens one after the other. Similar to complete method, we have stream_complete method which does streaming of tokens for completion.

# This will stream tokens one by one

streamed_response = prem_chat.stream_complete("hello how are you")

for response_delta in streamed_response:
    print(response_delta.delta, end="")

​

Embeddings

In this section we are going to dicuss how we can get access to different embedding model using PremAIEmbeddings with llama-index

from llama_index.embeddings.premai import PremAIEmbeddings

We support lots of state of the art embedding models. You can view our list of supported LLMs and embedding models here. For now let’s go for text-embedding-3-large model for this example.

import os
import getpass

if os.environ.get("PREMAI_API_KEY") is None:
    os.environ["PREMAI_API_KEY"] = getpass.getpass("PremAI API Key:")

prem_embedding = PremAIEmbeddings(
    project_id=8, model_name="text-embedding-3-large"
)

​

Calling the Embedding Model

Now we are all set. Now let’s start using our embedding model with a single query followed by multiple queries (which is also called as a document)

query = "Hello, this is a test query"
query_result = prem_embedding.get_text_embedding(query)

print(f"Dimension of embeddings: {len(query_result)}")

Dimension of embeddings: 3072

print(query_result[:5])

[-0.02129288576543331, 0.0008162345038726926, -0.004556538071483374, 0.02918623760342598, -0.02547479420900345]