LlamaIndex is a framework for building context-augmented LLM applications by applying LLMs to private or domain-specific data. It supports use cases like question-answering chatbots, document understanding and information extraction. LlamaIndex provides tools for data ingestion, processing, and implementing complex query workflows that combine data access with LLM prompting.

​
Installation and setup

We start by installing llama-index and premai-sdk. You can type the following command to install: 
pip install premai llama-index
Before proceeding further, please make sure that you have made an account on PremAI and already created a project. If not, please refer to the quick start guide to get started with the PremAI platform. Create your first project and grab your API key. 
from llama_index.llms.premai import PremAI
from llama_index.core.llms import ChatMessage

​
Setup PremAI client with LlamaIndex

Once we imported our required modules, let’s setup our client. For now let’s assume that our project_id is 8. But make sure you use your project-id, otherwise it will throw error. In order to use llama-index with PremAI, you do not need to pass any model name or set any parameters with our chat-client. By default it will use the model name and parameters used in the LaunchPad.
If you change the model or any other parameters like temperature or max_tokens while setting the client, it will override existing default configurations, that was used in LaunchPad.
import os
import getpass

if os.environ.get("PREMAI_API_KEY") is None:
    os.environ["PREMAI_API_KEY"] = getpass.getpass("PremAI API Key:")

prem_chat = PremAI(project_id=8)

​
Chat Completions

Now you are all set. We can now start with interacting with our application. Let’s start by building simple chat request and responses using llama-index. 
messages = [
    ChatMessage(role="user", content="What is your name"),
    ChatMessage(
        role="user", content="Write an essay about your school in 500 words"
    ),
]
Please note that: You can provide system prompt in ChatMessage like this: 
messages = [
    ChatMessage(role="system", content="Act like a pirate"),
    ChatMessage(role="user", content="What is your name"),
    ChatMessage(role="user", content="Where do you live, write an essay in 500 words"),
]
Additionally, you can also instantiate your client with system prompt like this: 
chat = PremAI(project_id=8, system_prompt="Act like nemo fish")
In both of the scenarios, you are going to override your system prompt that was fixed while deploying the application from the platform. And, specifically in this case, if you override system prompt while instantiating the PremAI class then system message in ChatMessage won’t provide any affect.So if you want to override system prompt for any experimental cases, either you need to provide that while instantiating the client or while writing it in ChatMessage with a role system.
Now let’s call the model 
response = prem_chat.chat(messages)
print(response)
You can also convert your chat function to a completion function. Here’s how it works 
completion = prem_chat.complete("Paul Graham is ")
If you are going to place system prompt here, then it will override your system prompt that was fixed while deploying the application from the platform.You can find all the optional parameters here. Any parameters other than these supported parameters will be automatically removed before calling the model.

​
Native RAG Support with Prem Repositories

Prem Repositories which allows users to upload documents (.txt, .pdf etc) and connect those repositories to the LLMs. You can think Prem repositories as native RAG, where each repository can be considered as a vector database. You can connect multiple repositories. You can learn more about repositories here. Repositories are also supported in langchain premai. Here is how you can do it. 
query = "what is the diameter of individual Galaxy"
repository_ids = [
    1991,
]
repositories = dict(ids=repository_ids, similarity_threshold=0.3, limit=3)
First we start by defining our repository with some repository ids. Make sure that the ids are valid repository ids. You can learn more about how to get the repository id here.
Please note: Similar like model_name when you invoke the argument repositories, then you are potentially overriding the repositories connected in the launchpad.Now, we connect the repository with our chat object to invoke RAG based generations.
messages = [
    ChatMessage(role="user", content=query),
]

response = prem_chat.chat(messages, repositories=repositories)
print(response)
So, this also means that you do not need to make your own RAG pipeline when using the Prem Platform. Prem uses it’s own RAG technology to deliver best in class performance for Retrieval Augmented Generations.
Ideally, you do not need to connect Repository IDs here to get Retrieval Augmented Generations. You can still get the same result if you have connected the repositories in prem platform.

​
Streaming

In this section, let’s see how we can stream tokens using llama-index and PremAI. It is very similar to above methods. Here’s how you do it. 
streamed_response = prem_chat.stream_chat(messages)

for response_delta in streamed_response:
    print(response_delta.delta, end="")
This will stream tokens one after the other. Similar to complete method, we have stream_complete method which does streaming of tokens for completion. 
# This will stream tokens one by one

streamed_response = prem_chat.stream_complete("hello how are you")

for response_delta in streamed_response:
    print(response_delta.delta, end="")

​
Embeddings

In this section we are going to discuss how we can get access to different embedding model using PremAIEmbeddings with llama-index 
from llama_index.embeddings.premai import PremAIEmbeddings
We support lots of state of the art embedding models. You can view our list of supported LLMs and embedding models here. For now let’s go for text-embedding-3-large model for this example. 
import os
import getpass

if os.environ.get("PREMAI_API_KEY") is None:
    os.environ["PREMAI_API_KEY"] = getpass.getpass("PremAI API Key:")

prem_embedding = PremAIEmbeddings(
    project_id=8, model_name="text-embedding-3-large"
)
Setting model_name argument in mandatory for PremAIEmbeddings unlike chat.

​
Calling the Embedding Model

Now we are all set. Now let’s start using our embedding model with a single query followed by multiple queries (which is also called as a document) 
query = "Hello, this is a test query"
query_result = prem_embedding.get_text_embedding(query)

print(f"Dimension of embeddings: {len(query_result)}")
Dimension of embeddings: 3072 
print(query_result[:5])
[-0.02129288576543331, 0.0008162345038726926, -0.004556538071483374, 0.02918623760342598, -0.02547479420900345]