DSPy
This example explains how to use DsPY by Stanford NLP to interact with different chat models.
DSPy by Stanford NLP is a framework designed to optimize language model (LM) prompts and weights, simplifying the process of building complex systems with LMs by automating and unifying techniques for prompting, fine-tuning, and reasoning. It provides composable and declarative modules in Pythonic syntax and includes an automatic compiler to instruct LMs on executing declarative steps.
Installation and Setup
We start by installing dspy and premai-sdk. Use the following commands to install them:
pip install dspy premai
Before proceeding further, please make sure that you have made an account on PremAI and already created a project. If not, please refer to the quick start guide to get started with the PremAI platform. Create your first project and grab your API key.
from dspy import PremAI
Setup PremAI instance with DsPY
Once we have imported our required modules, let’s set up our dspy-premai client. For now, let’s assume that our project_id is 123. However, be sure to use your actual project ID; otherwise, it will throw an error.
To use dspy with prem, you do not need to pass any model name or set any parameters with our chat-client. By default it will use the model name and parameters used in the LaunchPad.
If you change the model or any other parameters like temperature or max_tokens while setting the client, it will override existing default configurations, that was used in LaunchPad.
import os
import getpass
if "PREMAI_API_KEY" not in os.environ:
os.environ["PREMAI_API_KEY"] = getpass.getpass("PremAI API Key:")
Chat Completions
Here is a quick example of how to get a response from the model:
llm = PremAI(project_id=123, api_key="your-premai-api-key")
print(llm("What is a large language model?"))
You can also change the generation parameters while calling the model. Here’s how you can do that:
print(llm(
"What is a large language model?",
temperature=0.7, max_tokens=20
))
If you are going to place system prompt here, then it will override your system prompt that was fixed while deploying the application from the platform.
You can find all the optional parameters here. Any parameters other than these supported parameters will be automatically removed before calling the model.
Native RAG Support
Prem Repositories which allows users to upload documents (.txt, .pdf etc) and connect those repositories to the LLMs. You can think Prem repositories as native RAG, where each repository can be considered as a vector database. You can connect multiple repositories. You can learn more about repositories here.
Repositories are also supported in dspy premai. Here is how you can do it.
query = "what is the diameter of individual Galaxy"
repository_ids = [1991, ]
repositories = dict(
ids=repository_ids,
similarity_threshold=0.3,
limit=3
)
First we start by defining our repository with some repository ids. Make sure that the ids are valid repository ids. You can learn more about how to get the repository id here.
Please note: Similar like
modelwhen you invoke the argumentrepositories, then you are potentially overriding the repositories connected in the launchpad.
Now, we connect the repository with our chat object to invoke RAG based generations.
response = llm(query, max_tokens=100, repositories=repositories)
print(response)
print("---")
print(json.dumps(llm.history, indent=4))
Here is how an example generation would look like with Prem Repositories.
'The diameters of individual galaxies range from 80,000-150,000 light-years.'
---
[
{
"prompt": "what is the diameter of individual Galaxy",
"response": "The diameters of individual galaxies range from 80,000-150,000 light-years.",
"document_chunks": [
{
"repository_id": 1991,
"document_id": 1307,
"chunk_id": 173926,
"document_name": "Kegy 202 Chapter 2",
"similarity_score": 0.586126983165741,
"content": "n thousands\n of light-years. The diameters of individual\n galaxies range from 80,000-150,000 light\n "
},
{
"repository_id": 1991,
"document_id": 1307,
"chunk_id": 173925,
"document_name": "Kegy 202 Chapter 2",
"similarity_score": 0.4815782308578491,
"content": " for development of galaxies. A galaxy contains\n a large number of stars. Galaxies spread over\n vast distances that are measured in thousands\n "
},
{
"repository_id": 1991,
"document_id": 1307,
"chunk_id": 173916,
"document_name": "Kegy 202 Chapter 2",
"similarity_score": 0.38112708926200867,
"content": " was separated from the from each other as the balloon expands.\n solar surface. As the passing star moved away, Similarly, the distance between the galaxies is\n the material separated from the solar surface\n continued to revolve around the sun and it\n slowly condensed into planets. Sir James Jeans\n and later Sir Harold Jeffrey supported thisnot to be republishedalso found to be increasing and thereby, the\n universe is"
}
],
"kwargs": {
"max_tokens": 100,
"repositories": {
"ids": [
1991
],
"similarity_threshold": 0.3,
"limit": 3
}
},
"raw_kwargs": {
"max_tokens": 100,
"repositories": {
"ids": [
1991
],
"similarity_threshold": 0.3,
"limit": 3
}
}
}
]
So, this also means that you do not need to make your own RAG pipeline when using the Prem Platform. Prem uses it’s own RAG technology to deliver best in class performance for Retrieval Augmented Generations.
Ideally, you do not need to connect Repository IDs here to get Retrieval Augmented Generations. You can still get the same result if you have connected the repositories in prem platform.