Autonomous fine-tuning is a powerful feature that allows developers to create custom AI models without requiring machine learning expertise or human intervention.

Unlike traditional fine-tuning approaches, which require extensive experience and manual oversight, our autonomous fine-tuning agent only requires a well-curated dataset and a few granular parameters.

The user interface (UI) and user experience (UX) of the Autonomous Fine-Tuning agent are designed to be intuitive and easy to use, so any type of developer can use it and focus on building the AI-enhanced applications they want to build.

Under the hood, Prem has sophisticated data augmentation and distributed training architectures that maximize your custom model’s performance.

Save time, money and resources by letting Prem handle the heavy lifting.

This is one of our most detailed guides. So buckle up and get ready to fine-tune your models like a pro.

​

Best Practices

​

Data Preparation: Start Small, Iterate Frequently.

• Begin with a small well-curated examples for your specific use case. Even a small dataset can serve as a strong foundation when combined with the system’s autonomous data augmentation.

​

Use the Prem Playground for Rapid Feedback

• Leverage the interactive comparison feature in the Prem Playground to quickly gauge which model variant is performing well on your task.

• Perform red-teaming tests by providing diverse, challenging prompts. This not only uncovers vulnerabilities but also creates high-value data for subsequent fine-tuning stages.

​

Emphasize Quality in Data Collection

• Automated data harvesting from production systems is powerful, but ensure you incorporate clear filtering and quality checks. Use domain-specific rules and engagement metrics to keep your training data relevant and noise-free.

• Multi-dimensional annotations (accuracy, relevance, style, etc.) can provide richer signals for model training, resulting in more robust outcomes.

​

Make the Most of Autonomous Data Augmentation

• Rely on the platform’s multi-agent orchestration to expand your dataset systematically without compromising on semantic consistency.

• Configure domain constraints carefully to avoid generating irrelevant or contradictory samples. For instance, if you’re focusing on code generation, ensure that your sandboxed environment and validation criteria are well-defined before augmentation begins.

​

Choose the Right Model Family & Resources

• Select a model family that aligns with your task domain (NL2NL, NL2C, or specialized models like Text to SQL). Each family has different baseline capabilities and constraints.

​

Continuously Evaluate & Prune

• Run parallel trainings on different model candidates and let the system’s leaderboard identify top performers. This multi-model strategy helps surface unexpected winners (smaller models may outperform bigger ones for certain tasks).

• Prune underperforming models early to save computational resources. Focus on refining promising candidates with additional synthetic data or domain-specific fine-tuning.

​

Close the Loop with Active Learning (Coming Soon ⚠️)

• Continually gather new examples from both user feedback and system logs. The more “real-world” data you incorporate, the more resilient your models become.
• Use feedback modification tools in the Prem Platform to annotate or correct model responses. These corrected examples directly fuel the next iteration of fine-tuning.

​

Getting Started: Step by Step Guide

[
  {
    "messages": [
      {
        "role": "system",
        "content": "..."
      },
      {
        "role": "user",
        "content": "..."
      },
      {
        "role": "assistant",
        "content": "..."
      }
    ]
  }
]

{"messages": [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
{"messages": [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
{"messages": [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}

from premai import Prem

client = Prem(
    api_key=YOUR_API_KEY,
)

project_id = YOUR_PROJECT_ID

model = YOUR_FINE_TUNED_MODELS_NAME

messages = [
    {
        "role": "user",
        "content": "How do I check if TPM is enabled in Windows after configuring it in BIOS?",
    },
]

response = client.chat.completions.create(
    project_id=project_id,
    messages=messages,
    model=model,
    stream=False,
)

print(response)

​

Finished!

You’ve now successfully fine-tuned your model and launched it using the Launchpad.

Remember to check that Prem also has an API that works just like the SDK. So if you’re using a different language, you can still use Prem.