Helps to fine-tune Open Source models using different readily available datasets or custom datasets for Text to SQL tasks.
premsql tuner
is a module designed to fine-tune models specifically for text-to-SQL tasks. The module offers multiple ways of fine-tuning, providing flexibility based on the needs of your project.
Text2SQLPeftTuner
class manages the fine-tuning process. Initialize it with the model path and experiment name to store the logs.
train
function of the tuner class to start the fine-tuning process. You need to provide the following arguments:
train_datasets
: The merged datasets used for training.output_dir
: Directory to save the fine-tuned model weights.num_train_epochs
: Number of training epochs.per_device_train_batch_size
: Training batch size per device.gradient_accumulation_steps
: Number of steps for gradient accumulation.evaluation_dataset
: The dataset used for validation; set to None
if no evaluation is required.eval_steps
: Number of steps between evaluations.max_seq_length
: Maximum sequence length permissible by the model.executor
: Use an executor if you have defined an evaluation dataset.filter_eval_results_by
: Filters evaluation results based on specified criteria (e.g., difficulty).**kwargs
, which will override any default settings.
Hereβs how to initiate the training:
./output
directory, and
logs will be saved in ./experiments/train/<experiment-name>
.
For an end-to-end example, you can check out our
fine-tuning using LoRA script.
With this guide, you should now be able to fine-tune models using premsql efficiently! Also nothing
is tightly coupled with our datasets, if you have made the datasets (which in the end is the standard input and output tokens tensor)