Fine-tune Llama 3.2-3B with instruction datasets
llama-3-2-fine-tuning.yaml
file.
hf://
scheme — this goes the same for other cloud storages as well, s3://
for Amazon S3 for example.As referenced in main.py
we are using Meta’s Llama 3.2 hosted on Hugging Face.main.py
.vessl run create -f llama-3-2-fine-tuning.yaml
You can find the files under Storage.
.pt
files to mounted volumes or model registry and ensures seamless checkpointing of fine-tuning progress.DistributedDataParallel
and simplifies the process for setting up multi-cluster, multi-node distributed training.