Skip to content

Update TRL documentation - #9

Open
qgallouedec wants to merge 4 commits into
google-gemma:mainfrom
qgallouedec:sft-without-custom-collators
Open

qgallouedec wants to merge 4 commits into
google-gemma:mainfrom
qgallouedec:sft-without-custom-collators

Conversation

@qgallouedec

Copy link
Copy Markdown

Hey, TRL maintainer here 👋

This PR updates everything related to TRL to be more aligned with the current TRL API and the new best practices.

  • QLoRA guide:
    • the dataset is now prompt-completion, so TRL masks the prompt itself and the 70-line collator (skip_prepare_dataset, remove_unused_columns=False) goes away.
    • SFTTrainer gets the model id plus quantization_config and loads and quantizes the model itself, so from_pretrained and prepare_model_for_kbit_training go away too.
    • It still gets the -it tokenizer, because google/gemma-4-E2B ships no chat template. It's the tokenizer rather than the processor because, with the processor, TRL wraps string content into lists, and Gemma 4's template then adds a trailing space to the system turn that inference never sees.
  • Full fine-tune guide:

Both pages ran end to end on an H100 with a 3-step training run.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant