InternVL Quantization with VisualQ&A Dataset

[Aristarx-Lintter]

Hello, Everyone.
Here I see that I cannot use my own dataset for compression and calibration.
Could you assist me how to use my own dataset or I must use vanilla auto-awq with my handwritten dataloader?

[gyanu2507]

That assert is real. lmdeploy.lite calibration only accepts c4, ptb, wikitext2, or pileval. There is no --calib-dataset /path/to/jsonl on that code path. For InternVL visual calibration the same function still goes through those text HF datasets unless you change the file.

Two practical options:

1. Keep lmdeploy lite calibrate for a first pass on ptb / wikitext2, then accept that the clip range is text-domain. Fine for a language-only LLM. Weak for InternVL if you care about image tokens.

2. If you need your Visual Q&A set, do not fight that assert. Use AutoAWQ (or a tiny fork of calibrate.py that replaces assert calib_dataset in [...] with a load_dataset('json', data_files=...) / your DataLoader). The rest of the file only needs tokenized input_ids of length calib_seqlen. Dump ~128 of your VQA prompts (and images, if the VL loader is in play) into that format.

You are not missing a hidden flag in v0.5-era calibrate.py. The dataset list is hardcoded on purpose. Vanilla AutoAWQ with your own dataloader is the intended escape hatch.