Skip to content

Forward revision, cache_dir and token to the task loaders in main_export - #260

Open
Sonica-B wants to merge 1 commit into
huggingface:mainfrom
Sonica-B:fix/export-forward-hub-kwargs
Open

Sonica-B wants to merge 1 commit into
huggingface:mainfrom
Sonica-B:fix/export-forward-hub-kwargs

Conversation

@Sonica-B

Copy link
Copy Markdown

Exporting with a pinned revision, a custom cache, or an explicit token through the Python API exports the wrong model. main_export(..., revision="v2") and ExecuTorchModelForCausalLM.from_pretrained(model_id, export=True, revision="v2") both produce a .pte built from the main weights. They also look in the default cache and send no token, so a private repo fails and a pinned export is not reproducible. The text-generation, fill-mask, text2text-generation and automatic-speech-recognition tasks are affected.

What happens today

main_export stores revision, cache_dir, subfolder, trust_remote_code and force_download in kwargs. It drops token and local_files_only. The task loaders then call from_pretrained(model_name_or_path) without any of them. image-classification and object-detection already forward **kwargs, so they were not affected.

flowchart LR
  subgraph Before
    A1["main_export(revision='v2', cache_dir, token)"] --> B1["kwargs: revision, cache_dir, ...<br/>(token, local_files_only dropped)"]
    B1 --> C1["tasks/masked_lm.py etc.<br/>from_pretrained(model_name_or_path)"]
    C1 --> D1["weights from main, default cache, no token"]
  end
  subgraph After
    A2["main_export(revision='v2', cache_dir, token)"] --> B2["kwargs incl. token, local_files_only"]
    B2 --> C2["from_pretrained(model_name_or_path, **get_hub_kwargs(kwargs))"]
    C2 --> D2["weights from v2, given cache and token"]
  end
Loading

Reproducer

The script below needs no network access. It builds a Hub-style cache for demo/tiny-bert with two revisions (main: 1 layer, v2: 2 layers), then exports each revision with main_export(..., task="fill-mask", recipe="portable", revision=rev). It prints the depth of the model the task loader returned.

repro_revision.py
"""main_export(..., revision="v2") exports the `main` weights, not v2. Offline, tiny random BERT."""
import os, sys, tempfile
from pathlib import Path

root = Path(tempfile.mkdtemp(prefix="oe_rev_"))
os.environ["HF_HUB_CACHE"] = str(root / "hub")
os.environ["HF_HUB_OFFLINE"] = "1"

from transformers import BertConfig, BertForMaskedLM

# Build a Hub-style cache for "demo/tiny-bert" with two revisions: main = 1 layer, v2 = 2 layers.
repo = root / "hub" / "models--demo--tiny-bert"
for ref, sha, layers in [("main", "a" * 40, 1), ("v2", "b" * 40, 2)]:
    cfg = BertConfig(vocab_size=99, hidden_size=32, num_hidden_layers=layers, num_attention_heads=2, intermediate_size=37)
    BertForMaskedLM(cfg).save_pretrained(repo / "snapshots" / sha)
    (repo / "refs").mkdir(parents=True, exist_ok=True)
    (repo / "refs" / ref).write_text(sha)

from optimum.exporters.executorch import main_export
from optimum.exporters.executorch.task_registry import discover_tasks, task_registry

discover_tasks()
loader = task_registry["fill-mask"]
def spy(path, **kw):
    module = loader(path, **kw)
    print(f"requested revision={kw.get('revision')!r} -> exported model has num_hidden_layers={module.config.num_hidden_layers}")
    return module
task_registry["fill-mask"] = spy

for rev in ["main", "v2"]:
    out = root / f"out_{rev}"
    main_export("demo/tiny-bert", task="fill-mask", recipe=os.environ.get("RECIPE", "portable"), output_dir=out, revision=rev)
    print("  wrote", sorted(p.name for p in out.glob("*.pte")))

Before:

requested revision='main' -> exported model has num_hidden_layers=1
  wrote ['model.pte']
requested revision='v2' -> exported model has num_hidden_layers=1
  wrote ['model.pte']

After:

requested revision='main' -> exported model has num_hidden_layers=1
  wrote ['model.pte']
requested revision='v2' -> exported model has num_hidden_layers=2
  wrote ['model.pte']

Changes

  • exporters/executorch/utils.py: new get_hub_kwargs(kwargs). It returns the Hub loading options present in kwargs (cache_dir, force_download, local_files_only, revision, subfolder, token, trust_remote_code).
  • exporters/executorch/__main__.py: main_export now also stores token and local_files_only in kwargs.
  • tasks/masked_lm.py, tasks/seq2seq_lm.py, tasks/asr.py and tasks/causal_lm.py pass these options to from_pretrained. In causal_lm.py they also go to AutoConfig.from_pretrained, so the config and the weights come from the same revision.
  • __main__.py and optimum/executorch/modeling.py: the default cache_dir is now huggingface_hub.constants.HF_HUB_CACHE instead of the legacy HUGGINGFACE_HUB_CACHE. The legacy constant ignores the HF_HUB_CACHE environment variable. Before this change the default was never forwarded, so it did not matter. Once it is forwarded, it would override a user's HF_HUB_CACHE. HF_HUB_CACHE resolves to the same path transformers uses when cache_dir is None.

All forwarded defaults (revision="main", subfolder="", token=None, trust_remote_code=False, force_download=False, local_files_only=False) match the transformers defaults, so calls without these arguments behave as before.

Not in this PR: tasks/multimodal_text_to_text.py and the Whisper AutoProcessor.from_pretrained call in Seq2SeqLMExportableModule have the same gap. I left them out because several open PRs edit those files. I can follow up once those land.

Tests

MainExportHubKwargsTest in tests/models/test_modeling_common.py covers fill-mask (BERT), text-generation (Llama) and text2text-generation (T5). Each case builds a tiny two-revision cache in a temp dir, calls main_export(..., cache_dir=..., revision="v2", local_files_only=True) with export_to_executorch mocked, and checks that the loaded model has 2 layers. The test needs no network and takes about 40 s on CPU.

Results on Windows 11, Python 3.11, torch 2.12.0+cpu, executorch 1.3.1, transformers 5.0.0rc1:

  • With the change: pytest tests/models/test_modeling_common.py -k "MainExportHubKwargs or cli_helps" gives 4 passed.
  • Same test on main: 3 failed. The loaders ignore cache_dir and raise LocalEntryNotFoundError.
  • black --check . and ruff check . both pass.

The XNNPACK recipe could not be run on Windows because the executorch wheel has no flatc. The reproducer uses the portable recipe, and the Linux CI covers XNNPACK.

main_export stored revision, cache_dir, subfolder, trust_remote_code and
force_download in kwargs and dropped token and local_files_only, but the
fill-mask, text2text-generation, automatic-speech-recognition and
text-generation loaders called from_pretrained(model_name_or_path)
without them. An export with revision="v2" therefore loaded and exported
the weights from main, from the default cache, without the given token.

Collect these options with a get_hub_kwargs helper and pass them to
from_pretrained (and to AutoConfig in the causal LM loader). Default
cache_dir to HF_HUB_CACHE instead of the legacy HUGGINGFACE_HUB_CACHE
constant, so forwarding it does not override a user's HF_HUB_CACHE.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant