All issues

Search results

CUAD compute_score: undefined AUPR (NaN from np.trapz) is collapsed to 0, the worst attainable score

Status: Open.

#801 In huggingface/evaluate;

· buzzedaldrinx86-creator opened on Aug 23, 2026

EvalPort export/import helpers for bridging evaluate's aggregate-only API to portable per-example results

Status: Open.

#800 In huggingface/evaluate;

· adhabnr-ux opened on Aug 20, 2026

Proposal: Add ECE to evaluate

Status: Open.

#784 In huggingface/evaluate;

· itsfernn opened on Jul 31, 2026

Perplexity metric fails with GPT-2 tokenizer

Status: Open.

#766 In huggingface/evaluate;

· zanvari opened on Jun 17, 2026

New community metric: RAIL Score — responsible AI evaluation across 8 dimensions

Status: Open.

#742 In huggingface/evaluate;

· SumitVermakgp opened on Apr 1, 2026

Task Text Summarization does not work

Status: Open.

#741 In huggingface/evaluate;

· DominikHil opened on Mar 29, 2026

BERTScore: OverflowError with transformers>=5 due to undefined model_max_length

Status: Open.

#739 In huggingface/evaluate;

· uriyapesso opened on Mar 24, 2026

Proposal: Retrieval-Conditioned Confidence Metric (RCCS) for RAG evaluation

Status: Open.

#736 In huggingface/evaluate;

· devarakondasrikanth opened on Feb 26, 2026

BLEU: improve input validation and document edge-case behavior)

Status: Open.

#729 In huggingface/evaluate;

· Vangmay opened on Jan 28, 2026

Proposal: Make SQuAD QA F1 logic reusable as a generic QA metric in evaluate

Status: Open.

#724 In huggingface/evaluate;

· Ayesha-Imr opened on Dec 27, 2025

FileNotFoundError: Couldn't find a module script at /workspace/weiw13@xiaopeng.com/VLM/evaluate/meteor/meteor.py. Module 'meteor' doesn't exist on the Hugging Face Hub either.

Status: Open.

#723 In huggingface/evaluate;

· viviansmlie opened on Dec 18, 2025

Incompatibility with Gradio >= 4.0.0

Status: Open.

#713 In huggingface/evaluate;

· shivanraptor opened on Nov 3, 2025