Issues

• Optional EvalPort interop for RAGChecker results

  amazon-science/RAGChecker#40 · Filter by author adhabnr-ux opened on Aug 22, 2026

• Taxonomy question: real source vs unsupported claim support

  amazon-science/RAGChecker#38 · Filter by author reguorider-gif opened on May 18, 2026
  2

• CLI fails with Bedrock models unless

  amazon-science/RAGChecker#37 · Filter by author cventus opened on Jun 11, 2025

• Deprecation issue with the latest Transformers

  amazon-science/RAGChecker#34 · Filter by author AbhilashPal opened on May 9, 2025
  2

• What are the backbone models used in the output file without suffix "llama3"in meta_evalutation?

  amazon-science/RAGChecker#33 · Filter by author leekum2018 opened on May 5, 2025

• pip install ragchecker fails with Python 3.13 – works with Python 3.12

  amazon-science/RAGChecker#32 · Filter by author SeanShi23333 opened on Apr 16, 2025
  1

• Better with LLM

  amazon-science/RAGChecker#30 · Filter by author jacky68147527 opened on Mar 28, 2025

• RAGChecker Metric Input Requirements and Missing Data Handling

  amazon-science/RAGChecker#29 · Filter by author RyosukeSakaguchi opened on Mar 24, 2025
  1

• Differences in faithfulness metric

  amazon-science/RAGChecker#26 · Filter by author dropberry opened on Feb 21, 2025

• scores for metrics is Different when executed multiple times

  amazon-science/RAGChecker#24 · Filter by author daiDai-study opened on Dec 16, 2024
  2

• Hope to support the setting of multiple LLM configurations.

  amazon-science/RAGChecker#21 · Filter by author yidasanqian opened on Nov 15, 2024

• Which metrics are adopted in meta-evaluation by ragchecker?

  amazon-science/RAGChecker#20 · Filter by author hurenjun opened on Nov 4, 2024
  2

• how to use local LLM

  amazon-science/RAGChecker#19 · Filter by author cuyuhanhan opened on Oct 8, 2024
  2

• how to generate Rag_result with 'query_id', 'query','gt_answer', 'response'

  amazon-science/RAGChecker#17 · Filter by author adityamity opened on Sep 13, 2024
  2

• Reduce the number of installed packages to decrease Docker image size

  amazon-science/RAGChecker#15 · Filter by author ulan-yisaev opened on Sep 7, 2024
  1

• Does it support adding new model or new metrics?

  amazon-science/RAGChecker#14 · Filter by author zhimin-z opened on Sep 4, 2024
  2

• ValueError: axes don't match array

  amazon-science/RAGChecker#12 · Filter by author yidasanqian opened on Aug 30, 2024
  1

• Error Loading RagChecker

  amazon-science/RAGChecker#11 · Filter by author ss8319 opened on Aug 30, 2024
  11

• Intuitive explanation of metrics

  amazon-science/RAGChecker#4 · Filter by author shahules786 opened on Aug 14, 2024

• When will the Benchmark Dataset approximately be open-sourced?

  amazon-science/RAGChecker#2 · Filter by author Mervyn-1 opened on Jul 31, 2024
  5