Hi THUDM/LongWriter maintainers,
EvalScope now supports LongWriter / LongBench-Write. This gives users another way to run the benchmark when they want a standardized evaluation workflow around OpenAI-compatible model endpoints, saved predictions, scoring, and report generation.
This may be useful for users who find this repository while setting up LongWriter / LongBench-Write and want a quick path to evaluate a hosted or locally served model.
Example:
from evalscope.third_party.longbench_write import run_task
run_task(task_cfg={
'model': 'YOUR_MODEL',
'url': 'OPENAI_API_COMPAT_URL',
'api_key': 'EMPTY_TOKEN',
})
Docs: https://evalscope.readthedocs.io/en/latest/third_party/longwriter.html
EvalScope repo: https://github.com/modelscope/evalscope
We can also open a small PR adding this as a README/Docs note, linked to this issue.
Hi THUDM/LongWriter maintainers,
EvalScope now supports LongWriter / LongBench-Write. This gives users another way to run the benchmark when they want a standardized evaluation workflow around OpenAI-compatible model endpoints, saved predictions, scoring, and report generation.
This may be useful for users who find this repository while setting up LongWriter / LongBench-Write and want a quick path to evaluate a hosted or locally served model.
Example:
Docs: https://evalscope.readthedocs.io/en/latest/third_party/longwriter.html
EvalScope repo: https://github.com/modelscope/evalscope
We can also open a small PR adding this as a README/Docs note, linked to this issue.