server / ranking : add sorting and management of top_n

[YannFollet]

server / ranking : add sorting and management of top_n

exemple output for top_n = 4 :

{
  "model": "M",
  "object": "list",
  "usage": {
    "prompt_tokens": 218,
    "total_tokens": 218
  },
  "results": [
    {
      "index": 6,
      "relevance_score": 0.7979143261909485
    },
    {
      "index": 0,
      "relevance_score": 0.3886369466781616
    },
    {
      "index": 2,
      "relevance_score": 0.2865810990333557
    },
    {
      "index": 4,
      "relevance_score": -0.5105927586555481
    }
  ]
}

or

[
  {
    "index": 6,
    "score": 0.7979143261909485
  },
  {
    "index": 0,
    "score": 0.3886369466781616
  },
  {
    "index": 2,
    "score": 0.2865810990333557
  },
  {
    "index": 4,
    "score": -0.5105927586555481
  }
]

[ggerganov]

I don't think the change is backwards compatible - would it work if the request does not include "top_n"?

[YannFollet]

no it won't work if no top_n, I can change it to make it back compatible

[ggerganov]

Yes, it should remain backward compatible.

[ngxson]

adding a test case is also required for this type of change

[YannFollet]

adding a test case is also required for this type of change

What king of test case? one bash script in comment to test many cases ?

[YannFollet]

here is one script to test the reranking

#!/bin/bash

# Set default URL if not provided
URL=${1:-http://127.0.0.1:8181/}

#replace texts by textsx to cancel tei_format
#replace top_n by top_nx to simulate no top_n

curl "$URL/v1/rerank" -H "Content-Type: application/json" \
 -d '{ "model": "M", "query": "What is the recipe to make bread ?",
 "return_text" : true,
 "texts" : true,
 "top_n": 6,
 "documents": [
 "voici la recette pour faire du pain, il faut de la farine de l eau et du levain et du sel",
 "it is a bear",
 "bread recipe : floor, water, yest, salt",
 "The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.",
 "here is the ingedients to bake bread : 500g floor, 350g water, 120g fresh refresh yest, 15g salt",
 "recipe to make cookies : floor, eggs, water, chocolat",
 "here is the recipe to make bread : 500g floor, 350g water, 120g fresh refresh yest, 15g salt",
 "il fait tres beau aujourd hui",
 "je n ai pas faim, je ne veux pas manger",
 "je suis a paris"
 ] }' | jq

[ggerganov]

adding a test case is also required for this type of change

What king of test case? one bash script in comment to test many cases ?

You should add a server test: https://github.com/ggml-org/llama.cpp/tree/master/tools/server/tests

[YannFollet]

you need this ?

./tests.sh unit/test_rerank.py -v -x
================================================ test session starts =================================================
platform linux -- Python 3.12.3, pytest-8.3.5, pluggy-1.6.0 -- /home/yann/dev/yann/llama.cpp/tools/server/tests/test/bin/python3
cachedir: .pytest_cache
rootdir: /home/yann/dev/yann/llama.cpp/tools/server/tests
configfile: pytest.ini
plugins: anyio-4.11.0
collected 8 items                                                                                                    

unit/test_rerank.py::test_rerank PASSED                                                                        [ 12%]
unit/test_rerank.py::test_rerank_tei_format PASSED                                                             [ 25%]
unit/test_rerank.py::test_invalid_rerank_req[documents0] PASSED                                                [ 37%]
unit/test_rerank.py::test_invalid_rerank_req[None] PASSED                                                      [ 50%]
unit/test_rerank.py::test_invalid_rerank_req[123] PASSED                                                       [ 62%]
unit/test_rerank.py::test_invalid_rerank_req[documents3] PASSED                                                [ 75%]
unit/test_rerank.py::test_rerank_usage[Machine learning is-A machine-Learning is-19] PASSED                    [ 87%]
unit/test_rerank.py::test_rerank_usage[Which city?-Machine learning is -Paris, capitale de la-26] PASSED       [100%]

================================================= 8 passed in 4.32s ==================================================

[ngxson]

does this work?

Suggested change

	int top_n = documents.size(); // no top_n will return all the documents
	int top_n = json_value(body, "top_n", (int)documents.size());

[ngxson]

This part is probably redundant, as you can simply resize() the elements vector