Paged JSON support, 'highlight' and 'snippet' URL parameters#61
Paged JSON support, 'highlight' and 'snippet' URL parameters#61Yetangitu wants to merge 3 commits into
Conversation
parameter (or page=0) to get unpaged result - added 'snippets' and 'highlight' parameters to search url, use 'highlight=0' to disable search term highlighting, 'snippets=0' to disable snippet generation. - added optional usjon module support The first two changes can be used in combination with the Searx meta-search plugin to Recoll to enable Searx to search Recoll sites.
| return '</span>' | ||
| #}}} | ||
| #{{{ recoll_search | ||
| def recoll_search(q, dosnippets=True): |
There was a problem hiding this comment.
options are communicated through q, no need for dosnippets
| @bottle.route('/json') | ||
| def get_json(): | ||
| query = get_query() | ||
| query['page'] = 0 |
There was a problem hiding this comment.
This allows the generation of paged JSON, set page=0 or use without page parameter to get an unpaged result.
| qs = query_to_recoll_string(query) | ||
| bottle.response.headers['Content-Type'] = 'text/csv' | ||
| bottle.response.headers['Content-Disposition'] = 'attachment; filename=recoll-%s.csv' % normalise_filename(qs) | ||
| res, nres, timer = recoll_search(query, False) |
There was a problem hiding this comment.
options are communicated through query, no need for extra parameter
| import string | ||
| import shlex | ||
| import urllib | ||
|
|
There was a problem hiding this comment.
usjon is quite a bit faster but there seem to be some concerns about correctness
| 'stem': 1, | ||
| 'timefmt': '%c', | ||
| 'dirdepth': 3, | ||
| 'dirdepth': 2, |
There was a problem hiding this comment.
the web interface can get unbearably slow with a deep tree, setting the default depth to 2 solves this issue.
* add support for extra databases through RECOLL_EXTRA_DBS
parameter, set this to the path for one or more (colon-separated)
xapiandb directories to have the webui query those databases.
The code assumes there to be a recoll.conf file one level below
the indicated database directories from which it will pull
the indexed top directories. When using the 'dir' parameter to
limit a query to a given directory it will only query those
databases (one or more) which match the indicated top directory.
* JSON endpoint now publishes number of results in 'nres' parameter
* recoll_search now gracefully denies to produce non-existing result
pages instead of crashing and burning
|
The last commits add support for using more than one database by setting The reason for implementing this scheme is to limit the total set size for directed queries as this increases performance (less data to search through) and flexibility (easier to add indexed directories). |
|
Hi, I merged the "Paged JSON support, 'highlight' and 'snippet' URL parameters" into https://opensourceprojects.eu/p/recollwebui/code which is where I (recoll dev) maintain the webui until koniu reappears. I can't merge the extradbs thing because the assumption about the config directory relative to the db one is wrong. If dbdir is set, the xapiandb can live anywhere. For the main interface, extrabds contains index directories to make it clear that the configuration parameters are ignored. In your case, I would use a different environment variable, and list configuration directories, from which you can retrieve both topdirs and dbdir (if the latter is not set, the xapian index indeed lives in the xapiandb subdir of the config). |
Merge from koniu/recoll-webui#61 - Changed JSON endpoint to allow paged json, use without page parameter (or page=0) to get unpaged result - Added snippets and highlight parameters to search url, use highlight=0 to disable search term highlighting, snippets=0 to disable snippet generation. - Added optional usjon module support
1 participant
This PR encompasses the following changes:
pageparameter (orpage=0) to get unpaged resultsnippetsandhighlightparameters to search url, usehighlight=0to disable search term highlighting,snippets=0to disable snippet generation.The first two changes can be used in combination with the Searx meta-search plugin to Recoll to enable Searx to search Recoll sites.