forked from vufind-org/vufind
-
Notifications
You must be signed in to change notification settings - Fork 0
Web Search
Stefan Weil edited this page Apr 14, 2026
·
1 revision
The web search needs a crawler which collects web pages.
Then Tika is used to extract metadata and content from each web page.
The current VuFind code uses the Tika app. This Java application is run twice per page which is rather time consuming. Using a Tika server avoids this and is much faster.
Manual demo for extraction of metadata:
curl --silent -T https://www.bib.uni-mannheim.de/ihre-ub/ansprechpersonen/abteilungen/digitale-bibliotheksdienste/ http://localhost:9998/meta --header "Accept: application/json" | python3 -m json.tool
Manual demo for extraction of content:
curl --silent -T https://www.bib.uni-mannheim.de/ihre-ub/ansprechpersonen/abteilungen/digitale-bibliotheksdienste/ http://localhost:9998/tika/main --header "Accept: text/plain"