Skip to content

Commit 2364dc7

Browse files
alexklibiszclaude
andauthored
Dependencies: upgrade Elasticsearch to 8.18.5 (#834)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 125e791 commit 2364dc7

7 files changed

Lines changed: 110 additions & 5 deletions

File tree

‎.claude/settings.json‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"permissions": {
3+
"allow": [
4+
"Bash(task:*)",
5+
"Bash(sbt -client:*)",
6+
"Bash(git:*)"
7+
]
8+
}
9+
}
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
---
2+
name: upgrade-elasticsearch
3+
description: Upgrade Elasticsearch end-to-end: bump versions, check Lucene, compile, unit test, integration test, open PR.
4+
argument-hint: "[target version, e.g. 8.18.5]"
5+
---
6+
7+
Upgrade Elasticsearch to the version specified by the user (or infer it from context). Follow these steps in order, fixing any issues before moving to the next step:
8+
9+
1. **Bump versions** in all four places:
10+
- `ElasticsearchVersion` in `build.sbt`
11+
- Image tag in `docker/Dockerfile`
12+
- Image tag and release download URL in `docs/pages/installation.md`
13+
- `version` file (format: `X.Y.Z.0`)
14+
15+
2. **Check if LuceneVersion needs updating.** ES ships with a bundled Lucene. If it changed, the old pinned version will show as "evicted" in the dependency tree:
16+
```
17+
sbt -client "elastiknn-plugin/dependencyTree" | grep lucene
18+
```
19+
If evicted, bump `LuceneVersion` in `build.sbt` to match.
20+
21+
3. **Verify compilation:** `task jvmCompile`. Fix any issues before proceeding.
22+
23+
4. **Verify unit tests:** `task jvmUnitTest`. Fix any failures before proceeding.
24+
25+
5. **Verify integration tests:** `task dockerRunTestingCluster` then `task jvmIntegrationTest`. Fix any failures.
26+
27+
Once all steps pass, open a PR.

‎CLAUDE.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Build and Test Commands
6+
7+
### Task CLI
8+
9+
Use the `task` CLI — read Taskfile.yaml for all available tasks and to understand how they work.
10+
11+
### SBT
12+
13+
The underlying tool for most tasks is `sbt`, specifically `sbt -client` (thin client connecting to a persistent sbt server).
14+
Sometimes you need to use it directly rather than using
15+
16+
**Running a single test class:**
17+
```bash
18+
sbt -client "elastiknn-plugin/testOnly com.klibisz.elastiknn.query.SomeSpec"
19+
```
20+
21+
### Integration Testing
22+
23+
Integration tests require a Docker cluster (see `dockerRunTestingCluster` / `dockerStopTestingCluster` tasks in Taskfile.yaml).
24+
25+
### Java Version
26+
27+
The project uses Java 21 (see `.tool-versions`). Manage via asdf. Building with a newer JDK will embed that version in `plugin-descriptor.properties`, causing the plugin install to fail in the Docker container (which runs ES's bundled JDK).
28+
29+
## Project Structure
30+
31+
The project has seven SBT subprojects (Scala 3.3.6):
32+
33+
| Module | Role |
34+
|--------|------|
35+
| `elastiknn-api4s` | API types: (`Vec`, `Mapping`, `NearestNeighborsQuery`, etc.) XContent serialization |
36+
| `elastiknn-models` | LSH model implementations and vector similarity math (Java + Panama SIMD) |
37+
| `elastiknn-lucene` | Custom Lucene queries (`MatchHashesAndScoreQuery`) and field types |
38+
| `elastiknn-plugin` | ES plugin: mappers, query builders, score functions |
39+
| `elastiknn-plugin-integration-tests` | Full end-to-end tests against a live ES cluster |
40+
| `elastiknn-client-elastic4s` | Scala client library using elastic4s |
41+
| `elastiknn-jmh-benchmarks` | JMH microbenchmarks |
42+
43+
Dependency order: `api4s` ← `models` ← `lucene` ← `plugin` ← `integration-tests`
44+
45+
## Key Version Variables (build.sbt)
46+
47+
```scala
48+
ElasticsearchVersion // must match docker/Dockerfile and version file
49+
LuceneVersion // must match the Lucene bundled in the ES release
50+
ElastiknnVersion // read from the `version` file (e.g. 8.18.4.0)
51+
```
52+
53+
When upgrading ES, update all four: `build.sbt`, `docker/Dockerfile`, `docs/pages/installation.md`, and `version`. Also check whether the bundled Lucene version changed (`sbt elastiknn-plugin/dependencyTree | grep lucene`).
54+
55+
## Dependency Upgrades
56+
57+
Use `/upgrade-elasticsearch` to perform an Elasticsearch upgrade end-to-end.
58+
59+
Other JVM dependencies are mostly handled by Scala Steward via automated PRs (see git log for the pattern).
60+
61+
## SIMD / Panama Vector API
62+
63+
`PanamaFloatVectorOps` uses `jdk.incubator.vector` for SIMD-accelerated dot product, cosine, L1, and L2 computations. It is enabled at runtime via the `elastiknn.jdk-incubator-vector.enabled` setting. The sbt server and ES container both need `--add-modules jdk.incubator.vector` (already configured in `.sbtopts` and `build.sbt`).
64+
65+
SIMD `reduceLanes(ADD)` can produce slightly different floating-point results before vs. after JIT compilation — relevant when writing determinism tests.
66+
67+
## CI
68+
69+
`.github/workflows/ci.yaml` runs: lint → compile → assemble → unit tests → start Docker cluster → integration tests. Scala compiler is strict (`CiMode`) when `CI=true`; locally it uses `DevMode` (relaxed).

‎build.sbt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ Global / scalaVersion := "3.3.6"
77
Global / scalacOptions += "-explain"
88

99
lazy val CirceVersion = "0.14.14"
10-
lazy val ElasticsearchVersion = "8.18.4"
10+
lazy val ElasticsearchVersion = "8.18.5"
1111
lazy val Elastic4sVersion = "8.19.0"
1212
lazy val ElastiknnVersion = IO.read(file("version")).strip()
1313
lazy val LuceneVersion = "9.12.1"

‎docker/Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
FROM docker.elastic.co/elasticsearch/elasticsearch:8.18.4
1+
FROM docker.elastic.co/elasticsearch/elasticsearch:8.18.5
22
COPY elastiknn-plugin/target/elastiknn*.zip .
33
RUN elasticsearch-plugin install -b file:$(ls elastiknn*zip | sort | tail -n1)

‎docs/pages/installation.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,8 +42,8 @@ Make a Dockerfile like below.
4242
The image version (`elasticsearch:A.B.C`) must match the plugin's version (e.g. `A.B.C.x/elastiknn-A.B.C.x`).
4343

4444
```docker
45-
FROM docker.elastic.co/elasticsearch/elasticsearch:8.18.4
46-
RUN elasticsearch-plugin install --batch https://github.com/alexklibisz/elastiknn/releases/download/8.18.4.0/elastiknn-8.18.4.0.zip
45+
FROM docker.elastic.co/elasticsearch/elasticsearch:8.18.5
46+
RUN elasticsearch-plugin install --batch https://github.com/alexklibisz/elastiknn/releases/download/8.18.5.0/elastiknn-8.18.5.0.zip
4747
```
4848

4949
Build and run the Dockerfile. If you have any issues please refer to the [official docs.](https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html)

‎version‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
8.18.4.0
1+
8.18.5.0

0 commit comments

Comments
 (0)