Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,7 @@ on:
paths:
- 'native-compression/libaec-jna/**'
- 'native-compression/libaec-native/**'
- 'native-compression/build.gradle'
- '.github/workflows/test-native-compression.yml'
- '.github/workflows/test-native-compression-aec.yml'

permissions:
contents: read
Expand All @@ -22,7 +21,7 @@ jobs:
macos-15,
macos-15-intel
]
name: netCDF-Java Native Compression Tests
name: netCDF-Java Native Compression Tests (aec)
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v6
Expand Down
63 changes: 63 additions & 0 deletions .github/workflows/test-native-compression-blosc2.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
on:
workflow_dispatch:
pull_request:
paths:
- 'native-compression/libblosc2-jna/**'
- 'native-compression/libblosc2-native/**'
- 'test-native-compression-blosc2.yml'

permissions:
contents: read

jobs:
tests:
strategy:
matrix:
os: [
ubuntu-24.04,
ubuntu-24.04-arm,
windows-2022,
macos-14,
macos-15,
macos-15-intel
]
name: netCDF-Java Native Compression Tests (blosc2)
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v6
- name: Setup JDK 8, 17
if: ${{ matrix.os == 'ubuntu-24.04' || matrix.os == 'macos-15-intel' }}
uses: actions/setup-java@v5
with:
distribution: 'temurin'
java-version: |
8
17
- name: Setup JDK 21
if: ${{ matrix.os != 'ubuntu-24.04' && matrix.os != 'macos-15-intel' }}
uses: actions/setup-java@v5
with:
distribution: 'temurin'
java-version: '21'
- name: Cache Gradle packages
uses: actions/cache@v5
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
restore-keys: |
${{ runner.os }}-gradle-
- name: Run libblosc2 JNA tests
if: ${{ matrix.os == 'ubuntu-24.04' || matrix.os == 'macos-15-intel' }}
run: ./gradlew -Dorg.gradle.java.installations.auto-detect=true clean :libblosc2-jna:test
env:
JDK8: /usr/thredds-test-environment/temurin8
- name: Run libblosc2 JNA tests (JDK 21 tests)
if: ${{ matrix.os != 'ubuntu-24.04' && matrix.os != 'macos-15-intel' }}
run: ./gradlew clean :libblosc2-jna:test21
- uses: actions/upload-artifact@v6
if: failure()
with:
name: NativeCompression_JUnit_Results_${{ github.sha }}_-${{ matrix.os }}
path: native-compression/libblosc2-jna/build/reports/tests
2 changes: 1 addition & 1 deletion docs/src/site/_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -79,4 +79,4 @@ base_docs_url: https://docs.unidata.ucar.edu/netcdf-java/
java_version_build: 17
java_version_runtime: 8
libaec_version: 1.1.3.0
cblosc2_version: 2.22.0.0
cblosc2_version: 2.23.1.0
2 changes: 1 addition & 1 deletion docs/src/site/pages/netcdfJava/Upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Note: we are looking to update the minimum version of the JVM we support for the
Please consider taking a moment to participate in the [poll on GitHub](https://github.com/Unidata/netcdf-java/discussions/1468){:target="_blank"}.

The 5.10.0 release adds support for reading blosc compressed data using the C-Blosc2 native library.
The new artifact for JNA support is `edu.ucar.unidata:libblosc2-jna` (the current version is `2.22.0.0`).
The new artifact for JNA support is `edu.ucar.unidata:libblosc2-jna` (the current version is `{{ site.cblosc2_version }}`).

### Native jar group name changes

Expand Down
10 changes: 0 additions & 10 deletions native-compression/README

This file was deleted.

24 changes: 24 additions & 0 deletions native-compression/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Updating Native Jars

Updating a native jar requires at least two commits.
The first commit updates a GitHub workflow that generates the native libraries using GitHub Actions; the second will update the `*-native` subproject build to use the new native libraries.
1. The `native-compression/*-native` subprojects depend on a .zip file and checksum file that contain the native libraries, which are hosted on Unidata servers.
These files are generated on GitHub using the `.github/worlflows/lib*.yml`.
To update a native jar, you first update the associated workflow.
Once the workflow is on the main branch (currently `maint-5.x`), manually trigger the workflow on GitHub.
The workflow output will provide a zip file and sha-256 checksum value.
2. Copy the zip file generated by the workflow in step 1 to the Unidata server and generate a checksum for the file.
Update the `native-compression/lib*-native/build.gradle.kts` file with the new version information and checksum value from the GitHub output.
If everything looks good, and the draft PR with these changes passes, update the documentation to reflect the latest version of the native jar (see `docs/src/site/_config.yml`).
Once these changes are merged, the final step is to publish a new native jar.

## Publishing Native Jars

The native jars are versioned based on the native library version (plus a build number).
These jars are also published separate to the rest of the project jars.
To publish a native jar, you must set the `unidata.native.publish=true` system property when running the Gradle publish command.
For example, to publish the libblosc2-native jar, run:

```
./gradlew -D"unidata.native.publish=true" :libblosc2-native:publish
```
6 changes: 3 additions & 3 deletions native-compression/libblosc2-native/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ plugins {

group = "edu.ucar.unidata"

var bloscVersion = "2.22.0"
var bloscVersion = "2.23.1"
var build = "0"

version = "${bloscVersion}.${build}"
Expand All @@ -25,10 +25,10 @@ project.extra["project.title"] = "Native libraries for libblosc2."

// zip file produced by GitHub workflow
val libblosc2Native =
"libblosc2-native-${bloscVersion}-b4a818caa60bbf90bc92ec51ffd01ce3f293c8a4.zip"
"libblosc2-native-${bloscVersion}-c8d21a734aa58320135f5dfdc36e3d61e316e074.zip"

// sha256 checksum from GitHub workflow output
val expectedChecksum = "895226b29a314c0beae92de66a181b66833873e8c3a8a501d0b12193bde30610"
val expectedChecksum = "6cfb464e7bc666509644d3d419ec6d5a43831178cbf9105d2ecb2dbc97ffdf88"

val resourceZip = file("$rootDir/project-files/native/libblosc2/$libblosc2Native")
val fetchNativeResources =
Expand Down
Loading