Improve performance and offline robustness of taxonomic resource loading - #266
Conversation
Cache loaded resources in a package-private environment (.pkg_cache) so repeated calls to load_taxonomic_resources() within a session return immediately without re-downloading or re-processing. Only "stable" data is cached (keyed by version); "current" data is always reloaded. Adds clear_cached_resources() to allow users to force a reload, and tests covering all cache behaviours.
When no internet connection is available, default_version() now falls back to the most recently downloaded local version rather than returning NULL. dataset_get() reads local parquet files directly when offline instead of bailing out. Adds local_cached_versions() helper to discover locally available versions. Tests cover the offline fallback path for default_version(), dataset_get(), and load_taxonomic_resources().
Replace the single skipped connection test with thorough offline tests covering every network-touching function (default_version, dataset_get, dataset_access_function, load_taxonomic_resources, get_versions). Tests simulate offline via NETWORK_UP=FALSE so they require no internet and run on CRAN. Each test verifies both that a message is emitted and that NULL is returned rather than an error.
|
my claude tells me that your claude build this:
So the flow is: Same version called again this session → returns from memory instantly Note: "current" data is never cached at either level, since it's meant to be live. |
… add clear_cached_resources function
|
is there any way to cache the version number somehow? currently |
f58e536 to
ea4838b
Compare
There was a problem hiding this comment.
I went through the main code changes and changed documentation, then turned to testing functionality and got the expected results with and without internet, without and without clearing cached resources, trying all combinations. It enormously increased the speed of functions running.
Address #263
Summary
load_taxonomic_resources()now caches results in a package-private environment (.pkg_cache) so repeated calls within a session return immediately without re-downloading or re-processing. Addsclear_cached_resources()to force a reload. This is CRAN-compliant — the cache lives in the package namespace, not.GlobalEnv.default_version()falls back to the most recently downloaded local version, anddataset_get()reads cached parquet files directly. The package now works fully offline as long as resources have been downloaded at least once.NETWORK_UP=FALSEso they require no internet and run on CRAN.Test plan
devtools::test()passes with no failuresload_taxonomic_resources()returns cached result on second call (check for "Using cached" message)load_taxonomic_resources()works withNETWORK_UP=FALSEwhen parquet files are locally presentclear_cached_resources()forces a fresh load on next callR CMD checkto confirm CRAN compliance🤖 Generated with Claude Code