Skip to content

feat(llm): add QMD_FORCE_CPU env var for older GPUs#337

Open
JasonOA888 wants to merge 1 commit intotobi:mainfrom
JasonOA888:feat/force-cpu-env
Open

feat(llm): add QMD_FORCE_CPU env var for older GPUs#337
JasonOA888 wants to merge 1 commit intotobi:mainfrom
JasonOA888:feat/force-cpu-env

Conversation

@JasonOA888
Copy link

Problem

On older NVIDIA architectures (e.g., Pascal), qmd query fails with:

[node-llama-cpp] CUDA error: the function requires an architectural feature absent from the device

Workaround (Before)

Users had to run:

CUDA_VISIBLE_DEVICES="" qmd query "test"

Solution

Add QMD_FORCE_CPU environment variable:

export QMD_FORCE_CPU=1
qmd query "test"

# Or one-time:
QMD_FORCE_CPU=1 qmd query "test"

This disables GPU acceleration and forces CPU-only mode by passing gpu: false to getLlama().

Fixes #299

On older NVIDIA architectures (e.g., Pascal), CUDA can fail with:
  [node-llama-cpp] CUDA error: the function requires an
  architectural feature absent from the device

Users previously had to run:
  CUDA_VISIBLE_DEVICES="" qmd query "test"

Now they can use:
  export QMD_FORCE_CPU=1
  qmd query "test"

Or one-time:
  QMD_FORCE_CPU=1 qmd query "test"

Fixes tobi#299
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Force cpu run on older nvidia architecture (e.g. Pascal)

1 participant