Skip to content

[Bugfix][Core] Fix xgrammar import failure on unsupported platforms #27148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
+28 −3
Closed

[Bugfix][Core] Fix xgrammar import failure on unsupported platforms #27148

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion vllm/v1/engine/processor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,16 @@
from vllm.v1.structured_output.backend_outlines import (
validate_structured_output_request_outlines,
)
from vllm.v1.structured_output.backend_xgrammar import validate_xgrammar_grammar
from vllm.v1.structured_output.utils import is_xgrammar_supported

if is_xgrammar_supported():
from vllm.v1.structured_output.backend_xgrammar import validate_xgrammar_grammar
else:
def validate_xgrammar_grammar(params):
raise ValueError(

Check failure on line 38 in vllm/v1/engine/processor.py

View workflow job for this annotation

GitHub Actions / pre-commit 

All conditional function variants must have identical signatures  [misc]

Check failure on line 38 in vllm/v1/engine/processor.py

View workflow job for this annotation

GitHub Actions / pre-commit 

All conditional function variants must have identical signatures  [misc]

Check failure on line 38 in vllm/v1/engine/processor.py

View workflow job for this annotation

GitHub Actions / pre-commit 

All conditional function variants must have identical signatures  [misc]

Check failure on line 38 in vllm/v1/engine/processor.py

View workflow job for this annotation

GitHub Actions / pre-commit 

All conditional function variants must have identical signatures  [misc]
"xgrammar is not supported on this platform. "
"Please use a different backend."
)

logger = init_logger(__name__)

Expand Down
12 changes: 11 additions & 1 deletion vllm/v1/structured_output/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,17 @@
StructuredOutputBackend,
StructuredOutputGrammar,
)
from vllm.v1.structured_output.backend_xgrammar import XgrammarBackend
from vllm.v1.structured_output.utils import is_xgrammar_supported

if is_xgrammar_supported():
from vllm.v1.structured_output.backend_xgrammar import XgrammarBackend
else:
class XgrammarBackend:
def __init__(self, *args, **kwargs):

Check failure on line 23 in vllm/v1/structured_output/__init__.py

View workflow job for this annotation

GitHub Actions / pre-commit 

Name "XgrammarBackend" already defined (possibly by an import)  [no-redef]

Check failure on line 23 in vllm/v1/structured_output/__init__.py

View workflow job for this annotation

GitHub Actions / pre-commit 

Name "XgrammarBackend" already defined (possibly by an import)  [no-redef]

Check failure on line 23 in vllm/v1/structured_output/__init__.py

View workflow job for this annotation

GitHub Actions / pre-commit 

Name "XgrammarBackend" already defined (possibly by an import)  [no-redef]

Check failure on line 23 in vllm/v1/structured_output/__init__.py

View workflow job for this annotation

GitHub Actions / pre-commit 

Name "XgrammarBackend" already defined (possibly by an import)  [no-redef]
raise ValueError(
"xgrammar is not supported on this platform. "
"Cannot initialize XgrammarBackend."
)
Comment on lines +23 to +27
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The ValueError raised here appears to be unreachable. The validation logic in Processor._validate_structured_output should prevent XgrammarBackend from being instantiated on an unsupported platform by either rejecting the request (if xgrammar is explicitly requested) or falling back to another backend (in auto mode).

If this code path is indeed unreachable, raising NotImplementedError would be more appropriate than ValueError to signal a programming error if it's ever triggered. This makes the code more robust and self-documenting. A comment explaining why this path is unexpected would also be beneficial.

        def __init__(self, *args, **kwargs):
            # This path should be unreachable. The validation in
            # `Processor._validate_structured_output` should prevent
            # `XgrammarBackend` from being initialized on unsupported platforms.
            raise NotImplementedError(
                "xgrammar is not supported on this platform and should not be "
                "initialized."
            )


if TYPE_CHECKING:
import numpy as np
Expand Down
8 changes: 7 additions & 1 deletion vllm/v1/structured_output/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

import vllm.envs as envs
from vllm.logger import init_logger
from vllm.platforms import CpuArchEnum, current_platform
from vllm.utils.import_utils import LazyLoader

if TYPE_CHECKING:
Expand Down Expand Up @@ -182,7 +183,7 @@ def get_outlines_cache():


re_llama_byte_token = re.compile(r"^<0x[0-9A-F]{2}>$")
re_replacement_seq = re.compile(r"^.{0,6}�+.{0,6}$")
re_replacement_seq = re.compile(r"^.{0,6}\ufffd+.{0,6}$")


def _reduced_vocabulary(
Expand Down Expand Up @@ -458,3 +459,8 @@ def escape_ebnf_string(s: str) -> str:
escaped_choices = (escape_ebnf_string(c) for c in choice)
grammar = "root ::= " + " | ".join(f'"{c}"' for c in escaped_choices)
return grammar


def is_xgrammar_supported() -> bool:
arch = current_platform.get_cpu_architecture()
return arch in (CpuArchEnum.X86, CpuArchEnum.ARM)
Loading