Skip to content

MCP spec conformance: 5 requirement(s) violated (via @hasmcp/mcp-spec-test) — spec 2025-11-25 #1307

Description

@hasmcp-dev

When run over stdio with npx @apify/actors-mcp-server and no APIFY_TOKEN set, the server prints APIFY_TOKEN is required but not set... to stderr and exits immediately, before ever completing an MCP handshake. Testing specifically against the older 2025-11-25 spec revision (this server negotiates both 2026-07-28 and 2025-11-25), @hasmcp/mcp-spec-test finds 5 requirement violations — fewer than under 2026-07-28 because this revision's suite doesn't have a separate server/discover pre-handshake check, but the core issue is the same: no session is ever established because the process exits at startup. This is likely an environment/credentials artifact rather than an independent protocol bug for this revision, but is reported separately since the failing-case set differs from the 2026-07-28 run.

Conformance report

MCP 2025-11-25 conformance report
target npx -y @apify/actors-mcp-server
transport stdio
supported 2026-07-28, 2025-11-25

FAILED (5) — the server deviates from the spec here

Version negotiation
✗ a request with no version at all is served on the default
a version-less request must be served, got 0: {"_raw":"server exited"} 0 !== 200
✗ an unsupported version offered at the handshake is refused or downgraded, not echoed
expected either an error or a protocolVersion, got {"_raw":"server exited"}

Official SDK interop
✗ a stock official-SDK client completes the handshake
MCP error -32000: Connection closed
✗ the handshake settles on a revision inside the supported window
initialize must return a protocolVersion, got undefined
✗ a stock official-SDK client can list tools
MCP error -32000: Connection closed

NOT VERIFIED (13) — skipped; a skip is not a pass

Capability methods
– tools/list returns schema-conformant tools
– tools/call on an unknown tool is an error, not a crash
– tools/call returns a schema-conformant CallToolResult
– prompts/list returns schema-conformant prompts
– prompts/get returns messages with a role and content
– resources/list returns schema-conformant resources
– resources/templates/list returns schema-conformant templates
– resources/read returns contents for every sampled resource
– resources/read on an unknown uri is an error
– following nextCursor terminates and does not repeat a page
– an invalid pagination cursor is rejected (SHOULD)
cannot determine what the target supports — initialize returned no result (status 0)

Version negotiation
– initialize returns the schema-required fields
– the handshake settles on the revision under test
no session — initialize returned no result (status 0)

PASSED (2)

server/discover
✓ the suite is reading a schema that matches the features it selected

Official SDK interop
✓ the official SDK does not yet implement the newest revision

Summary
2 passed 5 failed 13 not verified
20 cases applied, in 24355ms

Verdict: not conformant — 5 requirements violated.

How to reproduce

With Docker:

docker pull hasmcp/mcp-spec-test:latest
docker run --rm hasmcp/mcp-spec-test -c "npx -y @apify/actors-mcp-server" --spec-version 2025-11-25

(or, if that image lacks a manifest for your platform: git clone https://github.com/hasmcp/mcp-spec-test && cd mcp-spec-test && docker build -t mcp-spec-test . && docker run --rm mcp-spec-test -c "npx -y @apify/actors-mcp-server" --spec-version 2025-11-25)

Without Docker (npx):

npx @hasmcp/mcp-spec-test@latest -c "npx -y @apify/actors-mcp-server" --spec-version 2025-11-25

This issue was generated by an automated MCP conformance sweep across public MCP server repos. If this is expected behavior for an unauthenticated/no-token environment (rather than a genuine protocol bug), feel free to close it — it may simply reflect the test harness not providing an APIFY_TOKEN.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions