Skip to content

MCP spec conformance: 10 requirement(s) violated (via @hasmcp/mcp-spec-test) — spec 2026-07-28 #1306

Description

@hasmcp-dev

When run over stdio with npx @apify/actors-mcp-server and no APIFY_TOKEN set, the server prints APIFY_TOKEN is required but not set... to stderr and exits immediately, before ever completing an MCP handshake. Because of this, @hasmcp/mcp-spec-test's conformance suite (which starts the server as a plain stdio child process, with no credentials configured) can't get past initialize/server/discover at all, and 10 requirement checks fail as a result — mostly cascading consequences of "no session was ever established" rather than 10 independent protocol bugs. This may well be intentional behavior for a hosted/credentialed service rather than a spec bug, but flagging it in case a "no-token" mode (e.g. limited to search-actors/fetch-actor-details, which the README describes as free/unauthenticated) is meant to be reachable via stdio too.

Conformance report

MCP 2026-07-28 conformance report
target npx -y @apify/actors-mcp-server
transport stdio
supported 2026-07-28, 2025-11-25

FAILED (10) — the server deviates from the spec here

server/discover
✗ server/discover is answered without a session or handshake
expected 200, got 0: {"_raw":"no response within 10000ms"} 0 !== 200
✗ server/discover advertises the versions the server can serve
Cannot read properties of undefined (reading 'supportedVersions')
✗ server/discover is a CacheableResult with usable cache hints
Cannot read properties of undefined (reading 'ttlMs')
✗ server/discover reports server identity and capabilities
Cannot read properties of undefined (reading 'capabilities')
✗ server/discover is stable across calls within its own TTL
Cannot read properties of undefined (reading 'supportedVersions')
✗ server/discover advertises a revision this suite supports
Cannot read properties of undefined (reading 'supportedVersions')

Version negotiation
✗ a request with no version at all is served on the default
a version-less request must be served, got 0: {"_raw":"server exited"} 0 !== 200

Official SDK interop
✗ a stock official-SDK client completes the handshake
MCP error -32000: Connection closed
✗ the handshake settles on a revision inside the supported window
initialize must return a protocolVersion, got undefined
✗ a stock official-SDK client can list tools
MCP error -32000: Connection closed

NOT VERIFIED (22) — skipped; a skip is not a pass

Capability methods
– tools/list returns schema-conformant tools
– tools/call on an unknown tool is an error, not a crash
– tools/call returns a schema-conformant CallToolResult
– prompts/list returns schema-conformant prompts
– prompts/get returns messages with a role and content
– resources/list returns schema-conformant resources
– resources/templates/list returns schema-conformant templates
– resources/read returns contents for every sampled resource
– resources/read on an unknown uri is an error
– following nextCursor terminates and does not repeat a page
– an invalid pagination cursor is rejected (SHOULD)
cannot determine what the target supports — server/discover returned no result (status 0)

Version negotiation
– a version declared in _meta is accepted
– an unsupported version is rejected with the supported list
– clientInfo is optional (SHOULD, not MUST)
cannot determine what the target supports — server/discover returned no result (status 0)

Result envelope
– every result carries the required resultType
– cacheable list results carry the schema-required cache hints
– results identify the server in _meta
– a client on an older version receives no newer-revision fields
cannot determine what the target supports — server/discover returned no result (status 0)

subscriptions/listen
– subscriptions/listen acknowledges only the opted-in notification types
– the acknowledgment carries the subscription id for correlation
– a listen requesting no notification types is not a subscription to everything
– a cancelled subscription ends with a conformant teardown result, if it sends one
cannot determine what the target supports — server/discover returned no result (status 0)

PASSED (4)

server/discover
✓ the suite is reading a schema that matches the features it selected

Result envelope
✓ schema sanity: the envelope fields match the features selected

Official SDK interop
✓ the official SDK does not yet implement the newest revision

subscriptions/listen
✓ schema sanity: SubscriptionsListenResult requires _meta and resultType

Summary
4 passed 10 failed 22 not verified
36 cases applied, in 26766ms

Verdict: not conformant — 10 requirements violated.

How to reproduce

With Docker:

docker pull hasmcp/mcp-spec-test:latest
docker run --rm hasmcp/mcp-spec-test -c "npx -y @apify/actors-mcp-server" --spec-version 2026-07-28

(or, if that image lacks a manifest for your platform: git clone https://github.com/hasmcp/mcp-spec-test && cd mcp-spec-test && docker build -t mcp-spec-test . && docker run --rm mcp-spec-test -c "npx -y @apify/actors-mcp-server" --spec-version 2026-07-28)

Without Docker (npx):

npx @hasmcp/mcp-spec-test@latest -c "npx -y @apify/actors-mcp-server" --spec-version 2026-07-28

This issue was generated by an automated MCP conformance sweep across public MCP server repos. If this is expected behavior for an unauthenticated/no-token environment (rather than a genuine protocol bug), feel free to close it — it may simply reflect the test harness not providing an APIFY_TOKEN.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions