GET /api/shovels/ endpoints are returning empty results

[ahives]

Describe the bug

It seems that GET /api/shovels/ is not working properly on RMQ 4.0.3. I am running from a Docker image. So, when I send a request to the GET endpoints I am getting an empty result. To put this into context, I can confirm that I have both the rabbitmq_shovel and rabbitmq_shovel_management plugins (below is a snapshot of my logs in Additional Context). Also, I see both the Shovel Status and Shovel Management on the admin web page.

Reproduction steps

1. Run the docker command to create and start a docker image of RMQ 4.0.3 docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:4.0-management
2. Run the following docker command to enable the rabbitmq_shovel plugin: docker exec rabbitmq rabbitmq-plugins enable rabbitmq_shovel
3. Run the following docker command to enable the rabbitmq_shovel_management plugin: docker exec rabbitmq rabbitmq-plugins enable rabbitmq_shovel_management
4. Create a dynamic shovel by posting to endpoint api/parameters/shovel/{vhost}/test-shovel payload

{
    "value": {
        "src-protocol": "amqp091",
        "src-uri": "amqp://user1@localhost",
        "src-queue": "queue1",
        "dest-protocol": "amqp091",
        "dest-uri": "amqp://user1@localhost",
        "dest-queue": "queue2",
        "ack-mode": "on-confirm",
        "src-delete-after": "queue-length",
        "src-prefetch-count": 1000
    }
}

5. Execute a GET command to the api/shovels/ endpoint

Expected behavior

Should return a list of shovel results.

Additional context

Below are the logs associated with the proper plugins being installed on the docker image.

% docker exec rabbitmq rabbitmq-plugins enable rabbitmq_shovel

Enabling plugins on node rabbit@123:
rabbitmq_shovel
The following plugins have been configured:
rabbitmq_management
rabbitmq_management_agent
rabbitmq_prometheus
rabbitmq_shovel
rabbitmq_web_dispatch
Applying plugin configuration to rabbit@123...
The following plugins have been enabled:
rabbitmq_shovel

started 1 plugins.
% docker exec rabbitmq rabbitmq-plugins enable rabbitmq_shovel_management

Enabling plugins on node rabbit@123:
rabbitmq_shovel_management
The following plugins have been configured:
rabbitmq_management
rabbitmq_management_agent
rabbitmq_prometheus
rabbitmq_shovel
rabbitmq_shovel_management
rabbitmq_web_dispatch
Applying plugin configuration to rabbit@123...
The following plugins have been enabled:
rabbitmq_shovel_management

started 1 plugins.

[michaelklishin]

@ahives the management UI uses the same HTTP API endpoints available to everyone, there are no "private endpoints".

I cannot reproduce it with 4.1.3, and neither can the rabbitmq_shovel or rabbitmqadmin test suites, althoug they mostly use GET /api/shovels/{vhost}.

With the Default Virtual Host

rabbitmqadmin shovels delete --name "sh-1"

rabbitmqadmin shovels declare_amqp091 --name "sh-1" --source-uri "amqp://localhost" --destination-uri "amqp://localhost" --source-queue "sh-1.src.q" --destination-queue "sh-1.dest.q"

curl -s -u guest:guest -X GET http://localhost:15672/api/shovels | jq
[
  {
    "node": "rabbit@[redacted]",
    "timestamp": "2025-08-08 12:08:26",
    "name": "sh-1",
    "vhost": "/",
    "type": "dynamic",
    "pending": 0,
    "remaining": "unlimited",
    "remaining_unacked": "unlimited",
    "forwarded": 0,
    "state": "running",
    "src_uri": "amqp://localhost",
    "src_protocol": "amqp091",
    "dest_protocol": "amqp091",
    "dest_uri": "amqp://localhost",
    "src_queue": "sh-1.src.q",
    "dest_queue": "sh-1.dest.q",
    "blocked_status": "running"
  }
]

With Two Virtual Hosts

rabbitmqadmin vhosts declare --name "vh-2"

rabbitmqadmin --vhost "vh-2" shovels delete --name "sh-1"; sleep 500ms; rabbitmqadmin --vhost "vh-2" shovels declare_amqp091 --name "sh-2" --source-uri "amqp://localhost" --destination-uri "amqp://localhost" --source-queue "sh-2.src.q" --destination-queue "sh-2.dest.q"

curl -s -u guest:guest -X GET http://localhost:15672/api/shovels | jq
[
  {
    "node": "rabbit@[redacted]",
    "timestamp": "2025-08-08 12:08:26",
    "name": "sh-1",
    "vhost": "/",
    "type": "dynamic",
    "pending": 0,
    "remaining": "unlimited",
    "remaining_unacked": "unlimited",
    "forwarded": 0,
    "state": "running",
    "src_uri": "amqp://localhost",
    "src_protocol": "amqp091",
    "dest_protocol": "amqp091",
    "dest_uri": "amqp://localhost",
    "src_queue": "sh-1.src.q",
    "dest_queue": "sh-1.dest.q",
    "blocked_status": "running"
  },
  {
    "node": "rabbit@[redacted]",
    "timestamp": "2025-08-08 12:11:34",
    "name": "sh-2",
    "vhost": "vh-2",
    "type": "dynamic",
    "pending": 0,
    "remaining": "unlimited",
    "remaining_unacked": "unlimited",
    "forwarded": 0,
    "state": "running",
    "src_uri": "amqp://localhost",
    "src_protocol": "amqp091",
    "dest_protocol": "amqp091",
    "dest_uri": "amqp://localhost",
    "src_queue": "sh-2.src.q",
    "dest_queue": "sh-2.dest.q",
    "blocked_status": "running"
  }
]

rabbitmqadmin shovels list_all
┌───────────────────┬──────┬───────┬─────────┬─────────┬──────────────────┬──────────────────┬────────────┬─────────────┬────────────────┬─────────────────────┬─────────────────┬──────────────────────┐
│ node              │ name │ vhost │ type    │ state   │ source_uri       │ destination_uri  │ source     │ destination │ source_address │ destination_address │ source_protocol │ destination_protocol │
├───────────────────┼──────┼───────┼─────────┼─────────┼──────────────────┼──────────────────┼────────────┼─────────────┼────────────────┼─────────────────────┼─────────────────┼──────────────────────┤
│ rabbit@W4P56M04DC │ sh-1 │ /     │ dynamic │ running │ amqp://localhost │ amqp://localhost │ sh-1.src.q │ sh-1.dest.q │                │                     │ AMQP 0-9-1      │ AMQP 0-9-1           │
├───────────────────┼──────┼───────┼─────────┼─────────┼──────────────────┼──────────────────┼────────────┼─────────────┼────────────────┼─────────────────────┼─────────────────┼──────────────────────┤
│ rabbit@W4P56M04DC │ sh-2 │ vh-2  │ dynamic │ running │ amqp://localhost │ amqp://localhost │ sh-2.src.q │ sh-2.dest.q │                │                     │ AMQP 0-9-1      │ AMQP 0-9-1           │
└───────────────────┴──────┴───────┴─────────┴─────────┴──────────────────┴──────────────────┴────────────┴─────────────┴────────────────┴─────────────────────┴─────────────────┴──────────────────────┘

[ahives]

I'm not sure what "private endpoints" are. I was referring to the HTTP endpoints. I will try with the version suggested in your response. Thanks again.

[michaelklishin]

An API endpoint that would only be available to, say, the RabbitMQ management plugin but not any other clients.

In both open source RabbitMQ and Tanzu RabbitMQ, such endpoints do not exist :)

[michaelklishin]

RabbitMQ 4.0.x is out community support.

I have demonstrated that GET /api/shovels behaves as expected with one and multiple virtual hosts.

Shovels are scoped by virtual host, because the runtime parameters they build on are. Management UI does use the GET /api/shovels endpoint at /#/shovels, as you can easily see in the Developer Tools' Network tab, although a lot of tools use its virtual host-scoped counterpart, GET /api/shovels/{vhost}. The rabbitmq_shovel_management test suite (http_SUITE.erl), rabbitmqadmin shovels tests and the underlying HTTP API client all touch that endpoint, even if GET /api/shovels/{vhost} is used a lot more often.

I could not find anything relevant in any of the release notes since 4.0.0.

For a version out of community support, that's a very detailed response, so I'll stop here.

[ahives]

One other note, I used the exact curl command and I got [] returned as I did with Postman.

[michaelklishin]

Then you may be querying a different cluster with curl.

Open the management UI, then Developer Tools, then the Networking tab, go to the shovel management page and see that it uses GET /api/shovels. There you can also copy an exact curl equivalent of a recorded HTTP request.

Then enable HTTP API logging and see that it's the node in question that handles your curl requests. GET /api/shovels uses runtime parameters, so, cluster-wide [but also virtual host-scoped] state but it's easy to forget a running node or container locally.

Sorry but I have provided ample evidence and no one has reported anything like this in the last ≈ year (and even before that). It's on you to investigate.

[ahives]

I only have one cluster running. I was able to use the HTTP API to create the shovel and confirmed through the Management UI.

[michaelklishin]

The source used by GET /api/shovels can be inspected with (one run per virtual host):

rabbitmqadmin --vhost "/" parameters list

or

rabbitmqctl --vhost "/" -s list_parameters --formatter=pretty_table

The specific functions used by the HTTP API endpoints that list shovels are right here.

Looking at the above function suggests that one other thing it does is filtering the virtual host the requesting user has access to:

%% Allow users to see things in the vhosts they are authorised. But
%% static shovels do not have a vhost, so only allow admins (not
%% monitors) to see them.
filter_vhost_user(List, _ReqData, #context{user = User = #user{tags = Tags}}) ->
    VHosts = rabbit_mgmt_util:list_login_vhosts_names(User, undefined),
    [I || I <- List, case pget(vhost, I) of
                         undefined -> lists:member(administrator, Tags);
                         VHost     -> lists:member(VHost, VHosts)
                     end].

status(ReqData, Context) ->
    filter_vhost_user(
        lists:append([status(Node) || Node <- [node() | nodes()]]),
        ReqData, Context).

Therefore your management UI user and the one used with curl likely are different and have different permissions.

[ahives]

So, I was able to confirm that scoped endpoint /api/parameters/shovel/{vhost}/{shovel_name} is working as expected. I don't think it's a credentials issue because I am using the default guest creds. There seems to be something wrong with /api/shovels. Here is a screenshot of the scoped endpoint working.

[michaelklishin]

Below are the logs associated with the proper plugins being installed on the docker image.

I'm sorry but that's not the logs you need to be looking at.

Try

• Actual node logs, not what rabbitmq-plugins enable outputs. That's where all the exceptions will be logged, for example
• Enable HTTP API request logging