Validate provider metadata list entries instead of ignoring them (#71104)

* Validate provider metadata list entries instead of ignoring them

The items blocks for queues, plugins and task-decorators listed their field
names directly under items, where they are not JSON Schema keywords, so the
blocks were inert and any shape passed. A provider author following what those
blocks appeared to document — a dict for queues, or a path key for a decorator
— got no error and was quietly dropped by the consumers instead. Pinning each
section to the shape the provider.yaml files and generated payloads actually
use turns that into a failure at authoring time. The custom-provider howto
named the same nonexistent path field.

Raised in review of #70190.

* Keep runtime provider metadata validation permissive

The runtime schema is documented as deliberately looser than the development
one, and its validate() call is unwrapped in provider discovery, so a single
installed provider emitting an outdated entry shape would abort discovery for
every provider in that process instead of costing just that entry a logged
warning. The structural descriptions stay behind so the schema no longer
documents a shape it never actually enforced.

* Reject unknown keys in provider.yaml decorator and plugin entries

Requiring the right keys does not stop a leftover one sitting beside them, so a
stale `path:` next to a correct `class-name:` still validated — exactly the
confusion this branch set out to remove. Every sibling item block in this
authoring schema already closes itself off this way.

Author: Lee-W

airflow-core/src/airflow/provider.yaml.schema.json

@@ -565,12 +565,22 @@
             "type": "array",
             "description": "Decorators to use with the TaskFlow API. Can be accessed by users via '@task.<name>'",
             "items": {
-                "name": {
-                    "type": "string"
+                "type": "object",
+                "properties": {
+                    "name": {
+                        "description": "Name the decorator is exposed under, following '@task.'",
+                        "type": "string"
+                    },
+                    "class-name": {
+                        "description": "Class name that implements the decorator",
+                        "type": "string"
+                    }
                 },
-                "path": {
-                    "type": "string"
-                }
+                "additionalProperties": false,
+                "required": [
+                    "name",
+                    "class-name"
+                ]
             }
         },
         "secrets-backends": {
@@ -688,24 +698,29 @@
             "type": "array",
             "description": "Plugins exposed by the provider",
             "items": {
-                "name": {
-                    "type": "string"
+                "type": "object",
+                "properties": {
+                    "name": {
+                        "description": "Name of the plugin",
+                        "type": "string"
+                    },
+                    "plugin-class": {
+                        "description": "Class name that implements the plugin",
+                        "type": "string"
+                    }
                 },
-                "plugin-class": {
-                    "type": "string"
-                }
+                "additionalProperties": false,
+                "required": [
+                    "name",
+                    "plugin-class"
+                ]
             }
         },
         "queues": {
             "type": "array",
-            "description": "Message Queues exposed by the provider",
+            "description": "Message queue provider class names",
             "items": {
-                "name": {
-                    "type": "string"
-                },
-                "message-queue-class": {
-                    "type": "string"
-                }
+                "type": "string"
             }
         },
         "source-date-epoch": {

airflow-core/src/airflow/provider_info.schema.json

@@ -432,39 +432,39 @@
             "type": "array",
             "description": "Apply custom decorators to the TaskFlow API. Can be accessed by users via '@task.<name>'",
             "items": {
-                "name": {
-                    "type": "string"
-                },
-                "path": {
-                    "type": "string"
+                "type": "object",
+                "properties": {
+                    "name": {
+                        "description": "Name the decorator is exposed under, following '@task.'",
+                        "type": "string"
+                    },
+                    "class-name": {
+                        "description": "Class name that implements the decorator",
+                        "type": "string"
+                    }
                 }
             }
         },
         "plugins": {
             "type": "array",
             "description": "Plugins provided by the provider",
             "items": {
-                "name": {
-                    "type": "string",
-                    "description": "Name of the plugin"
-                },
-                "plugin-class": {
-                    "type": "string",
-                    "description": "Class to instantiate the plugin"
+                "type": "object",
+                "properties": {
+                    "name": {
+                        "type": "string",
+                        "description": "Name of the plugin"
+                    },
+                    "plugin-class": {
+                        "type": "string",
+                        "description": "Class to instantiate the plugin"
+                    }
                 }
             }
         },
         "queues": {
             "type": "array",
-            "description": "Message Queues exposed by the provider",
-            "items": {
-                "name": {
-                    "type": "string"
-                },
-                "message-queue-class": {
-                    "type": "string"
-                }
-            }
+            "description": "Message queue provider class names"
         }
     },
     "definitions": {

providers-summary-docs/howto/create-custom-providers.rst

@@ -114,7 +114,7 @@ Exposing customized functionality to the Airflow's core:
 * ``sensors`` - this field should contain the list of all the sensor class names that the
   provider provides. See :doc:`apache-airflow:core-concepts/sensors` for description of the sensors.
 
-* ``task-decorators`` - this field should contain the list of dictionaries of name/path where the decorators
+* ``task-decorators`` - this field should contain the list of dictionaries of name/class-name where the decorators
   are available. See :doc:`apache-airflow:howto/create-custom-decorator` for description of how to add
   custom decorators.