OCI registry support for Bicep extension publishing (#18956)

# OCI Registry Support for Bicep Extension Publishing

## Description

Adds support for publishing and restoring Bicep modules and extensions
to/from non-Azure OCI-compliant container registries (GHCR, Docker Hub,
etc.).

Previously, `publish`, `restore`, and `publish-extension` only worked
with Azure Container Registry (ACR) because the transport was hard-coded
to the Azure SDK's `ContainerRegistryContentClient`. This change
introduces an alternative transport built on [ORAS (OCI Registry As
Storage)](https://oras.land/), so Bicep artifacts can be stored in any
compliant registry.

Related to [#4884](#4884).

---

## User Experience

### Feature flag

The feature is gated behind an experimental flag called `ociEnabled`.
Either of the following enables it:

1. CLI flag — pass `--oci-enabled` before the subcommand:

```bash
bicep --oci-enabled publish myModule.bicep \
  --target 'br:ghcr.io/myorg/bicep/modules/my-module:v1.0'
```

2. `bicepconfig.json`:

```json
{
  "experimentalFeaturesEnabled": {
    "ociEnabled": true
  }
}
```

### Authentication

When the flag is enabled and the target host is not an Azure-managed
registry, Bicep authenticates using Docker credentials:

1. Docker credential helpers — reads `~/.docker/config.json` and invokes
the configured `credsStore` or per-registry `credHelpers` (e.g.
`docker-credential-desktop`, `docker-credential-ecr-login`).
2. Static `auth` entries — falls back to base64-encoded
`username:password` entries in the `auths` section of
`~/.docker/config.json`.

ACR (`*.azurecr.io`, `*.azurecr.cn`, `*.azurecr.us`, `*.azurecr.de`,
`*.azurecr.gov`) and `mcr.microsoft.com` continue to use the existing
Azure SDK auth path regardless of the flag.

### Example: publish a module to GHCR

```bash
# Log in to GHCR (one-time, stores creds in Docker config)
echo $GITHUB_TOKEN | docker login ghcr.io -u USERNAME --password-stdin

# Publish
bicep --oci-enabled publish ./main.bicep \
  --target 'br:ghcr.io/myorg/bicep/modules/network:v1.0'

# Reference in another file:
#   module vnet 'br:ghcr.io/myorg/bicep/modules/network:v1.0' = { ... }

# Restore
bicep --oci-enabled restore ./consumer.bicep
```

### Example: publish an extension

```bash
bicep --oci-enabled publish-extension \
  --target 'br:myregistry.example.com/bicep/extensions/my-ext:v1.0' \
  --index-file ./out/index.json
```

---

## Design

### Session API

All push/pull/resolve operations go through a single `IRegistrySession`
interface:

- `PushAsync` — pushes config, layers, and manifest
- `PullAsync` — fetches manifest and layers, returns an
`OciArtifactResult`
- `ResolveAsync` — resolves a reference to `(digest, OciManifest)`

Two implementations:

| Session | Wraps | Used for |
|---|---|---|
| `AcrRegistrySession` | `AzureContainerRegistryManager` (Azure SDK) |
ACR / `mcr.microsoft.com` always; non-ACR hosts when `ociEnabled` is off
(preserves pre-flag behavior) |
| `OrasRegistrySession` |
[oras-dotnet](https://github.com/oras-project/oras-dotnet) | Non-ACR
hosts when `ociEnabled` is on |

Sessions are created by
`OciRegistryTransportFactory.CreateSession(reference, cloud)`, which
inspects the registry hostname (and the `ociEnabled` feature flag for
non-ACR hosts) to pick the implementation. There's no provider/strategy
plumbing — the routing is a single inline check.

### Transport vs. session

`IOciRegistryTransport` (singleton) is reserved for catalog enumeration
(`GetRepositoryNamesAsync`, `GetRepositoryTagsAsync`) used by
`PrivateAcrModuleMetadataProvider`. It always resolves to
`AzureContainerRegistryManager` because catalog APIs are ACR-specific
today.

`IRegistrySession` (per-call, scoped to a `(registry, repository,
credentials)` tuple) handles per-artifact push/pull/resolve. This split
keeps a clear separation between "list what's in the registry"
(singleton, ACR-only) and "act on a specific artifact" (per-reference,
transport-agnostic).

### Credentials

For non-ACR hosts, `DockerCredentialProvider` is injected directly into
`OrasRegistrySession`. It implements oras-dotnet's `ICredentialProvider`
and the [Docker credential helper
protocol](https://docs.docker.com/engine/reference/commandline/login/#credential-helpers)
— sends the registry hostname to `docker-credential-<helper> get` via
stdin and parses the JSON response. Supports username/password and
identity-token auth.

ACR credentials continue to flow through the Azure SDK's existing token
chain inside `AzureContainerRegistryManager`.

### Registry routing

Hosts matching `*.azurecr.{io,cn,us,de,gov}` or `mcr.microsoft.com` are
classified as Azure-SDK hosts
(`OciRegistryTransportFactory.IsAzureSdkHost`) and always route through
`AcrRegistrySession`. All other hosts route through
`OrasRegistrySession` when `ociEnabled` is on, and fall back to
`AcrRegistrySession` otherwise (preserving the pre-PR
anonymous-then-authenticated Azure SDK flow for non-ACR hosts).

ACR registries behind custom domains will fall through to the generic
ORAS path, which works via Docker credential helpers (e.g.
`docker-credential-acr-env`). A future improvement could use OCI auth
challenge detection to identify the backing provider regardless of
hostname.

### New dependency

Project reference to
[oras-dotnet](https://github.com/oras-project/oras-dotnet) for the
generic OCI transport.

---

## What's unchanged

- ACR workflows — still use the Azure SDK path, no behavioral changes.
- Module reference syntax — same `br:registry/path:tag` format for any
registry.
- Local artifact cache — non-Azure artifacts are cached the same way as
ACR artifacts.

---

## Checklist

- [x] I have read and adhere to the [contribution
guide](https://github.com/Azure/bicep/blob/main/CONTRIBUTING.md).


###### Microsoft Reviewers: [Open in
CodeFlow](https://microsoft.github.io/open-pr/?codeflow=https://github.com/Azure/bicep/pull/18956)

---------

Signed-off-by: willdavsmith <willdavsmith@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: willdavsmith

docs/experimental-features.md

@@ -30,6 +30,14 @@ Enables Bicep to run deployments locally, so that you can run Bicep extensions w
 Moves defining extension configurations to the module level rather than from within a template. The feature also
 includes enhancements for Deployment stacks extensibility integration. This feature is not ready for use.
 
+### `ociEnabled`
+
+Enables publishing and restoring Bicep modules and extensions from non-Azure OCI-compliant registries (e.g. GHCR, Docker Hub) using the ORAS transport.
+
+For security, Bicep only connects (and sends credentials) to registries on a trusted allowlist. Azure Container Registry (`*.azurecr.io`/`.cn`/`.us`), `mcr.microsoft.com`, `mcr.azure.cn`, and `ghcr.io` are trusted by default. To allow additional registries, set the `BICEP_TRUSTED_REGISTRIES` environment variable to a comma-separated list of hostnames or `*.suffix` wildcards (e.g. `harbor.contoso.com,*.contoso.io`). References to untrusted registries are rejected before any connection is made.
+
+When using ambient `DOCKER_USERNAME`/`DOCKER_PASSWORD` credentials, also set `DOCKER_REGISTRY` to the hostname those credentials belong to; they are only sent to a matching registry, never to arbitrary hosts. Per-registry credentials in your Docker `config.json` (`auths`/`credHelpers`/`credsStore`) continue to be matched by host.
+
 ### `resourceInfoCodegen`
 
 Enables the 'resourceInfo' function for simplified code generation.

src/Bicep.Cli.IntegrationTests/PublishExtensionCommandTests.cs

@@ -146,7 +146,7 @@ public async Task Publish_extension_should_fail_for_malformed_index()
         var indexPath = FileHelper.SaveResultFile(TestContext, "index.json", "malformed", outputDirectory);
 
         var result = await Bicep(InvocationSettings.Default, "publish-extension", indexPath, "--target", $"br:example.com/test/extension:0.0.1");
-        result.Should().Fail().And.HaveStderrMatch("*Extension package creation failed: 'm' is an invalid start of a value.*");
+        result.Should().Fail().And.HaveStderrMatch("*Extension package creation failed:*'m' is an invalid start of a value.*");
     }
 
     [TestMethod]

src/Bicep.Cli/Commands/PublishExtensionCommand.cs

@@ -63,6 +63,8 @@ public async Task<int> RunAsync(PublishExtensionArguments args, CancellationToke
             var reference = ValidateReference(args.TargetExtensionReference);
             var overwriteIfExists = args.Force;
 
+            Trace.WriteLine($"Preparing to publish extension \"{reference.FullyQualifiedReference}\" (force={overwriteIfExists}, indexFile={args.IndexFile ?? "<none>"}, binariesSpecified={args.Binaries.Count}).");
+
             var binaries = SupportedArchitectures.All.Select(TryGetBinary).WhereNotNull().ToImmutableArray();
             var tarPayload = await GetTypesTarPayload(args, binaries, cancellationToken);
 
@@ -82,6 +84,7 @@ private async Task<BinaryData> GetTypesTarPayload(PublishExtensionArguments args
                 var indexUri = inputOutputArgumentsResolver.PathToUri(args.IndexFile);
                 var indexFile = fileExplorer.GetFile(indexUri);
 
+                Trace.WriteLine($"Packaging extension types from index file \"{indexUri.Path}\".");
                 return await CreateTypesTar(indexFile);
             }
 
@@ -101,6 +104,7 @@ private async Task<BinaryData> GetTypesTarPayload(PublishExtensionArguments args
                 throw new BicepException($"Failed to load type information: Unable to find a binary for the current architecture ({architecture.Name}).");
             }
 
+            Trace.WriteLine($"Extracting extension types from binary \"{binaryUri.Path}\" for architecture \"{architecture.Name}\".");
             var indexHandle = await GetTypesFromExtension(binaryUri, cancellationToken);
             return await CreateTypesTar(indexHandle);
         }
@@ -114,7 +118,9 @@ private async Task PublishExtensionAsync(ArtifactReference target, ExtensionPack
                 {
                     throw new BicepException($"The extension \"{target.FullyQualifiedReference}\" already exists. Use --force to overwrite the existing extension.");
                 }
+                Trace.WriteLine($"Publishing extension package to \"{target.FullyQualifiedReference}\".");
                 await moduleDispatcher.PublishExtension(target, package);
+                Trace.WriteLine($"Successfully published extension package to \"{target.FullyQualifiedReference}\".");
             }
             catch (ExternalArtifactException exception)
             {
@@ -169,12 +175,14 @@ private static void ValidateExtension(BinaryData extension)
             }
         }
 
-        private static async Task<BinaryData> CreateTypesTar(IFileHandle indexHandle)
+        private async Task<BinaryData> CreateTypesTar(IFileHandle indexHandle)
         {
             try
             {
+                Trace.WriteLine($"Bundling extension types from index \"{indexHandle.Uri.Path}\".");
                 var tarPayload = await TypesV1Archive.PackIntoBinaryData(indexHandle);
                 ValidateExtension(tarPayload);
+                Trace.WriteLine($"Successfully bundled extension types from index \"{indexHandle.Uri.Path}\".");
 
                 return tarPayload;
             }
@@ -188,7 +196,9 @@ private async Task<IFileHandle> GetTypesFromExtension(IOUri binaryUri, Cancellat
         {
             await using var extension = await localExtensionFactory.Start(binaryUri);
 
+            Trace.WriteLine($"Requesting extension type definitions from binary \"{binaryUri.Path}\" via gRPC.");
             var typeFiles = await extension.GetTypeFiles(cancellationToken);
+            Trace.WriteLine($"Received extension type index and {typeFiles.TypeFileContents.Count} additional type file(s) from binary \"{binaryUri.Path}\".");
 
             var fileExplorer = new InMemoryFileExplorer();
 

src/Bicep.Core.UnitTests/BicepTestConstants.cs

@@ -14,6 +14,8 @@
 using Bicep.Core.Registry;
 using Bicep.Core.Registry.Catalog;
 using Bicep.Core.Registry.Oci;
+using Bicep.Core.Registry.Oci.Oras;
+using Bicep.Core.Registry.Sessions;
 using Bicep.Core.Semantics.Namespaces;
 using Bicep.Core.SourceGraph;
 using Bicep.Core.Syntax;
@@ -83,8 +85,15 @@ public static class BicepTestConstants
 
         public static readonly IServiceProvider EmptyServiceProvider = new Mock<IServiceProvider>(MockBehavior.Loose).Object;
 
-        public static IArtifactRegistryProvider CreateRegistryProvider(IServiceProvider services) =>
-            new DefaultArtifactRegistryProvider(TestRegistryConfiguration, services.GetRequiredService<IPublicModuleMetadataProvider>(), ClientFactory, TemplateSpecRepositoryFactory, FileExplorer);
+        public static IArtifactRegistryProvider CreateRegistryProvider(IServiceProvider services)
+        {
+            var transport = new AzureContainerRegistryManager(ClientFactory);
+            var dockerCredentials = new DockerCredentialProvider(TestEnvironment.Default, new System.IO.Abstractions.TestingHelpers.MockFileSystem());
+            var transportFactory = new OciRegistryTransportFactory(transport, dockerCredentials);
+            var publicMetadataProvider = (services.GetService(typeof(IPublicModuleMetadataProvider)) as IPublicModuleMetadataProvider)
+                ?? StrictMock.Of<IPublicModuleMetadataProvider>().Object;
+            return new DefaultArtifactRegistryProvider(TestRegistryConfiguration, transportFactory, publicMetadataProvider, TemplateSpecRepositoryFactory, FileExplorer);
+        }
 
         public static readonly RegistryConfiguration TestRegistryConfiguration = new(PermitUntrustedRegistries: true);
 

src/Bicep.Core.UnitTests/Configuration/ConfigurationManagerTests.cs

@@ -102,6 +102,7 @@ public void GetBuiltInConfiguration_NoParameter_ReturnsBuiltInConfigurationWithA
         },
         "experimentalFeaturesWarning": true,
         "experimentalFeaturesEnabled": {
+          "ociEnabled": false,
           "symbolicNameCodegen": false,
           "moduleExtensionConfigs": false,
           "resourceTypedParamsAndOutputs": false,
@@ -186,6 +187,7 @@ public void GetBuiltInConfiguration_DisableAllAnalyzers_ReturnsBuiltInConfigurat
         "analyzers": {},
         "experimentalFeaturesWarning": true,
         "experimentalFeaturesEnabled": {
+          "ociEnabled": false,
           "symbolicNameCodegen": false,
           "resourceTypedParamsAndOutputs": false,
           "sourceMapping": false,
@@ -292,6 +294,7 @@ public void GetBuiltInConfiguration_DisableAnalyzers_ReturnsBuiltInConfiguration
         },
         "experimentalFeaturesWarning": true,
         "experimentalFeaturesEnabled": {
+          "ociEnabled": false,
           "symbolicNameCodegen": false,
           "resourceTypedParamsAndOutputs": false,
           "sourceMapping": false,
@@ -376,6 +379,7 @@ public void GetBuiltInConfiguration_EnableExperimentalFeature_ReturnsBuiltInConf
             var configuration = IConfigurationManager.GetBuiltInConfiguration();
 
             ExperimentalFeaturesEnabled experimentalFeaturesEnabled = new(
+                OciEnabled: false,
                 SymbolicNameCodegen: false,
                 ResourceTypedParamsAndOutputs: false,
                 SourceMapping: false,
@@ -461,6 +465,7 @@ public void GetBuiltInConfiguration_EnableExperimentalFeature_ReturnsBuiltInConf
             },
             "experimentalFeaturesWarning": true,
             "experimentalFeaturesEnabled": {
+                "ociEnabled": false,
                 "symbolicNameCodegen": false,
                 "resourceTypedParamsAndOutputs": false,
                 "sourceMapping": false,
@@ -709,7 +714,9 @@ public void GetConfiguration_ValidCustomConfiguration_OverridesBuiltInConfigurat
                       },
                       "cacheRootDirectory": "/home/username/.bicep/cache",
                       "experimentalFeaturesWarning": false,
-                      "experimentalFeaturesEnabled": {},
+                      "experimentalFeaturesEnabled": {
+                        "ociEnabled": false
+                      },
                       "formatting": {
                         "indentKind": "Space",
                         "newlineKind": "LF",
@@ -813,6 +820,7 @@ public void GetConfiguration_ValidCustomConfiguration_OverridesBuiltInConfigurat
                   "cacheRootDirectory": "/home/username/.bicep/cache",
                   "experimentalFeaturesWarning": false,
                   "experimentalFeaturesEnabled": {
+                    "ociEnabled": false,
                     "symbolicNameCodegen": false,
                     "resourceTypedParamsAndOutputs": false,
                     "sourceMapping": false,

src/Bicep.Core.UnitTests/Features/FeatureProviderOverrides.cs

@@ -10,6 +10,7 @@ namespace Bicep.Core.UnitTests.Features;
 public record FeatureProviderOverrides(
     IDirectoryHandle? CacheRootDirectory = null,
     bool? RegistryEnabled = default,
+    bool? OciEnabled = default,
     bool? SymbolicNameCodegenEnabled = default,
     bool? AdvancedListComprehensionEnabled = default,
     bool? ResourceTypedParamsAndOutputsEnabled = default,
@@ -28,6 +29,7 @@ public record FeatureProviderOverrides(
     public FeatureProviderOverrides(
         TestContext testContext,
         bool? RegistryEnabled = default,
+        bool? OciEnabled = default,
         bool? SymbolicNameCodegenEnabled = default,
         bool? AdvancedListComprehensionEnabled = default,
         bool? ResourceTypedParamsAndOutputsEnabled = default,
@@ -44,6 +46,7 @@ public FeatureProviderOverrides(
         bool? DeployCommandsEnabled = default) : this(
             FileHelper.GetCacheRootDirectory(testContext),
             RegistryEnabled,
+            OciEnabled,
             SymbolicNameCodegenEnabled,
             AdvancedListComprehensionEnabled,
             ResourceTypedParamsAndOutputsEnabled,

src/Bicep.Core.UnitTests/Features/FeatureProviderTests.cs

@@ -1,6 +1,7 @@
 // Copyright (c) Microsoft Corporation.
 // Licensed under the MIT License.
 
+using System;
 using System.Diagnostics.CodeAnalysis;
 using System.IO.Abstractions.TestingHelpers;
 using Bicep.Core.Configuration;

src/Bicep.Core.UnitTests/Features/OverriddenFeatureProvider.cs

@@ -21,6 +21,8 @@ public OverriddenFeatureProvider(IFeatureProvider features, FeatureProviderOverr
 
     public IDirectoryHandle CacheRootDirectory => overrides.CacheRootDirectory ?? features.CacheRootDirectory;
 
+    public bool OciEnabled => overrides.OciEnabled ?? features.OciEnabled;
+
     public bool SymbolicNameCodegenEnabled => overrides.SymbolicNameCodegenEnabled ?? features.SymbolicNameCodegenEnabled;
 
     public bool ResourceTypedParamsAndOutputsEnabled => overrides.ResourceTypedParamsAndOutputsEnabled ?? features.ResourceTypedParamsAndOutputsEnabled;

src/Bicep.Core.UnitTests/Mock/Registry/RegistryCatalogMocks.cs

@@ -14,6 +14,7 @@
 using Bicep.Core.Registry.Catalog;
 using Bicep.Core.Registry.Catalog.Implementation;
 using Bicep.Core.Registry.Catalog.Implementation.PrivateRegistries;
+using Bicep.Core.Registry.Oci;
 using FluentAssertions;
 using Microsoft.WindowsAzure.ResourceStack.Common.Extensions;
 using Moq;
@@ -120,21 +121,21 @@ params Mock<IRegistryModuleMetadataProvider>[] privateProviders
         var privateFactory = StrictMock.Of<IPrivateAcrModuleMetadataProviderFactory>();
 
         // Default - when an unrecognized registry is requested, return a provider that fails to load (similar to real behavior)
-        privateFactory.Setup(x => x.Create(It.IsAny<CloudConfiguration>(), It.IsAny<string>(), It.IsAny<IContainerRegistryClientFactory>()))
-            .Returns((CloudConfiguration _, string registry, IContainerRegistryClientFactory _) =>
+        privateFactory.Setup(x => x.Create(It.IsAny<CloudConfiguration>(), It.IsAny<string>(), It.IsAny<IOciRegistryTransportFactory>()))
+            .Returns((CloudConfiguration _, string registry, IOciRegistryTransportFactory _) =>
                 MockFailingPrivateMetadataProvider(registry, new Exception($"Registry {registry} not found in mock")).Object);
 
         foreach (var privateProvider in privateProviders)
         {
             privateProvider.Object.Registry.Should().NotBe(PublicRegistry);
-            privateFactory.Setup(x => x.Create(It.IsAny<CloudConfiguration>(), privateProvider.Object.Registry, It.IsAny<IContainerRegistryClientFactory>()))
+            privateFactory.Setup(x => x.Create(It.IsAny<CloudConfiguration>(), privateProvider.Object.Registry, It.IsAny<IOciRegistryTransportFactory>()))
                 .Returns(privateProvider.Object);
         }
 
         var indexer = new RegistryModuleCatalog(
             publicProvider.Object,
             privateFactory.Object,
-            StrictMock.Of<IContainerRegistryClientFactory>().Object,
+            StrictMock.Of<IOciRegistryTransportFactory>().Object,
             BicepTestConstants.BuiltInOnlyConfigurationManager);
 
         return indexer;