Skip to content

Add Data Factory tools to Fabric MCP Server#2584

Open
Ebram-Tawfik wants to merge 19 commits intomicrosoft:mainfrom
Ebram-Tawfik:integrate-datafactory-mcp
Open

Add Data Factory tools to Fabric MCP Server#2584
Ebram-Tawfik wants to merge 19 commits intomicrosoft:mainfrom
Ebram-Tawfik:integrate-datafactory-mcp

Conversation

@Ebram-Tawfik
Copy link
Copy Markdown

@Ebram-Tawfik Ebram-Tawfik commented May 5, 2026

Summary

Adds Data Factory pipeline and dataflow management tools to the Fabric MCP Server, consuming the shared handler layer from the Microsoft.DataFactory.MCP.Core NuGet package.

New Commands

Command Description
list-pipelines List all pipelines in a workspace
create-pipeline Create a new pipeline
get-pipeline Get pipeline details by ID
run-pipeline Trigger a pipeline run
list-dataflows List all dataflows in a workspace
create-dataflow Create a new dataflow
execute-query Execute an M (Power Query) expression against a dataflow

Architecture

  • Thin GlobalCommand<T> wrappers following existing Fabric MCP patterns
  • Business logic delegated to PipelineHandler / DataflowHandler / DataflowQueryHandler from the NuGet Core package
  • Registered via DataFactoryAreaSetup : IAreaSetup in Program.cs
  • AOT-compatible with custom JsonSerializerContext

Changes

  • New project: tools/Fabric.Mcp.Tools.DataFactory/ — 7 commands, options, models, area setup
  • Program.cs: Register DataFactoryAreaSetup in area list
  • Fabric.Mcp.Server.slnx: Add project to solution
  • Directory.Packages.props: Add Microsoft.DataFactory.MCP.Core version entry (0.20.0-beta)
  • Cleanup: Removed submodule (replaced by NuGet package reference)

Copilot AI review requested due to automatic review settings May 5, 2026 19:52
@Ebram-Tawfik Ebram-Tawfik requested review from a team as code owners May 5, 2026 19:52
@github-project-automation github-project-automation Bot moved this to Untriaged in Azure MCP Server May 5, 2026
@Ebram-Tawfik Ebram-Tawfik force-pushed the integrate-datafactory-mcp branch from baefdd2 to ddb2304 Compare May 5, 2026 19:55
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds the DataFactory.MCP.Core submodule and wires it into Fabric.Mcp.Server as a project reference, along with centrally-managed package versions required to build the integrated dependency.

Changes:

  • Add DataFactory.MCP.Core project reference to Fabric.Mcp.Server
  • Add external/DataFactory.MCP git submodule configuration (pinned commit + .gitmodules)
  • Add missing central package versions needed by DataFactory.MCP.Core

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
servers/Fabric.Mcp.Server/src/Fabric.Mcp.Server.csproj Adds a project reference to the DataFactory MCP core project in the new submodule
external/DataFactory.MCP Pins the submodule to a specific commit for reproducible builds
Directory.Packages.props Adds central package versions required by the submodule project
.gitmodules Registers the DataFactory.MCP Git submodule source and path

Comment thread servers/Fabric.Mcp.Server/src/Fabric.Mcp.Server.csproj Outdated
Comment thread servers/Fabric.Mcp.Server/src/Fabric.Mcp.Server.csproj Outdated
Comment thread .gitmodules Outdated
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 5, 2026

Thank you for your contribution @Ebram-Tawfik! We will review the pull request and get back to you soon.

@Ebram-Tawfik Ebram-Tawfik force-pushed the integrate-datafactory-mcp branch from aa7af73 to d7fca50 Compare May 5, 2026 20:04
@Ebram-Tawfik Ebram-Tawfik changed the title Add DataFactory.MCP.Core project reference to Fabric.Mcp.Server Add DataFactory.MCP.Core as submodule reference in Fabric.Mcp.Server May 5, 2026
Copy link
Copy Markdown
Contributor

@jongio jongio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR introduces a git submodule to pull in DataFactory.MCP.Core. I have a few concerns that need to be addressed before merging.

CI will break: The build pipeline (eng/pipelines/templates/jobs/build.yml) uses checkout: self without submodules: recursive. Once merged, Fabric.Mcp.Server builds will fail because external/DataFactory.MCP won't exist in CI. The CONTRIBUTING.md update is helpful for local dev, but CI needs the equivalent configuration.

AOT compatibility: DataFactory.MCP.Core doesn't set IsAotCompatible=true. Fabric.Mcp.Server enables AOT/Trim analyzers with TreatWarningsAsErrors=true. Referencing a project without AOT annotations will likely produce analyzer warnings that become build-breaking errors.

Submodule vs NuGet package: Git submodules are fragile in practice - they break git clone without --recurse-submodules, complicate CI checkout steps, create merge conflicts on pointer updates, and confuse contributors. The conventional .NET approach is publishing DataFactory.MCP.Core as a NuGet package (internal feed works fine) and adding a PackageReference. This sidesteps the AOT/build-props conflicts entirely since the package arrives pre-compiled.

Dead code until follow-up: The project reference compiles DataFactory.MCP.Core into the binary (including Apache.Arrow and embedded HTML/CSS/JS resources) but nothing registers its tools via IAreaSetup. The server grows for zero functional benefit right now. Consider combining this with the registration PR so each merge leaves the server in a working incremental state.

I'd suggest either switching to a NuGet package reference, or at minimum addressing the CI and AOT issues first. Happy to discuss tradeoffs.

@github-project-automation github-project-automation Bot moved this from Untriaged to In Progress in Azure MCP Server May 6, 2026
- Add DataFactory.MCP as git submodule at external/DataFactory.MCP
- Add project reference using $(RepoRoot) for portability
- Add Apache.Arrow 22.1.0 to Directory.Packages.props (only new dependency needed)
- Microsoft.Identity.Client and Microsoft.Extensions.Http already satisfied transitively
- Build verified: 0 warnings, 0 errors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Ebram-Tawfik Ebram-Tawfik force-pushed the integrate-datafactory-mcp branch from d7fca50 to 20655d2 Compare May 6, 2026 18:50
…ory rename

Update submodule pointer and project reference to match the
renamed project in DataFactory.MCP repo.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Ebram-Tawfik Ebram-Tawfik force-pushed the integrate-datafactory-mcp branch from 6488a29 to 64ed49e Compare May 6, 2026 19:37
…Factory

Rename the external submodule path for consistency with Fabric naming
conventions. Updates .gitmodules, project reference, and docs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Ebram-Tawfik Ebram-Tawfik force-pushed the integrate-datafactory-mcp branch from 34653f1 to d84da2c Compare May 6, 2026 20:14
Ebram Tawfik and others added 2 commits May 6, 2026 14:04
The submodule now includes DataFactory.MCP.Fabric which provides
IAreaSetup + GlobalCommand<T> wrappers for direct Fabric MCP Server
integration without needing a separate wrapper project in this repo.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…module

The DataFactory.MCP.Fabric adapter project now lives in the DataFactory
repo itself (submodule), eliminating the need for a separate wrapper
project in this repo.

Changes:
- Delete tools/Fabric.Mcp.Tools.DataFactory/ (redundant wrapper)
- Add explicit ProjectReference to submodule's DataFactory.MCP.Fabric
- Update Program.cs to use DataFactory.MCP.Fabric.DataFactoryAreaSetup
- Fix submodule csproj with conditional paths for Microsoft.Mcp.Core

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@jongio jongio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the rework - the toolset structure, AOT annotations, and command patterns are solid. Two blockers remain:

  1. CI is still failing (10/15 checks): eng/pipelines/templates/jobs/build.yml uses checkout: self with no submodule initialization. The server's wildcard reference (tools\Fabric.*\src\*.csproj) picks up the new toolset, which depends on external/Fabric.Mcp.Tools.DataFactory/DataFactory.MCP.Core/DataFactory.MCP.Core.csproj - missing without git submodule update --init --recursive. Add a step before the build task or use checkout: self with submodules: recursive.

  2. No unit tests: This repo requires comprehensive unit tests for every command (Constructor, ExecuteAsync validation, deserialization, error handling, BindOptions). There are 5 commands here with zero test coverage.

Minor (non-blocking):

  • Microsoft.Mcp.slnx doesn't include the new project (IDE navigation only, builds still work via wildcard).
  • Submodule tracks fabric-mcp-tools-integration branch rather than a tagged release - worth pinning to a stable ref.

Ebram Tawfik and others added 6 commits May 6, 2026 15:35
- Copy source files from external submodule to tools/ folder
- Update namespaces from DataFactory.MCP.Fabric to Fabric.Mcp.Tools.DataFactory
- Add global:: prefix for DataFactory.MCP.Core namespace references
- Remove explicit ProjectReference to external csproj (wildcard covers it)
- Update Program.cs to use new namespace
- Add project to solution file
- Update submodule to latest commit
- All commands now delegate to PipelineHandler/DataflowHandler
- Commands are thin shims (~15 lines each)
- Handlers registered via AddDataFactoryMcpServices() in submodule
- Update submodule to latest

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace ProjectReference to submodule Core project with PackageReference
to Microsoft.DataFactory.MCP.Core 0.19.0-beta from nuget.org.
Add version entry in Directory.Packages.props for CPM compliance.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
No longer needed - consuming Microsoft.DataFactory.MCP.Core via NuGet package instead.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Ebram-Tawfik Ebram-Tawfik changed the title Add DataFactory.MCP.Core as submodule reference in Fabric.Mcp.Server Add Data Factory tools to Fabric MCP Server May 7, 2026
Ebram Tawfik and others added 3 commits May 6, 2026 21:34
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add ExecuteQueryCommand for executing M (Power Query) expressions against dataflows
- Add ExecuteQueryOptions, DataflowId/QueryName/QueryText option definitions
- Register ExecuteQueryCommand in DI and command group
- Add ExecuteQueryCommandResult to JSON source generation context
- Update Microsoft.DataFactory.MCP.Core package version to 0.19.1-beta

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Updated to version with source-generated JSON for full ILLinker
trim safety (zero IL2026 warnings).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Ebram-Tawfik Ebram-Tawfik requested a review from jongio May 7, 2026 19:30
Ebram Tawfik and others added 4 commits May 7, 2026 12:49
0.20.0-beta exists on nuget.org but hasn't propagated to the
azure-sdk-for-net DevOps feed upstream yet. Using 0.19.1-beta
to unblock CI; will bump once 0.20.0-beta is cached.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Package is now available on the azure-sdk-for-net DevOps feed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The smoke test runs 'fabmcp tools list' and parses stdout as JSON.
AuthenticationStateManager logs an info message during DI construction,
which pollutes stdout and causes ConvertFrom-Json to fail.

Match Azure.Mcp.Server pattern: LogToStandardErrorThreshold = LogLevel.Trace
sends all console logs to stderr, keeping stdout clean for JSON output.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Regenerate Microsoft.Mcp.slnx and Fabric.Mcp.Server.slnx via Update-Solution.ps1
- Fix IMPORTS ordering in DataFactoryAreaSetup.cs (dotnet format)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 10 comments.

Comment thread tools/Fabric.Mcp.Tools.DataFactory/src/DataFactoryAreaSetup.cs
Comment thread servers/Fabric.Mcp.Server/src/Program.cs
Create test project Fabric.Mcp.Tools.DataFactory.UnitTests with 45 tests:
- DataFactoryAreaSetupTests: name, title, service registration
- Command tests for all 7 commands: constructor, metadata, options, null guards
- Covers Pipeline (list, create, get, run) and Dataflow (list, create, execute-query)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@jongio jongio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addresses my previous feedback. The switch from submodule to NuGet package is the right call - CI is green across all 15 checks and the build story is much simpler.

Commands follow Fabric toolset patterns: sealed classes, primary constructors, CommandMetadata, proper option binding. Area setup registers all 7 commands correctly.

Follow-up items (non-blocking):

  • Unit tests cover constructor, metadata, options, and null guards but don't test ExecuteAsync paths. Other Fabric toolsets (OneLake) include execution success/error tests. Worth adding in a subsequent PR.

  • ExecuteQueryCommandResult.Data is typed as object?, which is tricky for source-generated System.Text.Json under AOT. Native builds pass today, but if the handler returns complex types, serialization could fail at runtime. Worth watching.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

5 participants