🍒 MCP & DAP Cherrypicks #11612
Open
🍒 MCP & DAP Cherrypicks #11612
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@swift-ci test |
|
@swift-ci test windows |
1 similar comment
|
@swift-ci test windows |
|
Failed were TestDAP_io.py & TestDAP_step.py |
|
@swift-ci test windows |
Reapply "[lldb] Update JSONTransport to use MainLoop for reading." (llvm#152155) This reverts commit cd40281. This also includes some updates to try to address the platforms with failing tests. I updated the JSONTransport and tests to use std::function instead of llvm:unique_function. I think the tests were failing due to the unique_function not being moved correctly in the loop on some platforms. (cherry picked from commit 45d4e84)
This is to unblock CI. Disabling the tests while I investigate the timeouts. (cherry picked from commit 3a36070)
This should reduce flakes observed in the Fuchsia AArch64 Linux LLDB CI builders. (cherry picked from commit 986d7aa)
… and transport layer. (llvm#153121) This abstracts the base Transport handler to have a MessageHandler component and allows us to generalize both JSON-RPC 2.0 for MCP (or an LSP) and DAP format. This should allow us to create clearly defined clients and servers for protocols, both for testing and for RPC between the lldb instances and an lldb-mcp multiplexer. This basic model is inspiried by the clangd/Transport.h file and the mlir/lsp-server-support/Transport.h that are both used for LSP servers within the llvm project. Additionally, this helps with testing by subclassing `Transport` to allow us to simplify sending/receiving messages without needing to use a toJSON/fromJSON and a pair of pipes, see `TestTransport` in DAP/TestBase.h. (cherry picked from commit 538bd83)
…40626 (llvm#153836) - VS Code extension: - Add module symbol table viewer using [Tabulator](https://tabulator.info/) for sorting and formatting rows. - Add context menu action to the modules tree. - lldb-dap - Add `DAPGetModuleSymbolsRequest` to get symbols from a module. Fixes llvm#140626 [Screencast From 2025-08-15 19-12-33.webm](https://github.com/user-attachments/assets/75e2f229-ac82-487c-812e-3ea33a575b70) (cherry picked from commit 8b64cd8)
Originally commited in 362b9d7 and then reverted in cb63b75. This re-lands a subset of the changes to dap_server.py/DebugCommunication and addresses the python3.10 compatibility issue. This includes less type annotations since those were the reason for the failures on that specific version of python. I've done additional testing on python3.8, python3.10 and python3.13 to further validate these changes. (cherry picked from commit 13eca52)
)" (llvm#154832) This reverts commit 0f33b90 and includes a fix for the added test that was submitted between my last update and pull. (cherry picked from commit 36d07ad)
…llvm#140626 (llvm#153836)" This reverts commit 8b64cd8. This breaks lldb-aarch64-* bots causing a crash in lldb-dap while running test TestDAP_moduleSymbols.py https://lab.llvm.org/buildbot/#/builders/59/builds/22959 https://lab.llvm.org/buildbot/#/builders/141/builds/10975 (cherry picked from commit 2b8e806)
This migrates the CompletionHandler to structured types and adds a new CompletionItem and CompletionItemType to the general types. --------- Co-authored-by: Ebuka Ezike <[email protected]> (cherry picked from commit 63f1008)
Re-land the symbol table feature in lldb-dap after it was [reverted](llvm@2b8e806) because of a crash in the `aarch64` tests, which was caused by dereferencing `SBSymbol::GetName` which might return `nullptr` for an invalid symbol. This patch reapplies the original commits and adds the missing null check. Also adding `skipIfWindows` for the module symbols tests, since LLDB doesn't recognize the symbols from a.out there. (cherry picked from commit 428ffbd)
This PR adopts JSONTransport in the MCP server implementation. It required a slight change in design in the relationship between the two server classes. Previously, these two had an "is-a" connection, while now they have a "has-a" connection. The "generic" protocol server in Protocol/MCP now operates using a single connection (Transport). This matches the design in DAP where each DAP instance has its own connection. The protocol server in Plugins still supports multiple clients and creates a new server instance for each connection. I believe the new design makes sense in the long term (as proved by DAP) and allows us to make the server stateful if we choose to do so. There's no reason that multiple client support can't live in the generic protocol library, but for now I kept it in ProtocolServerMCP to avoid creating unnecessary abstractions. (cherry picked from commit a49df8e)
Reverts llvm#155034 because the unit tests are flakey on the Debian bot: https://lab.llvm.org/buildbot/#/builders/162. (cherry picked from commit aa1dd4b)
* apply odd table rows color from vscode theme * apply hover color from vscode theme [Screencast From 2025-08-23 14-48-44.webm](https://github.com/user-attachments/assets/a738ac3c-3e56-4a57-b713-7430c614c415) (cherry picked from commit 3cbbc07)
This PR adopts JSONTransport in the MCP server implementation. It required a slight change in design in the relationship between the two server classes. Previously, these two had an "is-a" connection, while now they have a "has-a" connection. The "generic" protocol server in Protocol/MCP now operates using a single connection (Transport). This matches the design in DAP where each DAP instance has its own connection. The protocol server in Plugins still supports multiple clients and creates a new server instance for each connection. I believe the new design makes sense in the long term (as proved by DAP) and allows us to make the server stateful if we choose to do so. There's no reason that multiple client support can't live in the generic protocol library, but for now I kept it in ProtocolServerMCP to avoid creating unnecessary abstractions. This is a reland of llvm#155034 but with significant changes to the tests. The unit tests now test the generic server implementation, which matches the original intent. This also means the test are now single threaded and therefore fully deterministic using the MainLoop. (cherry picked from commit 1ba8b36)
This patch fixes: lldb/unittests/Protocol/ProtocolMCPServerTest.cpp:285:14: error: unused variable 'mutex' [-Werror,-Wunused-variable] (cherry picked from commit acf9611)
pathFormat is an optional field in `initializeAruguments`. (cherry picked from commit 749537f)
This adds or renames existing types to match the names of the types on https://modelcontextprotocol.io/specification/2025-06-18/schema for the existing calls. The new types are used in the unit tests and server implementation to remove the need for crafting various `llvm::json::Object` values by hand. (cherry picked from commit a67257b)
These have been flakey in the last few days on Windows on Arm. llvm#137660 (cherry picked from commit f3b542e)
Moving `lldb_protocol::mcp::MCPTransport` out of Server.h and into its own file and simplifying the name to `Transport`. (cherry picked from commit 71a065e)
Add the scaffolding for a new tool called lldb-mcp. This utility is meant to replace netcat and acts a proxy between the LLM and one or more LLDB instances. In its current form, the utility is a trivial MCP server without any tools or resources. (cherry picked from commit aa71d95)
This fixes build errors like these:
lldb/tools/lldb-mcp/lldb-mcp.cpp:45:41: error: use of undeclared identifier '_O_BINARY'
45 | int result = _setmode(fileno(stdout), _O_BINARY);
| ^
lldb/tools/lldb-mcp/lldb-mcp.cpp:47:36: error: use of undeclared identifier '_O_BINARY'
47 | result = _setmode(fileno(stdin), _O_BINARY);
| ^
(cherry picked from commit 1ec0688)
Fixes linker error: ``` /usr/bin/ld: /tmp/lto-llvm-013f16.o: in function `lldb_private::JSONTransport<lldb_protocol::mcp::Request, lldb_protocol::mcp::Response, lldb_protocol::mcp::Notification>::OnRead(lldb_private::MainLoopBase&, lldb_private::Transport<lldb_protocol::mcp::Request, lldb_protocol::mcp::Response, lldb_protocol::mcp::Notification>::MessageHandler&)': /usr/bin/../lib/gcc/x86_64-redhat-linux/15/../../../../include/c++/15/bits/unique_ptr.h:1085:(.text._ZN12lldb_private13JSONTransportIN13lldb_protocol3mcp7RequestENS2_8ResponseENS2_12NotificationEE6OnReadERNS_12MainLoopBaseERNS_9TransportIS3_S4_S5_E14MessageHandlerE+0x7fc): undefined reference to `lldb_private::TransportUnhandledContentsError::TransportUnhandledContentsError(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)' ``` (cherry picked from commit a4ac04d)
…vm#153536) # Changes 1. Add a new debug configuration field called `debugAdapterEnv`. It accepts a string-valued dictionary or array. 1. This input format is consistent with the (program's) `env` debug configuration. 2. In the adapter descriptor factory, honor the said field before looking at the VS Code settings `Lldb-dap: Environment `. 1. This order is consistent with how things are done for other debug configuration fields, e.g. lldb-dap path and args. 3. In the lldb-dap server, note down the environment entries as a part of the adapter spawn info (now it becomes "path + args + env"), and prompt the user to restart server if such info has changed. # Motivation 1. Adapter environment can be set in `launch.json`. 2. Other debugger extensions can invoke the lldb-dap extension with adapter environment (via debug configuration). # Tests See PR llvm#153536. (cherry picked from commit 20b8664)
It is flakey lately on our Windows on Arm bot: https://lab.llvm.org/buildbot/#/builders/141/builds/11169 ====================================================================== ERROR: test_startDebugging (TestDAP_startDebugging.TestDAP_startDebugging.test_startDebugging) Tests the "startDebugging" reverse request. It makes sure that the IDE can ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 2067, in tearDown Base.tearDown(self) File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 1105, in tearDown hook() # try the plain call and hope it works ^^^^^^ File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\packages\Python\lldbsuite\test\tools\lldb-dap\lldbdap_testcase.py", line 518, in cleanup self.dap_server.terminate() File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\packages\Python\lldbsuite\test\tools\lldb-dap\dap_server.py", line 1593, in terminate raise DebugAdapterProcessError(process.returncode) dap_server.DebugAdapterProcessError: lldb-dap returned non-zero exit status 1. Config=aarch64-C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\build\bin\clang.exe ---------------------------------------------------------------------- See llvm#137660. I am aware that I am disabling these tests with little thought, and I would like to be more careful with it but I don't have the knowledge to really debug these issues. Having buildbot results we can properly triage is more important to me personally than finding whatever the underlying issue is. I'm sure DAP experts will figure it out eventually. (cherry picked from commit 7dd879b)
Flakey on Windows on Arm: https://lab.llvm.org/buildbot/#/builders/141/builds/11181 See llvm#137660. (cherry picked from commit 71cae12)
Flaky on our Windows on Arm bot: https://lab.llvm.org/buildbot/#/builders/141/builds/11465 See llvm#137660 (cherry picked from commit 83b48b1)
Flakey on our Windows on Arm bot: https://lab.llvm.org/buildbot/#/builders/141/builds/11516 See llvm#137660 (cherry picked from commit 757bb36)
Flakey on Windows on Arm: https://lab.llvm.org/buildbot/#/builders/141/builds/11540 See llvm#137660 (cherry picked from commit a0a82ee)
…59542) This reverts the following commits: a0a82ee 757bb36 83b48b1 b45f1fb d2e1539 e2d1bbe 71cae12 7dd879b f3b542e Where I had disabled specific tests due to them being flakey on our Windows on Arm bot. Clearly this strategy isn't working because every day I see a new random test failing. Until something can be done about this, disable every lldb-dap test on Windows on Arm. The coverage we get is just not worth spamming contributors who have nothing to do with lldb-dap. See llvm#137660 (cherry picked from commit 1dc6bf3)
JDevlieghere
force-pushed
the
dap-mcp-cherrypicks
branch
from
800584a to
98db3c1
Compare
|
@swift-ci test |
) After PR 155021 landed, some typescript 'promises' in async functions were not 'await'ed, which caused us some build failures. This fixes that by adding 'await' in the appropriate places. (cherry picked from commit 6d43551)
TLDR ---------- This PR adds `--no-lldbinit` as a new CLI flag to the `lldb-dap` Motivation ----------- Rcently Users reported being unable to control `.lldbinit` file sourcing when debugging through VS Code. llvm#155802. VS Code extensions cannot easily inject custom parameters into the DAP initialize request. Adding `--no-lldbinit` as a CLI flag solves this problem by allowing the decision to skip `.lldbinit` files to be made at debugger startup, before any initialization requests are processed. VS Code extensions can control this behavior by specifying the flag through `debugAdapterArgs` or similar mechanisms in launch configurations. ``` { "type": <extension-type>, "request": "launch", "name": "Debug with --no-lldbinit", "program": "${workspaceFolder}/your-program", "debugAdapterArgs": ["--no-lldbinit"] } ``` Summary ---------- This PR introduces a new command-line flag `--no-lldbinit` (with alias `-x`) to `lldb-dap`. The flag prevents automatic parsing of `.lldbinit` files during debugger initialization, giving users control over whether their LLDB initialization scripts are loaded. ### Key Changes: 1. **CLI Option Definition** (`Options.td`): Added the `--no-lldbinit` flag with `-x` alias 2. **Core Implementation** (`DAP.cpp`): Added support for storing and using the no-lldbinit flag 3. **Initialization Handler** (`InitializeRequestHandler.cpp`): Modified to respect the flag during debugger initialization 4. **Main Tool** (`lldb-dap.cpp`): Added argument parsing for the new flag 5. **Test Infrastructure** (`dap_server.py & lldbdap_testcase.py`): Enhanced test framework to support additional arguments Test Plan --------- ### New Test Coverage (`TestDAP_launch.py`) **Test Method:** `test_no_lldbinit_flag()` **Test Strategy:** 1. **Setup**: Creates a temporary `.lldbinit` file with specific settings that would normally be loaded 2. **Execution**: Launches lldb-dap with the `--no-lldbinit` flag 3. **Verification**: Confirms that the settings from `.lldbinit` are NOT applied, proving the flag works correctly **Test Environment:** * Uses a temporary home directory with a custom `.lldbinit` file * Sets specific LLDB settings (`stop-disassembly-display never`, `target.x86-disassembly-flavor intel`) * Launches debug adapter with `--no-lldbinit` flag via `additional_args` parameter **Validation Approach:** * Executes `settings show stop-disassembly-display` command during initialization * Verifies the output does NOT contain "never" (which would indicate `.lldbinit` was sourced) * Confirms that initialization commands are still executed properly ### Testing Infrastructure Enhancements **File Modifications:** * `dap_server.py`: Enhanced to accept `additional_args` parameter for passing extra CLI flags * `lldbdap_testcase.py`: Updated `build_and_create_debug_adapter()` method to support additional arguments and environment variables ### Unit Test Integration **Unit Test Updates** (`DAPTest.cpp`): * Added initialization of the new flag in test setup to ensure consistent test behavior **Test Run** <img width="1759" height="1373" alt="Screenshot 2025-08-29 at 5 56 18 PM" src="https://github.com/user-attachments/assets/769b319a-5009-4ade-aff8-c5f548b38123" /> --------- Co-authored-by: Piyush Jaiswal <[email protected]> (cherry picked from commit db3054a)
(cherry picked from commit b259dbd)
# Patch Currently, in Server Mode (i.e. `--connection`), all debuggers are destroyed when the **lldb-dap process** terminates. This causes logging and release of resources to be delayed. This can also cause congestion if multiple debuggers have the same destroy callbacks, which will fight for the same resources (e.g. web requests) at the same time. Instead, the debuggers can be destroyed as early as when the **debug session** terminates. This way, logging and release of release of resources can happen as soon as possible. Congestion can also be naturally reduced, because it's unlikely that all debug sessions will terminate at the same time. # Tests See PR llvm#156231. (cherry picked from commit 10bcf0c)
…7150) Co-authored-by: Jonas Devlieghere <[email protected]> (cherry picked from commit a652979)
# Usage
This is an optional new command line option to use with `--connection`.
```
--connection-timeout <timeout> When using --connection, the number of seconds to
wait for new connections after the server has started and after all clients
have disconnected. Each new connection will reset the timeout. When the
timeout is reached, the server will be closed and the process will exit.
Not specifying this argument or specifying non-positive values will cause
the server to wait for new connections indefinitely.
```
A corresponding extension setting `Connection Timeout` is added to the
`lldb-dap` VS Code extension.
# Benefits
Automatic release of resources when lldb-dap is no longer being used
(e.g. release memory used by module cache).
# Test
See PR.
(cherry picked from commit acb38c8)
See llvm#156803 (comment) for full context. Summary: * In llvm#156803, we see a builtbot failure for `TestDAP_server.py` on Debian. Note that macOS and other Linux distributions (like CentOS, or whatever Linux the [required buildbot](https://github.com/llvm/llvm-project/actions/runs/17594257911/job/49982560193#logs) uses) seem to pass the tests. * In the 3 newly added tests, the most complex test case passed, while the other easier ones failed. ``` PASS: LLDB (/home/worker/2.0.1/lldb-x86_64-debian/build/bin/clang-x86_64) :: test_connection_timeout_multiple_sessions (TestDAP_server.TestDAP_server.test_connection_timeout_multiple_sessions) FAIL: LLDB (/home/worker/2.0.1/lldb-x86_64-debian/build/bin/clang-x86_64) :: test_connection_timeout_long_debug_session (TestDAP_server.TestDAP_server.test_connection_timeout_long_debug_session) FAIL: LLDB (/home/worker/2.0.1/lldb-x86_64-debian/build/bin/clang-x86_64) :: test_connection_timeout_at_server_start (TestDAP_server.TestDAP_server.test_connection_timeout_at_server_start) ``` * The error is that `process.wait(timeout)` timed out during the teardown of the tests. * The above suggests that maybe the root cause is that the timeout is set too strictly (and that maybe the server termination on Debian is slower than the other Linux distribution for some reason). * This patch loosens that timeout from 2s to 5s. Let's see if this works. * FWIW, I cannot verify the root cause, because I don't have a Debian machine. (cherry picked from commit 11a4f5b)
This patch fixes the problem, when after a `setVariable` request pointers and references to the variable are not updated. VSCode doesn't send a `variables` request after a `setVariable` request, so we should trigger it explicitly via`invalidated` event .Also, updated `writeMemory` request in similar way. (cherry picked from commit 770cd43)
llvm#157980) # Changes llvm#153536 added a new debug configuration field called `debugAdapterEnv` and enabled it in `launch.json` **for `launch` requests**. This patch enables the same for **`attach` requests**. This patch also improves the regex used in this field, i.e. shortens it and fixes the double backslashes (`\\`) in `debug-adapter-factory.ts` (note: the ones in `package.json` need the double backslashes). # Test Manually tested the following values in `attach` requests (so that we are testing both changes at the same time): ``` // Accepted "debugAdapterEnv": [ "AAA=BBB", ], "debugAdapterEnv": [ "AAA=", ], "debugAdapterEnv": [ "AAA", ], // Rejected "debugAdapterEnv": [ "=AAA", ], "debugAdapterEnv": [ "=", ], "debugAdapterEnv": [ "", ], ``` (cherry picked from commit 13daa1e)
As far as I understand, lldb-dap does not currently support stdio redirection. I have added support for this via a new field in the launch configuration named `stdio`. It was inspired by the same named field in [CodeLLDB](https://github.com/vadimcn/codelldb/blob/master/MANUAL.md#stdio-redirection). (cherry picked from commit 1819798)
Fixed a typo in the `invalidated` event according to [DAP](https://microsoft.github.io/debug-adapter-protocol/specification#Events_Invalidated) specification. While the field is `frameId` elsewhere, it must be `stackFrameId` in this event. (cherry picked from commit 90d96b3)
This patch adds support for the `memory` event from the [DAP](https://microsoft.github.io/debug-adapter-protocol/specification#Events_Memory). After this event is emitted, VS Code refetches the corresponding memory range and re-renders the memory view. I think this patch and [PR](llvm#151884) can improve experience for users who use `setVariable` request. (cherry picked from commit c744f61)
Bumps form-data from 4.0.1 to 4.0.4 to resolve a critical security vulnerability in form-data. (cherry picked from commit 082d1d9)
…59797) # Motiviation This helps the development of the lldb-dap binary. For example, when testing a locally built lldb-dap binary, one probably wants to restart the server after each build in order to use the latest binary. Not doing so leads to confusion like "why the new lldb-dap isn't doing what I just changed?". This patch adds dependency to the `chokidar` package. The reason is that we need something to detect changes to the `lldb-dap` binary file and `chokidar` appears to be the most reliable package to do so. An alternative solution which doesn't require adding dependencies is discussed below (solution 1). # Two different solutions considered Solution 1: Restart server when lldb-dap binary's modification time changes. llvm#159481 implements solution 1. Solution 2: Restart server when lldb-dap binary has changed (as detected by a file system watcher). This patch implements solution 2 (using `chokidar`). # This patch (solution 2) If the lldb-dap binary has changed, the next time the user start a debug session, a dialog box will show up and prompt the user to restart the server. Depend on what has changed, the dialog box will show different content (see below) * When both the lldb-dap binary and the arguments have changed: <img width="520" height="357" alt="diff_args_and_binary" src="https://github.com/user-attachments/assets/6580e05f-05c3-4d80-8c1a-9c3abf279218" /> * When only the lldb-dap binary has changed: <img width="260" height="384" alt="diff_binary" src="https://github.com/user-attachments/assets/933b8987-9c21-44f3-ad68-57b8eeb58525" /> * When only the arguments have changed (existing): <img width="520" height="343" alt="diff_args" src="https://github.com/user-attachments/assets/c0e6771c-ad32-4fb8-b5df-b34cce48ed1a" /> (cherry picked from commit 6007a4d)
(cherry picked from commit 89d2b7e)
The most recent version of the lldb-dap VS Code extension (0.2.17) fails to activate because it's missing the `chokidar` dependency in the `.vsix`. In order to prevent this from happening in the future, this PR adds a `bundle-extension` step to extension packaging in order to bundle the extension into a single `.js` file using esbuild. All dependencies will be included in this bundle so that the `.vscodeignore` doesn't need to be updated. (cherry picked from commit 2a5c6e1)
Bump the version to 0.2.18 to address the missing chokidar dependency in the .vsix. (cherry picked from commit 18cffb8)
|
@swift-ci test |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
18 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.