feat(deploy): post-deploy smoke tests from project manifest (#753)

docs/COMMAND_REFERENCE.md

@@ -108,7 +108,12 @@ starforge wallet multisig notify proposal.json --message "Please sign the treasu
 | `inspect storage` | Deep storage inspection |
 | `deploy --wasm <FILE>` | Prepare Soroban deployment |
 
-**`deploy` flags:** `--network`, `--wallet`, `--optimize`, `--simulate`, `--yes`, `--execute`, `--policy`, `--checklist`
+**`deploy` flags:** `--network`, `--wallet`, `--optimize`, `--simulate`, `--yes`, `--execute`, `--policy`, `--checklist`, `--skip-smoke`
+
+After a successful `--execute`, `deploy` runs the `[[smoke_tests]]` declared in
+`starforge-project.toml` against the new contract. A smoke failure exits with
+code **9** (`SMOKE_TEST_FAILURE`), which is distinct from a failed deploy.
+`--skip-smoke` opts out. See [SMOKE_TESTS.md](SMOKE_TESTS.md).
 
 Deploy policy files (`starforge-deploy-policy.toml`) gate allowed networks,
 required reviewers, and checklist items. Validate in CI with

docs/CONFIGURATION.md

@@ -93,6 +93,10 @@ Security model:
 - `starforge config show` tells you which lockfile (if any) is participating,
   so an override is never mistaken for a personal setting.
 
+The lockfile can also declare `[[smoke_tests]]`, which `starforge deploy
+--execute` runs after a successful deploy. These entries are not config
+overrides. See [SMOKE_TESTS.md](SMOKE_TESTS.md) for the schema.
+
 Commands that only read configuration use the effective config. Commands
 that write configuration still operate on the user config only — project
 overrides are an input, never something persisted back.

docs/SMOKE_TESTS.md

@@ -0,0 +1,107 @@
+# Post-deploy smoke tests
+
+A project can declare smoke tests in its `starforge-project.toml`. When
+`starforge deploy --execute` succeeds, StarForge runs them against the freshly
+deployed contract. A broken initialisation or configuration then shows up in
+the deploy itself instead of in the first user report.
+
+A complete sample project lives in
+[`examples/smoke-tests/`](../examples/smoke-tests/).
+
+## When smoke tests run
+
+| Situation | Smoke tests |
+|-----------|-------------|
+| `deploy --execute` succeeds and the Stellar CLI returns a contract ID | **Run**, in declaration order |
+| `deploy --execute` fails | Not run (the deploy error is reported as before) |
+| `deploy --execute` succeeds but no contract ID appears in the CLI output | Not run; a warning is printed |
+| `deploy` without `--execute`, `--dry-run`, or `--json` | Not run |
+| `--skip-smoke` is passed | Not run, and the manifest's `smoke_tests` are not loaded |
+| No `smoke_tests` in the manifest, or no manifest | Nothing changes: deploy behaves exactly as before |
+
+Every test runs, even after one fails, so a single report shows everything
+that is broken.
+
+## Manifest schema
+
+Add one `[[smoke_tests]]` table per test:
+
+```toml
+[[smoke_tests]]
+name = "hello-returns-greeting"             # required, unique
+invoke = { function = "hello", args = ["--to", "world"] }
+expect_contains = "world"
+timeout_secs = 60
+
+[[smoke_tests]]
+name = "health-script"
+command = "./scripts/health.sh"
+expect_exit = 0
+```
+
+| Key | Type | Required | Meaning |
+|-----|------|----------|---------|
+| `name` | string | yes | Unique, non-empty name shown in the report |
+| `invoke` | table | one of `invoke` / `command` | Contract call on the deployed contract, run as `stellar contract invoke --id <contract> --source <deployer> --network <network> -- <function> <args...>` |
+| `invoke.function` | string | yes, with `invoke` | Contract function name |
+| `invoke.args` | array of strings | no | Arguments after the function name, passed as-is with no shell involved |
+| `command` | string | one of `invoke` / `command` | Shell command, run with `sh -c` (`cmd /C` on Windows) from the directory that contains `starforge-project.toml` |
+| `expect_exit` | integer | no (default `0`) | Exit code the check must return |
+| `expect_output` | string | no | Stdout must equal this exactly, with surrounding whitespace trimmed |
+| `expect_contains` | string | no | Stdout must contain this substring |
+| `timeout_secs` | integer, 1–3600 | no (default `60`) | Longer runs are killed and count as a failure |
+
+When several assertions are set, all of them must hold. The exit code is
+checked first.
+
+`command` checks receive the deployment through environment variables:
+
+| Variable | Value |
+|----------|-------|
+| `STARFORGE_CONTRACT_ID` | Contract ID returned by the deploy |
+| `STARFORGE_NETWORK` | Network deployed to (`testnet` / `mainnet`) |
+| `STARFORGE_SOURCE` | Source account the deploy used |
+
+### Validation
+
+Smoke test declarations are validated when the manifest loads, which happens
+**before** the deploy starts. A mistake therefore never surfaces after a
+contract is already live. The deploy is refused with a configuration error
+(exit code 3) when:
+
+- a test is missing `name`, or two tests share a name
+- a test sets both `command` and `invoke`, or neither of them
+- `command` or `invoke.function` is empty
+- `timeout_secs` is outside 1–3600
+- a key is unknown (for example the typo `expect_contain`)
+
+## Results and exit codes
+
+Each test prints one labelled line:
+
+```text
+  SMOKE PASS  hello-returns-greeting (412 ms)
+  SMOKE FAIL  counter-starts-at-zero (388 ms): expected output "0", got "7"
+  SMOKE FAIL  health-script (60001 ms): timed out after 60s
+```
+
+A smoke failure is kept separate from a deploy failure:
+
+| Code | Name | Meaning |
+|------|------|---------|
+| 0 | SUCCESS | Deploy succeeded and every smoke test passed |
+| 6 | EXECUTION_ERROR | The deploy itself failed; smoke tests did not run |
+| 3 | CONFIG_ERROR | Invalid `smoke_tests` declaration; nothing was deployed |
+| **9** | **SMOKE_TEST_FAILURE** | The deploy succeeded, but at least one smoke test failed or timed out |
+
+When a smoke test fails, the deploy is still recorded as successful in
+`starforge deployments`, because the contract is on-chain. The error message
+says so:
+
+```text
+Post-deploy smoke tests failed (1 of 3). The deployment itself succeeded:
+contract CABC…XYZ is live on testnet.
+```
+
+In CI, a check like `if [ $? -eq 9 ]` separates "the deployed contract
+misbehaves" from "the deploy never happened".

docs/SUMMARY.md

@@ -23,6 +23,7 @@
 # Deploying and operating
 
 - [Deploy policy as code](DEPLOY_POLICY.md)
+- [Post-deploy smoke tests](SMOKE_TESTS.md)
 - [Deployment checkpoints](DEPLOYMENT_CHECKPOINTS.md)
 - [Deployment scaling](DEPLOYMENT_SCALING.md)
 - [Simulation resource fees](SIMULATION_RESOURCES.md)

examples/smoke-tests/README.md

@@ -0,0 +1,46 @@
+# Sample: post-deploy smoke tests
+
+This sample shows the `[[smoke_tests]]` section of `starforge-project.toml`.
+The tests target the `simple-counter` template.
+
+| File | What it shows |
+|------|---------------|
+| [`starforge-project.toml`](starforge-project.toml) | Passing tests: two contract invocations and one shell check |
+| [`failing/starforge-project.toml`](failing/starforge-project.toml) | One deliberately failing assertion |
+
+## Try it
+
+1. Build a counter contract from `templates/examples/simple-counter`. It
+   exposes `get_count`, `increment` and `reset`.
+2. Copy `starforge-project.toml` from this directory to the root of that
+   project.
+3. Deploy from inside the project. The smoke tests run once the deploy is
+   confirmed:
+
+```bash norun
+starforge deploy --wasm path/to/counter.wasm \
+  --wallet deployer --network testnet --yes --execute
+```
+
+Expected output after the deploy report:
+
+```text
+  SMOKE PASS  counter-starts-at-zero (402 ms)
+  SMOKE PASS  increment-returns-one (5130 ms)
+  SMOKE PASS  contract-id-is-a-strkey (6 ms)
+
+✓ 3 smoke test(s) passed
+```
+
+With `failing/starforge-project.toml` in place instead, the deploy still
+succeeds, but the command exits with code **9** (`SMOKE_TEST_FAILURE`):
+
+```text
+  SMOKE PASS  counter-starts-at-zero (398 ms)
+  SMOKE FAIL  counter-is-42 (401 ms): expected output "42", got "0"
+
+✗ 1 of 2 smoke test(s) failed. The deployment succeeded and the contract is live.
+```
+
+Pass `--skip-smoke` to deploy without running the tests. The full schema is
+in [docs/SMOKE_TESTS.md](../../docs/SMOKE_TESTS.md).

examples/smoke-tests/failing/starforge-project.toml

@@ -0,0 +1,16 @@
+# Deliberately failing smoke test, for the docs and for trying out the
+# failure path. The deploy succeeds, then `counter-is-42` fails, so
+# `starforge deploy --execute` exits with code 9 (SMOKE_TEST_FAILURE),
+# not with a deploy error code.
+
+network = "testnet"
+
+[[smoke_tests]]
+name = "counter-starts-at-zero"
+invoke = { function = "get_count" }
+expect_output = "0"
+
+[[smoke_tests]]
+name = "counter-is-42"
+invoke = { function = "get_count" }
+expect_output = "42"

examples/smoke-tests/starforge-project.toml

@@ -0,0 +1,27 @@
+# Sample project manifest with post-deploy smoke tests (#753).
+#
+# These tests target the `simple-counter` template
+# (templates/examples/simple-counter). `starforge deploy --execute` runs them
+# in order after the deploy is confirmed. See docs/SMOKE_TESTS.md.
+
+network = "testnet"
+
+# A freshly deployed counter must start at zero.
+[[smoke_tests]]
+name = "counter-starts-at-zero"
+invoke = { function = "get_count" }
+expect_output = "0"
+
+# The write path works and returns the new value.
+[[smoke_tests]]
+name = "increment-returns-one"
+invoke = { function = "increment" }
+expect_output = "1"
+timeout_secs = 90
+
+# Shell checks receive STARFORGE_CONTRACT_ID / STARFORGE_NETWORK /
+# STARFORGE_SOURCE. This one is POSIX sh; use `cmd` syntax on Windows.
+[[smoke_tests]]
+name = "contract-id-is-a-strkey"
+command = 'test "${#STARFORGE_CONTRACT_ID}" -eq 56 && test "${STARFORGE_CONTRACT_ID#C}" != "$STARFORGE_CONTRACT_ID"'
+timeout_secs = 5

src/commands/deploy.rs

@@ -6,7 +6,9 @@ use crate::utils::{
         DeployRecord, DeployStatus,
     },
     deploy_policy, deployment_monitor, horizon, notifications, optimizer, output, print as p,
-    simulation_resources, soroban, wallet_signer,
+    project_config, simulation_resources,
+    smoke_tests::{self, SmokeContext, SmokeTest},
+    soroban, wallet_signer,
     wasm_hash::{compute_wasm_hash, BuildEnvironment},
     wasm_preflight,
 };
@@ -78,13 +80,65 @@ pub struct DeployArgs {
     /// Comma-separated checklist item ids satisfied for this deploy (see deploy policy)
     #[arg(long, value_delimiter = ',')]
     pub checklist: Option<Vec<String>>,
-    /// Human-readable note explaining why this deployment is happening;
-    /// persisted with the deployment history record for auditability
+    /// Do not run the `[[smoke_tests]]` declared in `starforge-project.toml`
+    /// after a successful `--execute` deploy
     #[arg(long)]
-    pub note: Option<String>,
-    /// Change-log snippet attached to this deployment's history record
-    #[arg(long)]
-    pub changelog: Option<String>,
+    pub skip_smoke: bool,
+}
+
+/// Smoke tests declared in the discovered project manifest, plus the
+/// directory they run from. Loading validates them, so a bad declaration
+/// stops the command before anything is deployed.
+fn load_smoke_tests(skip: bool) -> Result<Option<(Vec<SmokeTest>, PathBuf)>> {
+    if skip {
+        return Ok(None);
+    }
+    let start = std::env::current_dir().unwrap_or_else(|_| PathBuf::from("."));
+    Ok(project_config::find_and_load_project_lockfile(&start)?
+        .filter(|(_, lockfile)| !lockfile.smoke_tests.is_empty())
+        .map(|(path, lockfile)| {
+            (
+                lockfile.smoke_tests,
+                smoke_tests::workdir_for_manifest(&path),
+            )
+        }))
+}
+
+/// Run post-deploy smoke tests against a confirmed deployment and report
+/// each result. Returns a [`smoke_tests::SmokeTestFailure`] error if any failed.
+fn run_post_deploy_smoke_tests(
+    tests: &[SmokeTest],
+    workdir: PathBuf,
+    contract_id: &str,
+    network: &str,
+    source: &str,
+) -> Result<()> {
+    p::header("Post-deploy Smoke Tests");
+    let ctx = SmokeContext {
+        contract_id: contract_id.to_string(),
+        network: network.to_string(),
+        source: source.to_string(),
+        workdir,
+    };
+    let report = smoke_tests::run_all(tests, &ctx);
+    for (line, result) in report.lines().iter().zip(&report.results) {
+        if result.outcome.is_pass() {
+            println!("  {}", line.green());
+        } else {
+            println!("  {}", line.red());
+        }
+    }
+    println!();
+    if report.all_passed() {
+        p::success(&format!("{} smoke test(s) passed", report.passed()));
+    } else {
+        p::error(&format!(
+            "{} of {} smoke test(s) failed. The deployment succeeded and the contract is live.",
+            report.failed(),
+            report.results.len()
+        ));
+    }
+    smoke_tests::into_result(&report, &ctx)
 }
 
 /// Extract a Soroban contract id (56-char `C…` strkey) from CLI stdout/stderr.
@@ -466,6 +520,10 @@ pub async fn handle(args: DeployArgs) -> Result<()> {
         );
     }
 
+    // Validate declared smoke tests up front: a broken declaration must fail
+    // here, never after a contract is already live.
+    let smoke = load_smoke_tests(args.skip_smoke)?;
+
     let mut wasm_path = args.wasm.clone();
     let mut wasm_bytes = fs::read(&wasm_path)?;
     let mut wasm_size_kb = wasm_bytes.len() as f64 / 1024.0;
@@ -1029,7 +1087,31 @@ pub async fn handle(args: DeployArgs) -> Result<()> {
             p::kv("Changelog", changelog);
         }
         println!("{}", stdout);
+
+        // Smoke tests run only against a confirmed deployment, which means
+        // the Stellar CLI returned a contract ID.
+        if let Some((tests, workdir)) = smoke {
+            match parsed_contract_id.as_deref() {
+                Some(contract_id) => run_post_deploy_smoke_tests(
+                    &tests,
+                    workdir,
+                    contract_id,
+                    &args.network,
+                    &wallet.public_key,
+                )?,
+                None => p::warn(&format!(
+                    "Skipped {} smoke test(s): no contract ID in Stellar CLI output to test against.",
+                    tests.len()
+                )),
+            }
+        }
     } else {
+        if let Some((tests, _)) = &smoke {
+            p::info(&format!(
+                "{} smoke test(s) declared; they run only after an executed deploy.",
+                tests.len()
+            ));
+        }
         p::info("Dry-run complete. Use --execute to deploy for real.");
     }
 

src/main.rs

@@ -540,7 +540,17 @@ fn recovery_hints(command: &str, err: &anyhow::Error) -> Vec<String> {
             }
         }
         "deploy" => {
-            if msg.contains("wasm") || msg.contains("not found") || msg.contains("no such file") {
+            if msg.contains("smoke tests failed") {
+                hints.push("The contract is deployed; only the post-deploy checks failed.".into());
+                hints.push(
+                    "Fix the contract or the [[smoke_tests]] entry in starforge-project.toml."
+                        .into(),
+                );
+                hints.push("Deploy without running smoke tests: --skip-smoke".into());
+            } else if msg.contains("wasm")
+                || msg.contains("not found")
+                || msg.contains("no such file")
+            {
                 hints.push("Build your contract first: stellar contract build".into());
                 hints.push("Make sure you pass the correct --wasm path to deploy.".into());
             } else if msg.contains("account") || msg.contains("not found on") {