Skip to content

Commit f3b4cd9

Browse files
Anai-Guoclaude
Anai-Guo
and
claude
committed
Architecture consolidation: eliminate duplicates, add e2e demo, fill docs
Architecture: - agent/loop.py: 152→49 lines, now a thin wrapper around harness engine - server.py: all 8 MCP tools now delegate to harness/tools/ (no duplicate logic) - New: generate_skill_tool.py added to harness tools registry (9 tools total) - Single source of truth for all tool logic: harness/tools/ End-to-end demo: - examples/iv_demo.py: full simulated IV workflow scan → classify → propose → validate → execute (diode IV curve) → export CSV+JSON - Proves the system works end-to-end without real instruments Documentation: - 5 feature pages rewritten with real content (discovery, planning, safety, analysis, web-gui) - 5 placeholder pages deleted (terminal, templates/overview, templates/catalog, api/cli, api/mcp) - mkdocs.yml nav simplified 140 tests passing, lint clean. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 1b203bb commit f3b4cd9

19 files changed

Lines changed: 2103 additions & 296 deletions

File tree

‎docs/api/cli.md‎

Lines changed: 0 additions & 5 deletions
This file was deleted.

‎docs/api/mcp.md‎

Lines changed: 0 additions & 5 deletions
This file was deleted.

‎docs/features/analysis.md‎

Lines changed: 131 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,134 @@
11
# AI Analysis
22

3-
LabAgent provides AI-powered data analysis that runs automatically after each measurement. It performs curve fitting, extracts key parameters, identifies anomalies, and generates publication-ready plots.
3+
LabAgent provides a three-tier data analysis pipeline that scales from fully deterministic template scripts to AI-generated custom analysis with physics interpretation. All tiers produce publication-ready plots (PNG at 300 dpi and PDF) and extract key physical quantities from measurement data.
44

5-
Analysis results include statistical summaries, physical interpretations, and suggestions for follow-up measurements.
5+
## Three-Tier Analysis Architecture
6+
7+
### Tier 1: Template-Based Analysis
8+
9+
Built-in analysis scripts handle common measurement types with zero configuration. Each template is a complete Python script with placeholder substitution for the data path and output directory.
10+
11+
Available analysis templates:
12+
13+
| Measurement Type | Template | Key Outputs |
14+
|-----------------|----------|-------------|
15+
| AHE | `ahe.py` | Anomalous Hall resistance, coercive field, loop area |
16+
| IV | `iv.py` | Differential resistance, threshold voltage, conductance |
17+
| MR | `mr.py` | Magnetoresistance ratio, coercive field, saturation field |
18+
| RT | `rt.py` | Residual resistance ratio, transition temperature, TCR |
19+
20+
```bash
21+
# Analyze data with the built-in IV template
22+
labharness analyze data.csv --type IV
23+
```
24+
25+
Template scripts use numpy, scipy, and matplotlib. They handle common edge cases (empty data, NaN values, comment-line headers) and print extracted values in a `key = value` format that the analyzer parses automatically.
26+
27+
### Tier 2: AI-Generated Analysis
28+
29+
When no built-in template exists for a measurement type, or when you need custom analysis, the AI generator creates a tailored Python script. It receives:
30+
31+
- The measurement type
32+
- A preview of the data (column headers, row count, first 20 rows)
33+
- Optional custom instructions from the user
34+
35+
The LLM generates a complete, self-contained script that follows the same conventions as the built-in templates: load data, process, extract values, save figures.
36+
37+
```bash
38+
# Force AI-generated analysis even when a template exists
39+
labharness analyze data.csv --type AHE --ai
40+
41+
# Add custom analysis instructions
42+
labharness analyze data.csv --type IV --ai \
43+
--instructions "Fit the forward bias region to extract ideality factor"
44+
```
45+
46+
If no template is found and no LLM is configured, the system raises an error with a clear message listing available templates.
47+
48+
### Tier 3: AI Interpretation
49+
50+
After analysis (from either tier), the interpreter provides physics-level insights about the results. It examines extracted values and script output to deliver:
51+
52+
1. **Physical meaning** -- what the extracted values indicate about the sample
53+
2. **Reasonableness check** -- whether results are physically plausible
54+
3. **Literature comparison** -- how values compare to typical ranges
55+
4. **Anomaly detection** -- flags unexpected or suspicious results
56+
5. **Follow-up suggestions** -- recommends additional measurements if warranted
57+
58+
```bash
59+
# Run analysis with AI interpretation
60+
labharness analyze data.csv --type AHE --interpret
61+
```
62+
63+
## Python API
64+
65+
The `Analyzer` class provides programmatic access to all three tiers:
66+
67+
```python
68+
from pathlib import Path
69+
from lab_harness.analysis.analyzer import Analyzer
70+
71+
analyzer = Analyzer(output_dir=Path("./results"))
72+
73+
# Full pipeline: template script -> execute -> interpret
74+
result = analyzer.analyze(
75+
data_path=Path("ahe_data.csv"),
76+
measurement_type="AHE",
77+
interpret=True,
78+
)
79+
80+
print(result.extracted_values) # {"R_AHE": "0.15 Ohm", "H_c": "250 Oe"}
81+
print(result.figures) # ["./results/ahe_Rxy_vs_H.png", ...]
82+
print(result.ai_interpretation) # Physics insights string
83+
```
84+
85+
### AnalysisResult Fields
86+
87+
| Field | Type | Description |
88+
|-------|------|-------------|
89+
| `measurement_type` | str | Type of measurement analyzed |
90+
| `script_path` | str | Path to the generated analysis script |
91+
| `script_source` | str | Full source code of the script |
92+
| `figures` | list[str] | Paths to generated PNG and PDF figures |
93+
| `extracted_values` | dict | Key-value pairs of extracted physical quantities |
94+
| `ai_interpretation` | str | AI-generated physics insights (empty if not requested) |
95+
| `stdout` | str | Raw script output |
96+
97+
## Script Execution
98+
99+
Analysis scripts run in a sandboxed subprocess with a configurable timeout (default 120 seconds). The analyzer:
100+
101+
1. Saves the generated script to the output directory
102+
2. Runs it with `python` in a subprocess
103+
3. Captures stdout/stderr
104+
4. Parses extracted values from stdout lines matching `key = value`
105+
5. Collects all PNG and PDF files from the output directory as figures
106+
107+
If the script fails (non-zero exit code), the error is raised with the first 500 characters of stderr for debugging.
108+
109+
## Supported Output Formats
110+
111+
Analysis figures are saved in both PNG (300 dpi, for viewing) and PDF (vector, for publication). The data export system supports three formats for raw data:
112+
113+
| Format | Extension | Features |
114+
|--------|-----------|----------|
115+
| CSV | `.csv` | Metadata header as comments, standard column format |
116+
| JSON | `.json` | Structured metadata + data array, human-readable |
117+
| HDF5 | `.h5` | Columnar datasets with metadata attributes (requires h5py) |
118+
119+
```bash
120+
# Export measurement data
121+
labharness export data.csv --format json
122+
labharness export data.csv --format hdf5
123+
```
124+
125+
## Extending Analysis Templates
126+
127+
To add a new analysis template:
128+
129+
1. Create a Python script in `src/lab_harness/analysis/templates/` named `{type}.py`
130+
2. Use `{{DATA_PATH}}` and `{{OUTPUT_DIR}}` as placeholders
131+
3. Print extracted values as `key = value` lines to stdout
132+
4. Save figures to `{{OUTPUT_DIR}}` as PNG and PDF
133+
134+
The template will be automatically discovered by the analyzer for that measurement type.

‎docs/features/discovery.md‎

Lines changed: 101 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,104 @@
11
# Instrument Discovery
22

3-
LabAgent automatically scans VISA resources to detect connected instruments. It identifies instrument models, capabilities, and communication parameters, then maps them to compatible measurement templates.
3+
LabAgent automatically discovers laboratory instruments connected via GPIB, USB, serial, and TCP/IP buses. The discovery pipeline scans all available communication interfaces, queries each instrument for its identity, and classifies instruments into measurement roles using a two-stage approach: deterministic dictionary lookup followed by an optional LLM fallback.
44

5-
Use `labharness scan` to discover all instruments on your GPIB, USB, and TCP/IP buses.
5+
## How Scanning Works
6+
7+
### PyVISA Bus Scanning
8+
9+
The primary scanner uses PyVISA to enumerate all VISA resources on the system. For each discovered resource, it sends a standard IEEE 488.2 `*IDN?` query and parses the four-field response (vendor, model, serial number, firmware version).
10+
11+
```bash
12+
labharness scan
13+
```
14+
15+
Supported bus types detected from the resource string:
16+
17+
| Bus Type | Resource Pattern | Example |
18+
|----------|-----------------|---------|
19+
| GPIB | `GPIB0::N::INSTR` | `GPIB0::5::INSTR` |
20+
| USB | `USB0::...::INSTR` | `USB0::0x05E6::0x2400::INSTR` |
21+
| Serial (VISA) | `ASRL` or `COM` | `ASRL1::INSTR` |
22+
| Ethernet | `TCPIP::...` | `TCPIP0::192.168.1.10::INSTR` |
23+
24+
The scanner is configurable with a timeout (default 2000 ms per instrument) and can optionally skip the `*IDN?` query if you only need resource enumeration.
25+
26+
### Serial Port Scanning
27+
28+
For instruments that do not appear on the VISA bus (standalone serial devices, Arduino-based controllers, custom hardware), a separate serial scanner enumerates all available COM ports using pyserial. Serial scanning is non-invasive: it lists ports and their metadata (manufacturer, description, serial number) without sending any commands, since probing serial devices can interfere with running equipment.
29+
30+
```bash
31+
# Serial ports are included automatically in the scan
32+
labharness scan
33+
```
34+
35+
### Scan Output
36+
37+
Each discovered instrument is represented as an `InstrumentRecord` containing:
38+
39+
- **resource** -- VISA resource string or COM port path
40+
- **vendor** -- manufacturer name (e.g., KEITHLEY, Lakeshore)
41+
- **model** -- model number (e.g., MODEL 2400, 335)
42+
- **serial** -- instrument serial number
43+
- **firmware** -- firmware version
44+
- **bus** -- communication bus type (gpib, usb, serial, ethernet)
45+
46+
You can save the full inventory to a JSON file for reuse:
47+
48+
```bash
49+
labharness scan -o inventory.json
50+
```
51+
52+
## AI Instrument Classifier
53+
54+
After scanning, the classifier maps instruments to measurement roles. This is a two-stage process.
55+
56+
### Stage 1: Dictionary Lookup
57+
58+
A built-in database of known instruments provides instant, deterministic classification. The database covers common lab equipment from Keithley, Lakeshore, and Keysight:
59+
60+
| Model | Vendor | Assigned Roles | Capabilities |
61+
|-------|--------|----------------|--------------|
62+
| 2400, 2410 | Keithley | source_meter | source/measure IV |
63+
| 2000 | Keithley | dmm | measure V, R |
64+
| 2182, 2182A | Keithley | nanovoltmeter | low-noise V measurement |
65+
| 6221 | Keithley | ac_current_source | pulse/AC current source |
66+
| 6517, 6517B | Keithley | electrometer | high-R measurement |
67+
| 425, 455 | Lakeshore | gaussmeter | magnetic field measurement |
68+
| 335, 340, 350 | Lakeshore | temperature_controller | temperature control |
69+
| E4980 | Keysight | lcr_meter | capacitance/impedance |
70+
71+
### Stage 2: LLM Fallback
72+
73+
When the dictionary lookup cannot assign all required roles for a measurement type, the classifier invokes an LLM. The LLM receives the list of unmatched instruments (with their `*IDN?` responses) and the list of still-needed roles. It returns structured JSON with role assignments, confidence scores, and reasoning.
74+
75+
Safety guardrails ensure the LLM can only assign roles that are genuinely unassigned -- it cannot override dictionary-based assignments or assign duplicate roles.
76+
77+
```bash
78+
# Classify instruments for an AHE measurement
79+
labharness classify AHE
80+
81+
# Use a saved inventory file
82+
labharness classify AHE --inventory inventory.json
83+
```
84+
85+
### Measurement Role Requirements
86+
87+
Each measurement type defines its required instrument roles. Example configurations:
88+
89+
| Measurement | Required Roles |
90+
|-------------|---------------|
91+
| IV | source_meter |
92+
| AHE | source_meter, dmm, gaussmeter |
93+
| MR | source_meter, dmm, gaussmeter |
94+
| RT | source_meter, temperature_controller |
95+
| SOT | source_meter, ac_current_source, dmm, gaussmeter |
96+
| CV | lcr_meter, temperature_controller |
97+
| Hall | source_meter, dmm, magnet |
98+
| PPMS_RT | ppms |
99+
100+
The full classifier supports 40+ measurement types across electrical, magnetic, thermoelectric, optical, superconducting, electrochemical, and biosensor disciplines.
101+
102+
## Extending the Instrument Database
103+
104+
To add support for new instruments, update the `KNOWN_INSTRUMENTS` dictionary in `src/lab_harness/discovery/classifier.py` with the model string, vendor, assignable roles, and capabilities. Instruments not in the database will be handled by the LLM fallback if an API key is configured.

‎docs/features/planning.md‎

Lines changed: 114 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,117 @@
11
# Measurement Planning
22

3-
LabAgent uses AI to generate complete measurement plans from natural language descriptions. Given your research goal and available instruments, it produces step-by-step procedures with parameter ranges, safety limits, and expected outcomes.
3+
LabAgent generates complete, validated measurement plans from YAML templates. Rather than letting an AI freely invent measurement parameters, the system uses curated templates as a safety-anchored starting point and applies optional AI optimization on top. This design ensures reproducibility and prevents dangerous parameter choices.
44

5-
Use `labharness propose <measurement_type>` to generate a plan, then review and approve before execution.
5+
## Template System
6+
7+
Every measurement plan starts from a YAML template that defines:
8+
9+
- **Sweep axis** -- parameter to sweep (label, unit, start/stop/step, instrument role)
10+
- **Data channels** -- quantities to record at each sweep point
11+
- **Safety limits** -- maximum current, voltage, field, and temperature
12+
- **Execution parameters** -- settling time, number of averages, output directory
13+
- **Outer sweep** (optional) -- secondary sweep for 2D measurements (e.g., temperature loop around a field sweep)
14+
15+
### Example: IV Template
16+
17+
```yaml
18+
name: "IV Measurement"
19+
description: "Sweep source current and measure voltage"
20+
21+
x_axis:
22+
label: "Source Current"
23+
unit: "mA"
24+
start: -1.0
25+
stop: 1.0
26+
step: 0.01
27+
role: "source_meter"
28+
29+
y_channels:
30+
- label: "Voltage"
31+
unit: "V"
32+
role: "source_meter"
33+
34+
max_current_a: 0.01
35+
max_voltage_v: 20.0
36+
settling_time_s: 0.1
37+
num_averages: 1
38+
```
39+
40+
### Building a Plan
41+
42+
```bash
43+
# Generate a plan from template defaults
44+
labharness propose IV
45+
46+
# The Python API supports overrides and sample-aware optimization
47+
from lab_harness.planning.plan_builder import build_plan_from_template
48+
49+
plan = build_plan_from_template(
50+
"AHE",
51+
overrides={"x_axis": {"start": -5000, "stop": 5000, "step": 50}},
52+
sample_description="20nm CoFeB/MgO thin film",
53+
)
54+
```
55+
56+
User-provided overrides are deep-merged into the template, with user values taking precedence over both template defaults and AI suggestions.
57+
58+
## 46 Templates Across 9 Disciplines
59+
60+
LabAgent ships with 46 built-in measurement templates organized by scientific discipline:
61+
62+
| Discipline | Templates | Examples |
63+
|-----------|-----------|----------|
64+
| Electrical Characterization | 11 | IV, AHE, MR, RT, SOT, CV, Delta, High-R, Transfer, Output, Breakdown |
65+
| Thermoelectric | 2 | Seebeck, Thermal Conductivity |
66+
| Magnetic | 3 | Hall, FMR, Hysteresis |
67+
| Optical / Photonic | 2 | Photocurrent, Photoresponse |
68+
| Superconductivity | 2 | Tc, Jc |
69+
| Dielectric / Ferroelectric | 2 | P-E Loop, Pyroelectric |
70+
| Chemistry / Electrochemistry | 4 | Cyclic Voltammetry, EIS, Chronoamperometry, Potentiometry |
71+
| Biology / Biosensors | 2 | Impedance Biosensor, Cell Counting |
72+
| Materials Science / Environmental | 7 | Strain Gauge, Fatigue, Humidity Response, Gas Sensor, pH Calibration, DLTS, Capacitance-Frequency |
73+
| Semiconductor (additional) | 2 | Photo-IV, Tunneling |
74+
| Quantum Design Integration | 6 | PPMS-RT, PPMS-MR, PPMS-Hall, PPMS-HC, MPMS-MH, MPMS-MT |
75+
| General Purpose | 2 | Custom Sweep, Custom |
76+
77+
Each template defines safe defaults for its measurement type. For the full machine-readable catalog, see [CATALOG.md](https://github.com/Anai-Guo/LabAgent/blob/main/CATALOG.md).
78+
79+
## AI Parameter Optimization
80+
81+
When a `sample_description` is provided, the plan builder invokes an LLM to suggest optimized parameters for the specific material. The AI considers:
82+
83+
- Material properties (film thickness, composition, expected resistance range)
84+
- Instrument capabilities and typical operating ranges
85+
- Signal-to-noise tradeoffs (source current vs. sample damage risk)
86+
- Literature-typical ranges for the measurement type
87+
88+
The optimizer returns suggested overrides with reasoning. Critical safety constraint: **AI suggestions are clamped to template safety limits and can never exceed them.** If the AI suggests `max_current_a = 0.5` but the template maximum is `0.01`, the suggestion is silently dropped.
89+
90+
The override priority chain is:
91+
92+
1. Template defaults (base)
93+
2. AI-suggested optimizations (applied on top, clamped to safety limits)
94+
3. User overrides (highest priority, applied last)
95+
96+
## Role Validation
97+
98+
When instrument role assignments are provided, the plan builder validates them against the template's requirements. It warns about:
99+
100+
- **Missing roles** -- roles required by the template but not assigned to any instrument
101+
- **Extra roles** -- instruments assigned roles not used by this template
102+
103+
This ensures you have the right equipment connected before starting a measurement.
104+
105+
## Batch Campaigns
106+
107+
For systematic studies, the campaign system generates multiple measurement plans by sweeping parameters:
108+
109+
```bash
110+
# Create a campaign sweeping temperature and field
111+
labharness campaign AHE \
112+
--sweep "temperature=10,50,100,200,300" \
113+
--sweep "max_field_oe=1000,5000,10000" \
114+
--preview
115+
```
116+
117+
Each combination produces a validated plan, and the full set is saved as a campaign JSON file for sequential execution.

0 commit comments

Comments
 (0)