外观
自动生成
本页由 pnpm nodes 从 midscene-node-spec.web.md 生成，不要手改。改了 nodes/ 下的自定义节点后重新生成即可。
Midscene Test Node Spec
This document describes the Nodes available in YAML test files (.yaml or .yml) for the Midscene Test framework (@midscene/test). It lists the Nodes registered by the current Test Project, their inputs, and how to call them in test steps and lifecycle hooks.
Learn about the framework and YAML test authoring:
- Midscene Test overview: framework concepts and the role of Nodes.
- Create and use test projects: write YAML cases, run tests, and inspect results.
- Configure test projects: register Nodes, select case files, and manage runtime resources.
Available Nodes
This table lists all available Nodes. See Node Details below for each Node's usage and input schema.
| Node | Description |
|---|---|
| aiAct | Perform a natural-language task with a Midscene UI Agent. |
| aiAssert | Assert a natural-language condition with a Midscene UI Agent. |
| action | Call a platform or custom action. Its ActionSpace schema validates params when the action executes. |
| aiAsk | Run aiAsk with a Midscene UI Agent and store its value. |
| aiBoolean | Run aiBoolean with a Midscene UI Agent and store its value. |
| aiClearInput | Locate an input element and clear its value with a Midscene UI Agent. |
| aiDoubleClick | Locate and double-click an element with a Midscene UI Agent. |
| aiDragAndDrop | Locate source and destination elements, then drag the source to the destination. |
| aiHover | Locate and hover over an element with a Midscene UI Agent. |
| aiInput | Locate an input element and enter a value with a Midscene UI Agent. |
| aiKeyboardPress | Press a key or key combination, optionally targeting a located element. |
| aiLocate | Locate an element from a natural-language description and store the result. |
| aiLongPress | Locate and long-press an element with a Midscene UI Agent. |
| aiMiddleClick | Locate and middle-click an element with a Midscene UI Agent. |
| aiNumber | Run aiNumber with a Midscene UI Agent and store its value. |
| aiPinch | Perform a pinch-in or pinch-out gesture with a Midscene UI Agent. |
| aiQuery | Extract structured data from the current interface and store the result. |
| aiRightClick | Locate and right-click an element with a Midscene UI Agent. |
| aiScroll | Scroll the page or a located region with a Midscene UI Agent. |
| aiString | Run aiString with a Midscene UI Agent and store its value. |
| aiTap | Locate and tap an element with a Midscene UI Agent. |
| aiWaitFor | Wait until a natural-language condition is satisfied. |
| clearCookies | Clear all cookies from the current browser context, or only cookies matching name, domain, or path. |
| gotoUrl | Navigate the current browser Page to an absolute HTTP(S) URL or a path relative to testRunner.baseURL or the current page URL. |
| javascript | Evaluate JavaScript in the current interface and store the result. |
| recordToReport | Add text or screenshots to the current Midscene report. |
| runGherkinScenario | Execute a Gherkin scenario with the current Midscene UI Agent. |
| setAIContext | Set or remove Agent-level AI guidance for subsequent calls on the same Agent. |
| setCookies | Load cookies from an environment variable, configured profile, or browser storage-state file without persisting cookie values in workflow input or output. |
| setViewportSize | Set the current browser Page viewport size in CSS pixels and return the effective size. |
| sleep | Wait for a fixed number of milliseconds, recording standard UI snapshots and honoring cancellation. |
| todo.captureState | Read the current TodoMVC DOM without asserting it, for lifecycle diagnostics and history. |
| todo.expectState | Read TodoMVC from the DOM and require the ordered titles and completed states to match exactly. |
| todo.seed | Create an ordered list of Todos through the visible input after todo.setup. |
| todo.setup | Open React TodoMVC in All view, clear its storage, reload, and require an empty list before seeding. |
| wait | Wait for a fixed duration while honoring cancellation. |
Where to use these Nodes
Use these Nodes in Workflow Documents (.yaml or .yml files) selected by an Execution Project. Case file patterns and the config file path below are relative to this spec file. Case file patterns use glob syntax. A custom --config path does not change where case files are selected.
Projects created by midscene-test create select cases/**/*.{yaml,yml} by default, so you can write test cases in cases/example.yaml or subdirectories of cases/. If an Execution Project has no files configuration, Midscene Test searches **/*.{yaml,yml} recursively instead. Check projects[].files in your configuration before choosing a file location.
Config file: midscene.config.ts
Case files: cases/web/**/*.{yaml,yml} (Execution Project: web)
Files matching files.exclude are not selected. Midscene Test always ignores .git/, .midscene/, midscene_run/, and node_modules/ directories, including nested ones.
Call a Node in cases[].steps, or in the Workflow Document lifecycle steps beforeAll, beforeEach, afterEach, and afterAll. Root Nodes are shared defaults; projects[].nodes adds local Nodes and overrides same-named root Nodes only within that Execution Project. Each Execution Project supplies its own runtime environment.
Workflow Step Contract
Each step uses a node name as its key. The $ object contains engine metadata and is not part of the node input.
yaml
cases:
- name: Example case
steps:
- node.name:
prompt: Describe the task.
$:
timeout: 30000
continue-on-error: falseA Node may map a string step value to one declared input field. Each Node section states whether it supports this shorthand and names the target field.
Each Node's returned data is saved on that Step's result. Results are not named or automatically passed to later Steps. A failed Case does not stop later Cases by default; test.bail controls early stopping across the run.
Node Details
aiAct
Perform a natural-language task with a Midscene UI Agent.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"anyOf": [
{
"const": "unset"
},
{
"type": "boolean"
}
]
},
"effort": {
"enum": [
"fast",
"balance",
"deepThink"
]
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"fileChooserAllowedDir": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}aiAssert
Assert a natural-language condition with a Midscene UI Agent.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"message": {
"type": "string"
},
"options": {
"additionalProperties": false,
"properties": {
"context": {
"type": "string"
},
"domIncluded": {
"anyOf": [
{
"type": "boolean"
},
{
"const": "visible-only"
}
]
},
"keepRawResponse": {
"type": "boolean"
},
"screenshotIncluded": {
"type": "boolean"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}action
Call a platform or custom action. Its ActionSpace schema validates params when the action executes.
String shorthand: Not supported by this Node.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"params": {}
},
"required": [
"name"
],
"type": "object"
}aiAsk
Run aiAsk with a Midscene UI Agent and store its value.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"context": {
"type": "string"
},
"domIncluded": {
"anyOf": [
{
"type": "boolean"
},
{
"const": "visible-only"
}
]
},
"screenshotIncluded": {
"type": "boolean"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}aiBoolean
Run aiBoolean with a Midscene UI Agent and store its value.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"context": {
"type": "string"
},
"domIncluded": {
"anyOf": [
{
"type": "boolean"
},
{
"const": "visible-only"
}
]
},
"screenshotIncluded": {
"type": "boolean"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}aiClearInput
Locate an input element and clear its value with a Midscene UI Agent.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"type": "boolean"
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"xpath": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}aiDoubleClick
Locate and double-click an element with a Midscene UI Agent.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"type": "boolean"
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"xpath": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}aiDragAndDrop
Locate source and destination elements, then drag the source to the destination.
String shorthand: Not supported by this Node.
Input Schema
json
{
"$defs": {
"__schema0": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"from": {
"$ref": "#/$defs/__schema0"
},
"options": {
"additionalProperties": false,
"properties": {
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"type": "boolean"
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"xpath": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"to": {
"$ref": "#/$defs/__schema0"
}
},
"required": [
"from",
"to"
],
"type": "object"
}aiHover
Locate and hover over an element with a Midscene UI Agent.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"type": "boolean"
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"xpath": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}aiInput
Locate an input element and enter a value with a Midscene UI Agent.
String shorthand: Not supported by this Node.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"autoDismissKeyboard": {
"type": "boolean"
},
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"type": "boolean"
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"inputStrategy": {
"enum": [
"legacy",
"sequential",
"bulk"
]
},
"keyboardTypeDelay": {
"minimum": 0,
"type": "number"
},
"mode": {
"enum": [
"replace",
"clear",
"typeOnly",
"append"
]
},
"xpath": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
},
"value": {
"anyOf": [
{
"type": "string"
},
{
"type": "number"
}
]
}
},
"required": [
"prompt",
"value"
],
"type": "object"
}aiKeyboardPress
Press a key or key combination, optionally targeting a located element.
String shorthand: Maps to { keyName: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"keyName": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"options": {
"additionalProperties": false,
"properties": {
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"type": "boolean"
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"xpath": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"keyName"
],
"type": "object"
}aiLocate
Locate an element from a natural-language description and store the result.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"type": "boolean"
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"xpath": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}aiLongPress
Locate and long-press an element with a Midscene UI Agent.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"type": "boolean"
},
"duration": {
"minimum": 0,
"type": "number"
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"xpath": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}aiMiddleClick
Locate and middle-click an element with a Midscene UI Agent.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"type": "boolean"
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"xpath": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}aiNumber
Run aiNumber with a Midscene UI Agent and store its value.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"context": {
"type": "string"
},
"domIncluded": {
"anyOf": [
{
"type": "boolean"
},
{
"const": "visible-only"
}
]
},
"screenshotIncluded": {
"type": "boolean"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}aiPinch
Perform a pinch-in or pinch-out gesture with a Midscene UI Agent.
String shorthand: Not supported by this Node.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"direction": {
"enum": [
"in",
"out"
]
},
"options": {
"additionalProperties": false,
"properties": {
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"type": "boolean"
},
"distance": {
"type": "number"
},
"duration": {
"minimum": 0,
"type": "number"
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"xpath": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"direction"
],
"type": "object"
}aiQuery
Extract structured data from the current interface and store the result.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"context": {
"type": "string"
},
"domIncluded": {
"anyOf": [
{
"type": "boolean"
},
{
"const": "visible-only"
}
]
},
"screenshotIncluded": {
"type": "boolean"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": {
"type": "string"
},
"propertyNames": {
"type": "string"
},
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}aiRightClick
Locate and right-click an element with a Midscene UI Agent.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"type": "boolean"
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"xpath": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}aiScroll
Scroll the page or a located region with a Midscene UI Agent.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"type": "boolean"
},
"direction": {
"enum": [
"down",
"up",
"right",
"left"
]
},
"distance": {
"anyOf": [
{
"type": "number"
},
{
"type": "null"
}
]
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"scrollType": {
"enum": [
"singleAction",
"scrollToBottom",
"scrollToTop",
"scrollToRight",
"scrollToLeft",
"once",
"untilBottom",
"untilTop",
"untilRight",
"untilLeft"
]
},
"xpath": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [],
"type": "object"
}aiString
Run aiString with a Midscene UI Agent and store its value.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"context": {
"type": "string"
},
"domIncluded": {
"anyOf": [
{
"type": "boolean"
},
{
"const": "visible-only"
}
]
},
"screenshotIncluded": {
"type": "boolean"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}aiTap
Locate and tap an element with a Midscene UI Agent.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"type": "boolean"
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"xpath": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}aiWaitFor
Wait until a natural-language condition is satisfied.
String shorthand: Maps to { prompt: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"checkIntervalMs": {
"exclusiveMinimum": 0,
"type": "number"
},
"context": {
"type": "string"
},
"domIncluded": {
"anyOf": [
{
"type": "boolean"
},
{
"const": "visible-only"
}
]
},
"screenshotIncluded": {
"type": "boolean"
},
"timeoutMs": {
"exclusiveMinimum": 0,
"type": "number"
}
},
"required": [],
"type": "object"
},
"prompt": {
"anyOf": [
{
"format": "regex",
"pattern": "\\S",
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"convertHttpImage2Base64": {
"type": "boolean"
},
"images": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"name",
"url"
],
"type": "object"
},
"minItems": 1,
"type": "array"
},
"prompt": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"prompt"
],
"type": "object"
}
]
}
},
"required": [
"prompt"
],
"type": "object"
}clearCookies
Title: Clear browser cookies
Clear all cookies from the current browser context, or only cookies matching name, domain, or path.
String shorthand: Not supported by this Node.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"domain": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"name": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"path": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [],
"type": "object"
}gotoUrl
Title: Open a Web URL
Navigate the current browser Page to an absolute HTTP(S) URL or a path relative to testRunner.baseURL or the current page URL.
String shorthand: Maps to { url: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"timeoutMs": {
"default": 60000,
"exclusiveMinimum": 0,
"maximum": 600000,
"type": "number"
},
"url": {
"minLength": 1,
"type": "string"
},
"waitUntil": {
"default": "domcontentloaded",
"enum": [
"commit",
"domcontentloaded",
"load",
"networkidle",
"networkidle0",
"networkidle2"
]
}
},
"required": [
"url"
],
"type": "object"
}javascript
Evaluate JavaScript in the current interface and store the result.
String shorthand: Maps to { script: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"script": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"script"
],
"type": "object"
}recordToReport
Add text or screenshots to the current Midscene report.
String shorthand: Maps to { title: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"content": {
"type": "string"
},
"screenshotBase64": {
"type": "string"
},
"screenshots": {
"items": {
"additionalProperties": false,
"properties": {
"base64": {
"minLength": 1,
"type": "string"
},
"description": {
"type": "string"
}
},
"required": [
"base64"
],
"type": "object"
},
"minItems": 1,
"type": "array"
}
},
"required": [],
"type": "object"
},
"title": {
"type": "string"
}
},
"required": [],
"type": "object"
}runGherkinScenario
Execute a Gherkin scenario with the current Midscene UI Agent.
String shorthand: Maps to { scenario: value }.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"options": {
"additionalProperties": false,
"properties": {
"cacheable": {
"type": "boolean"
},
"context": {
"type": "string"
},
"deepLocate": {
"type": "boolean"
},
"deepThink": {
"anyOf": [
{
"const": "unset"
},
{
"type": "boolean"
}
]
},
"effort": {
"enum": [
"fast",
"balance",
"deepThink"
]
},
"fileChooserAccept": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
]
},
"fileChooserAllowedDir": {
"type": "string"
}
},
"required": [],
"type": "object"
},
"scenario": {
"format": "regex",
"pattern": "\\S",
"type": "string"
}
},
"required": [
"scenario"
],
"type": "object"
}setAIContext
Set or remove Agent-level AI guidance for subsequent calls on the same Agent.
String shorthand: Not supported by this Node.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"context": {
"type": "string"
},
"target": {
"enum": [
"default",
"aiAct",
"aiTap",
"aiRightClick",
"aiMiddleClick",
"aiDoubleClick",
"aiHover",
"aiInput",
"aiKeyboardPress",
"aiScroll",
"aiPinch",
"aiLongPress",
"aiClearInput",
"aiLocate",
"aiQuery",
"aiBoolean",
"aiNumber",
"aiString",
"aiAsk",
"aiAssert",
"aiWaitFor"
]
}
},
"required": [
"target"
],
"type": "object"
}setCookies
Title: Set browser cookies
Load cookies from an environment variable, configured profile, or browser storage-state file without persisting cookie values in workflow input or output.
String shorthand: Not supported by this Node.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"cookiesEnv": {
"format": "regex",
"pattern": "^[A-Za-z_][A-Za-z0-9_]*$",
"type": "string"
},
"profile": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"storageStatePath": {
"format": "regex",
"pattern": "\\S",
"type": "string"
},
"url": {
"format": "uri",
"type": "string"
}
},
"required": [],
"type": "object"
}setViewportSize
Title: Set the browser viewport size
Set the current browser Page viewport size in CSS pixels and return the effective size.
String shorthand: Not supported by this Node.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"height": {
"exclusiveMinimum": 0,
"maximum": 9007199254740991,
"type": "integer"
},
"width": {
"exclusiveMinimum": 0,
"maximum": 9007199254740991,
"type": "integer"
}
},
"required": [
"width",
"height"
],
"type": "object"
}sleep
Wait for a fixed number of milliseconds, recording standard UI snapshots and honoring cancellation.
String shorthand: Not supported by this Node.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"ms": {
"exclusiveMinimum": 0,
"type": "number"
}
},
"required": [
"ms"
],
"type": "object"
}todo.captureState
Title: Capture the current Todo state
Read the current TodoMVC DOM without asserting it, for lifecycle diagnostics and history.
String shorthand: Not supported by this Node.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {},
"required": [],
"type": "object"
}todo.expectState
Title: Expect the exact Todo state
Read TodoMVC from the DOM and require the ordered titles and completed states to match exactly.
String shorthand: Not supported by this Node.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"items": {
"items": {
"additionalProperties": false,
"properties": {
"completed": {
"type": "boolean"
},
"title": {
"minLength": 1,
"type": "string"
}
},
"required": [
"title",
"completed"
],
"type": "object"
},
"type": "array"
}
},
"required": [
"items"
],
"type": "object"
}todo.seed
Title: Seed the Todo list
Create an ordered list of Todos through the visible input after todo.setup.
String shorthand: Not supported by this Node.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"items": {
"items": {
"minLength": 1,
"type": "string"
},
"minItems": 1,
"type": "array"
}
},
"required": [
"items"
],
"type": "object"
}todo.setup
Title: Prepare an empty TodoMVC page
Open React TodoMVC in All view, clear its storage, reload, and require an empty list before seeding.
String shorthand: Not supported by this Node.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {},
"required": [],
"type": "object"
}wait
Wait for a fixed duration while honoring cancellation.
String shorthand: Not supported by this Node.
Input Schema
json
{
"$schema": "https://json-schema.org/draft-2020-12/schema",
"additionalProperties": false,
"properties": {
"duration": {
"exclusiveMinimum": 0,
"type": "number"
},
"unit": {
"default": "ms",
"enum": [
"ms",
"s",
"min"
]
}
},
"required": [
"duration"
],
"type": "object"
}