a2nr
/
amr-ros-k4
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Enhance Blockly UI with new client-side blocks and HMI widgets 

- Removed Blockly UI Enhancement from potential enhancements in README.md.
- Added client-side statement blocks that directly call JavaScript functions without needing Python handlers.
- Introduced HMI widget blocks for real-time updates in the HMI panel, including LED, Number, Text, and Gauge widgets.
- Updated BLOCKS.md to include detailed descriptions and examples for new block types and their execution models.
- Revised README.md to reflect changes in async code generation and debug engine functionalities.
- Modified workspace.json to include new HMI widgets and adjust existing block configurations.
master
a2nr 2026-03-18 09:03:02 +07:00
parent f2c482fe6e
commit 086e5dce0c
6 changed files with 741 additions and 133 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
DOCUMENTATION.md
View File

@ -9,12 +9,12 @@ See [readme.md](readme.md) for project overview and status.
| Topic | File | | Topic | File |
|---|---| |---|---|
| System architecture & Blockly–ROS2 integration flow | [docs/architecture.md](docs/architecture.md) | | System architecture, HMI Panel & concurrent execution | [docs/architecture.md](docs/architecture.md) |
| Installation, directory structure & running | [docs/installation.md](docs/installation.md) | | Installation, directory structure & running | [docs/installation.md](docs/installation.md) |
| Troubleshooting & known issues | [docs/troubleshooting.md](docs/troubleshooting.md) | | Troubleshooting & known issues | [docs/troubleshooting.md](docs/troubleshooting.md) |
| Guide: adding a new ROS2 package | [docs/ros2-package-guide.md](docs/ros2-package-guide.md) | | Guide: adding a new ROS2 package | [docs/ros2-package-guide.md](docs/ros2-package-guide.md) |
| `blockly_app` — file reference | [src/blockly_app/README.md](src/blockly_app/README.md) | | `blockly_app` — file reference (incl. HMI core modules) | [src/blockly_app/README.md](src/blockly_app/README.md) |
| `blockly_app` — creating custom blocks (full guide + reference) | [src/blockly_app/BLOCKS.md](src/blockly_app/BLOCKS.md) | | `blockly_app` — creating blocks (ROS2 action + client-side + HMI) | [src/blockly_app/BLOCKS.md](src/blockly_app/BLOCKS.md) |
| `blockly_executor` — file reference, handlers & testing guide | [src/blockly_executor/README.md](src/blockly_executor/README.md) | | `blockly_executor` — file reference, handlers & testing guide | [src/blockly_executor/README.md](src/blockly_executor/README.md) |
| `blockly_interfaces` — ROS2 action & message interfaces | [src/blockly_interfaces/README.md](src/blockly_interfaces/README.md) | | `blockly_interfaces` — ROS2 action & message interfaces | [src/blockly_interfaces/README.md](src/blockly_interfaces/README.md) |
| `gpio_node` — Raspberry Pi GPIO node (C++, libgpiod) | [src/gpio_node/README.md](src/gpio_node/README.md) | | `gpio_node` — Raspberry Pi GPIO node (C++, libgpiod) | [src/gpio_node/README.md](src/gpio_node/README.md) |

176
docs/architecture.md
View File

@ -18,7 +18,10 @@
│ │ variables (native)│ When encountering a robot │ │ │ variables (native)│ When encountering a robot │
│ │ • Block highlighting│ action block, Blockly calls │ │ │ • Block highlighting│ action block, Blockly calls │
│ │ during execution │ Python and WAITS for result. │ │ │ during execution │ Python and WAITS for result. │
│ │ │ │ │ │ • HMI Panel (right) │ │
│ │ LED/Number/Text/ │ Client-side blocks (print, HMI) │
│ │ Gauge widgets │ call JS functions directly — no │
│ │ │ ROS2 round-trip. │
│ │ [Run] [Stop] │ │ │ │ [Run] [Stop] │ │
│ └──────────┬───────────┘ │ │ └──────────┬───────────┘ │
│ │ JS ↔ Python bridge (pywebview API) │ │ │ JS ↔ Python bridge (pywebview API) │
@ -115,9 +118,125 @@ This interface is **generic by design** — adding new commands never requires m
--- ---
### 2.4 HMI Panel — LabVIEW-style Front Panel
Terinspirasi oleh LabVIEW yang memiliki **Front Panel** (controls & indicators) dan **Block Diagram** (visual programming). Blockly sudah menjadi "Block Diagram". HMI Panel adalah "Front Panel" — menampilkan widget indikator (LED, Number, Text, Gauge) yang dikontrol programmatically dari generated code.
```
┌────────────────────────────────────────────────────────────────────────┐
│ Toolbar: [Run] [Debug] [Step] [Stop] [Blocks|Code] [Save] [Open]│
├──────────────────────────┬──────────┬──────────────────────────────────┤
│ │ drag ↔ │ │
│ Blockly Workspace │ divider │ HMI Panel (gridstack.js) │
│ │ │ │
│ ┌─────────────────────┐ │ │ ┌──────┐ ┌──────────────────┐ │
│ │ Main Program │ │ │ │ LED1 │ │ Heading [rad] │ │
│ │ set [odom] to ... │ │ │ │ * ON │ │ 1.57 │ │
│ │ digital_out(17,1) │ │ │ └──────┘ └──────────────────┘ │
│ │ delay(1000) │ │ │ ┌──────────────────────────┐ │
│ └─────────────────────┘ │ │ │ Speed [cm/s] │ │
│ ┌─────────────────────┐ │ │ │ 42.5 │ │
│ │ HMI Program │ │ │ └──────────────────────────┘ │
│ │ HMI LED "LED1" ... │ │ │ ┌──────────────────────────┐ │
│ │ HMI Number "Hd".. │ │ │ │ Battery ### 72% │ │
│ └─────────────────────┘ │ │ └──────────────────────────┘ │
│ │ │ │
├──────────────────────────┴──────────┴──────────────────────────────────┤
│ drag ↕ divider │
├────────────────────────────────────────────────────────────────────────┤
│ Console: === Program started === │
│ USER LOG >>> Hello World │
│ === Program completed === │
└────────────────────────────────────────────────────────────────────────┘
```
**Key components:**
| Module | File | Fungsi |
|--------|------|--------|
| HMI Manager | `core/hmi-manager.js` | Global `HMI` object — `setLED()`, `setNumber()`, `setText()`, `setGauge()`, `clearAll()`. GridStack integration, layout serialization, mode management (design/runtime). |
| HMI Preview | `core/hmi-preview.js` | Workspace change listener — widgets appear/disappear saat block di-place/delete (design-time preview). Reconcile function handles undo/redo edge cases. |
| Resizable Panels | `core/resizable-panels.js` | Drag-to-resize dividers: vertical (Blockly↔HMI) dan horizontal (workspace↔console). Auto-resize Blockly canvas via `Blockly.svgResize()`. |
**Two modes:**
- **Design mode**: Grid unlocked (drag/resize widgets), preview values, dimmed appearance. Active saat tidak ada program berjalan.
- **Runtime mode**: Grid locked, live values dari running code, bright appearance. Active saat program berjalan.
**Widget types:**
| Widget | JS API | Fungsi |
|--------|--------|--------|
| LED | `HMI.setLED(name, state, color)` | On/off indicator with configurable color |
| Number | `HMI.setNumber(name, value, unit)` | Numeric display with unit label |
| Text | `HMI.setText(name, text)` | Text string display |
| Gauge | `HMI.setGauge(name, value, min, max)` | Horizontal bar gauge with range |
### 2.5 Concurrent Execution — Main Program + HMI Program
Saat workspace memiliki **dua program block** (`main_program` + `main_hmi_program`), keduanya berjalan bersamaan via `Promise.all()`.
```
generateCode(workspace)
|
v
Returns { definitions, mainCode, hmiCode }
|
v
debug-engine.js detects structured result -> _runConcurrent()
|
v
Single execution with shared scope:
+----------------------------------------------------------+
| (async function() { |
| // -- Shared definitions (variables, functions) ----- |
| var myVar; |
| async function myFunc() { ... } |
| |
| var _main = (async function() { |
| // Main program -- has highlightBlock (visual+stop) |
| await highlightBlock('...'); |
| await executeAction('digital_out', {...}); |
| })(); |
| |
| var _hmi = (async function() { |
| // HMI program -- highlightBlock shadowed to no-op |
| var highlightBlock = async function() { |
| if (debugState.stopRequested) throw ...; |
| }; |
| while (!debugState.stopRequested) { |
| HMI.setLED('LED1', true, '#4caf50'); |
| HMI.setNumber('X', myVar); |
| await new Promise(r => setTimeout(r, 50)); //20Hz |
| } |
| })(); |
| |
| await _main; // main drives completion|
| debugState.stopRequested = true; // signal HMI to stop|
| await _hmi; // wait for HMI cleanup |
| })() |
+----------------------------------------------------------+
```
**Design decisions:**
- **Shared scope**: Definitions (variables, functions) ada di outer IIFE — kedua program share via closure. Variable yang diubah di main langsung terlihat di HMI.
- **Main drives completion**: Saat main selesai, `stopRequested = true` -> HMI loop keluar.
- **HMI full speed**: `highlightBlock` di-shadow ke async no-op (hanya check stop). Tidak ada visual delay.
- **Debug mode**: Hanya main program yang memiliki stepping/breakpoints. HMI berjalan tanpa interupsi.
### 2.6 Two Block Execution Models
Blocks terbagi dua berdasarkan cara eksekusi:
| Model | Blocks | Mechanism | Latency |
|-------|--------|-----------|---------|
| **ROS2 action** | `digitalOut`, `digitalIn`, `delay`, `pwmWrite`, `odometryRead` | `await executeAction()` -> Python -> ROS2 -> handler -> result | Network round-trip (~10-100ms) |
| **Client-side** | `print`, `hmiSetLed`, `hmiSetNumber`, `hmiSetText`, `hmiSetGauge` | Direct JS function call (`consoleLog()`, `HMI.set*()`) | Instant (~0ms) |
Client-side blocks **tidak membutuhkan Python handler** — generator langsung memanggil fungsi JavaScript global. Ini membuat HMI responsif karena tidak ada overhead jaringan.
--- ---
## 8. Blockly–ROS2 Integration Flow ## 8. Blockly-ROS2 Integration Flow
### 8.1 End-to-End Execution Flow ### 8.1 End-to-End Execution Flow
@ -183,11 +302,23 @@ User presses [Run]
Each custom block has a **code generator** defined in its block file (e.g., [`blocks/digitalOut.js`](../src/blockly_app/blockly_app/ui/blockly/blocks/digitalOut.js)) that produces JavaScript code. For example, the `digitalOut` block with gpio=17 and state=true generates: Each custom block has a **code generator** defined in its block file (e.g., [`blocks/digitalOut.js`](../src/blockly_app/blockly_app/ui/blockly/blocks/digitalOut.js)) that produces JavaScript code. For example, the `digitalOut` block with gpio=17 and state=true generates:
```javascript ```javascript
highlightBlock('block_abc123'); await highlightBlock('block_abc123');
await executeAction('digital_out', { gpio: '17', state: 'true' }); await executeAction('digital_out', { gpio: '17', state: 'true' });
``` ```
Native Blockly blocks (loops, conditionals, variables) use Blockly's built-in JavaScript generators. Client-side blocks (print, HMI) generate direct JS function calls instead of `executeAction`:
```javascript
await highlightBlock('block_def456');
consoleLog(String(' USER LOG >>> ' + myVar), 'print');
await highlightBlock('block_ghi789');
HMI.setLED('LED1', Boolean(true), '#4caf50');
```
`generateCode(ws)` (in [`async-procedures.js`](../src/blockly_app/blockly_app/ui/blockly/core/async-procedures.js)) replaces direct `workspaceToCode()`. When a `main_program` block exists, only function definitions + main body are generated. See §8.6 for full details.
Native Blockly blocks (loops, conditionals) use Blockly's built-in generators. `variables_set` is overridden to add `highlightBlock()` for debug support.
### 8.3 pywebview Bridge Mechanism ### 8.3 pywebview Bridge Mechanism
@ -226,10 +357,37 @@ def _wait_for_future(future, timeout_sec=30.0):
When Debug Mode is enabled: When Debug Mode is enabled:
1. [`runDebug()`](../src/blockly_app/blockly_app/ui/blockly/core/debug-engine.js:87) wraps `executeAction` with breakpoint checking 1. [`runDebug()`](../src/blockly_app/blockly_app/ui/blockly/core/debug-engine.js:200) wraps `executeAction` with breakpoint checking
2. Before each action, it checks if `debugState.currentBlockId` is in `activeBreakpoints` 2. `highlightBlock()` is overridden to an async version that can pause execution
3. If a breakpoint is hit, execution pauses via a `Promise` that only resolves when the user clicks Step Over/Step Into 3. All pause logic lives in the async `highlightBlock()` override — this is the single pause point
4. A 300ms delay is added between blocks for visual feedback 4. Pause conditions: first block, breakpoints, step-over/step-into boundaries
5. Stop sets `stopRequested = true` and resolves any pending pause Promise, causing the next `executeAction` call to throw `'STOP_EXECUTION'` 5. A 300ms delay is added between blocks for visual feedback (auto-run mode)
6. Stop sets `stopRequested = true` and resolves any pending pause Promise, causing the next `highlightBlock` call to throw `'STOP_EXECUTION'`
**Call depth tracking** (for Step Over):
- `enterFunction()` / `exitFunction()` injected around procedure calls by `async-procedures.js`
- Step Over skips blocks deeper than `stepStartDepth` (function bodies)
- Step Into pauses at every `highlightBlock()` regardless of depth
**Concurrent debug mode** (main + HMI programs):
- Main program: full debug stepping (breakpoints, step over/into, visual highlighting)
- HMI program: runs uninterrupted at full speed — `highlightBlock` shadowed to async stop check
- Both share variable scope via closure in single outer IIFE
### 8.6 Code Generation Pipeline — `generateCode(ws)`
[`async-procedures.js`](../src/blockly_app/blockly_app/ui/blockly/core/async-procedures.js) provides `generateCode(ws)` which replaces `workspaceToCode()`:
| Workspace state | Return value | Execution path |
|----------------|-------------|----------------|
| No `main_program` block | Plain string (backward compatible) | `_runSingle()` |
| `main_program` only | Plain string (definitions + main body) | `_runSingle()` |
| `main_program` + `main_hmi_program` | `{ definitions, mainCode, hmiCode }` | `_runConcurrent()` |
**Built-in generator overrides** in `async-procedures.js`:
- `procedures_defreturn` / `procedures_defnoreturn` → generates `async function` instead of `function`
- `procedures_callreturn` → wraps in async IIFE with `enterFunction()` / `exitFunction()`
- `procedures_callnoreturn` → adds `await highlightBlock()` + call depth tracking
- `variables_set` → adds `await highlightBlock()` before variable assignment
--- ---

2
readme.md
View File

@ -36,7 +36,7 @@ jelaskan apa yang dimaksut untuk menyelesaikan task
# Potential Enhancements # Potential Enhancements
this list is short by priority this list is short by priority
- **Blockly UI Enhancement**: - **add more block in HMI**: i like to add block button, slider, and switch
- **Feasibility Study to implement Controller**: mobile robot need controller to move flawlesly. - **Feasibility Study to implement Controller**: mobile robot need controller to move flawlesly.
- **Launch files**: ROS2 launch files to start all nodes with one command includ node in raspberry pi - **Launch files**: ROS2 launch files to start all nodes with one command includ node in raspberry pi
- **Simulation**: Integrate with Gazebo/Isaac Sim for testing Kiwi Wheel kinematics before deploying to hardware - **Simulation**: Integrate with Gazebo/Isaac Sim for testing Kiwi Wheel kinematics before deploying to hardware

191
src/blockly_app/BLOCKS.md
View File

@ -899,7 +899,171 @@ BlockRegistry.register({
--- ---
### 7.15 `executeAction()` and `highlightBlock()` Reference ### 7.15 Template E — Client-side Statement Block (no Python handler)
Client-side blocks call JavaScript functions directly — no `executeAction()`, no Python handler, no ROS2 round-trip. Used for **print** (console output) and **HMI widgets** (LED, Number, Text, Gauge).
**Print block** — calls `consoleLog()` directly:
```js
// src/blockly_app/blockly_app/ui/blockly/blocks/print.js
BlockRegistry.register({
name: 'print',
category: 'Program',
categoryColor: '#FF9800',
color: '#FFCA28',
tooltip: 'Print a value to the console for debugging',
definition: {
init: function () {
this.appendValueInput('TEXT')
.appendField('Print');
this.setPreviousStatement(true, null);
this.setNextStatement(true, null);
this.setColour('#FFCA28');
this.setTooltip('Print a value to the console for debugging');
},
},
generator: function (block) {
var value = Blockly.JavaScript.valueToCode(
block, 'TEXT', Blockly.JavaScript.ORDER_ATOMIC
) || "''";
return (
"await highlightBlock('" + block.id + "');\n" +
"consoleLog(String( ' USER LOG >>> ' + " + value + "), 'print');\n"
// ^^^^^^^^^^^^^^^^ direct JS function call — no executeAction!
);
},
});
```
**No Python handler needed.** The block works entirely in the browser. Add the JS filename to `manifest.js` and it's done.
---
### 7.16 Template F — HMI Widget Block (client-side, design-time preview)
HMI blocks create/update widgets in the HMI Panel. They call `HMI.set*()` functions directly (client-side) and support design-time preview — widgets appear when blocks are placed, not just at runtime.
**HMI LED block:**
```js
// src/blockly_app/blockly_app/ui/blockly/blocks/hmiSetLed.js
BlockRegistry.register({
name: 'hmiSetLed',
category: 'HMI',
categoryColor: '#00BCD4',
color: '#00BCD4',
tooltip: 'Set an LED indicator on/off in the HMI panel',
definition: {
init: function () {
this.appendValueInput('STATE')
.appendField('HMI LED')
.appendField(new Blockly.FieldTextInput('LED1'), 'NAME')
.appendField('color')
.appendField(new Blockly.FieldDropdown([
['green', '#4caf50'],
['red', '#f44336'],
['yellow', '#ffeb3b'],
['blue', '#2196f3'],
['orange', '#ff9800'],
['white', '#ffffff'],
]), 'COLOR')
.setCheck('Boolean')
.appendField('state:');
this.setPreviousStatement(true, null);
this.setNextStatement(true, null);
this.setColour('#00BCD4');
this.setTooltip('Set an LED indicator on/off in the HMI panel');
},
},
generator: function (block) {
var name = block.getFieldValue('NAME');
var color = block.getFieldValue('COLOR');
var state = Blockly.JavaScript.valueToCode(
block, 'STATE', Blockly.JavaScript.ORDER_ATOMIC
) || 'false';
return (
"await highlightBlock('" + block.id + "');\n" +
"HMI.setLED('" + name + "', Boolean(" + state + "), '" + color + "');\n"
// ^^^^^^^^^^ direct call to HMI manager — no executeAction!
);
},
});
```
**HMI Number block** — numeric display with unit:
```js
generator: function (block) {
var name = block.getFieldValue('NAME');
var unit = block.getFieldValue('UNIT');
var value = Blockly.JavaScript.valueToCode(
block, 'VALUE', Blockly.JavaScript.ORDER_ATOMIC
) || '0';
return (
"await highlightBlock('" + block.id + "');\n" +
"HMI.setNumber('" + name + "', Number(" + value + "), '" + unit + "');\n"
);
},
```
**HMI Gauge block** — bar gauge with min/max:
```js
generator: function (block) {
var name = block.getFieldValue('NAME');
var min = block.getFieldValue('MIN');
var max = block.getFieldValue('MAX');
var value = Blockly.JavaScript.valueToCode(
block, 'VALUE', Blockly.JavaScript.ORDER_ATOMIC
) || '0';
return (
"await highlightBlock('" + block.id + "');\n" +
"HMI.setGauge('" + name + "', Number(" + value + "), " + min + ", " + max + ");\n"
);
},
```
**Key differences from ROS2 action blocks:**
- Generator calls `HMI.set*()` instead of `executeAction()`
- No Python handler file needed
- Widgets get design-time preview via `hmi-preview.js` (auto-detected by block type)
- All HMI blocks use `FieldTextInput('Name')` for widget identifier (`NAME` field)
- Category: `HMI`, color: `#00BCD4`
**Adding a new HMI widget type:**
1. Add render function `_renderFoo(body, widget)` in `core/hmi-manager.js`
2. Add public API method `setFoo(name, ...)` that stores state and calls `_scheduleRender(name)`
3. Add `case 'foo':` to the `_render()` switch
4. Create block file `blocks/hmiFoo.js` — generator calls `HMI.setFoo(...)`
5. Add to `manifest.js`
6. Add entry to `HMI_BLOCK_TYPES` in `core/hmi-preview.js` for design-time preview
---
### 7.17 Block Type Overview — All Categories
| Category | Blocks | Execution Model |
|----------|--------|-----------------|
| **Program** | `main_program`, `main_hmi_program`, `print` | Entry points / client-side |
| **Robot** | `digitalOut`, `digitalIn`, `delay`, `pwmWrite`, `odometryRead`, `odometryGet` | ROS2 action (except `odometryGet`) |
| **HMI** | `hmiSetLed`, `hmiSetNumber`, `hmiSetText`, `hmiSetGauge` | Client-side (`HMI.set*()`) |
| **Logic** | `controls_if`, `logic_compare`, `logic_operation`, `logic_boolean` | Built-in Blockly |
| **Loops** | `controls_repeat_ext`, `controls_whileUntil` | Built-in Blockly |
| **Math** | `math_number`, `math_arithmetic` | Built-in Blockly |
| **Text** | `text`, `text_join`, `text_length` | Built-in Blockly |
| **Variables** | `variables_set`, `variables_get` | Built-in Blockly (with highlight override) |
| **Functions** | `procedures_defnoreturn`, `procedures_callnoreturn`, etc. | Built-in Blockly (with async override) |
---
### 7.18 `executeAction()` and `highlightBlock()` Reference
Both functions are provided by [bridge.js](blockly_app/ui/blockly/core/bridge.js) and injected into the eval context by [debug-engine.js](blockly_app/ui/blockly/core/debug-engine.js). Both functions are provided by [bridge.js](blockly_app/ui/blockly/core/bridge.js) and injected into the eval context by [debug-engine.js](blockly_app/ui/blockly/core/debug-engine.js).
@ -932,7 +1096,7 @@ Visually highlights the currently executing block in the workspace. Call it **fi
--- ---
### 7.16 Quick Reference: Blockly Field Types ### 7.19 Quick Reference: Blockly Field Types
| Field | Constructor | Returns | Use Case | | Field | Constructor | Returns | Use Case |
|-------|-------------|---------|----------| |-------|-------------|---------|----------|
@ -948,7 +1112,7 @@ All field values retrieved via `block.getFieldValue('FIELD_NAME')` are **strings
--- ---
### 7.17 Quick Reference: Input and Connection Types ### 7.20 Quick Reference: Input and Connection Types
| Method | What it creates | When to use | | Method | What it creates | When to use |
|--------|----------------|-------------| |--------|----------------|-------------|
@ -963,7 +1127,7 @@ All field values retrieved via `block.getFieldValue('FIELD_NAME')` are **strings
--- ---
### 7.18 Naming Conventions ### 7.21 Naming Conventions
| Item | Convention | Example | | Item | Convention | Example |
|------|-----------|---------| |------|-----------|---------|
@ -976,11 +1140,14 @@ All field values retrieved via `block.getFieldValue('FIELD_NAME')` are **strings
--- ---
### 7.19 Step-by-Step Checklist — Adding a New Block ### 7.22 Step-by-Step Checklist — Adding a New Block
**Path A — ROS2 Action Block** (robot commands, sensor reads):
``` ```
1. Create src/blockly_app/…/ui/blockly/blocks/<name>.js 1. Create src/blockly_app/…/ui/blockly/blocks/<name>.js
└─ Call BlockRegistry.register({ name, category, color, definition, generator }) └─ Call BlockRegistry.register({ name, category, color, definition, generator })
└─ generator calls await executeAction('name', { key: 'value' })
└─ name must exactly match the Python @handler string └─ name must exactly match the Python @handler string
2. Edit src/blockly_app/…/ui/blockly/blocks/manifest.js 2. Edit src/blockly_app/…/ui/blockly/blocks/manifest.js
@ -998,6 +1165,20 @@ All field values retrieved via `block.getFieldValue('FIELD_NAME')` are **strings
pixi run test -- src/blockly_executor/test/test_block_<name>.py -v pixi run test -- src/blockly_executor/test/test_block_<name>.py -v
``` ```
**Path B — Client-side Block** (print, HMI, pure JS):
```
1. Create src/blockly_app/…/ui/blockly/blocks/<name>.js
└─ Call BlockRegistry.register({ name, category, color, definition, generator })
└─ generator calls JS function directly (consoleLog(), HMI.set*(), etc.)
└─ NO executeAction(), NO Python handler needed
2. Edit src/blockly_app/…/ui/blockly/blocks/manifest.js
└─ Add '<name>.js' to the BLOCK_FILES array
3. Test pixi run app — drag block, click Run (no executor needed for client-side blocks)
```
No changes needed to `index.html`, `executor_node.py`, `handlers/__init__.py`, or `BlocklyAction.action`. No changes needed to `index.html`, `executor_node.py`, `handlers/__init__.py`, or `BlocklyAction.action`.

88
src/blockly_app/README.md
View File

@ -73,17 +73,34 @@ Both functions use `async/await` — they return after the file dialog closes an
**Purpose:** Provides `executeAction(command, params)` which calls Python via the pywebview JS-to-Python bridge. Falls back to a mock when running outside pywebview (browser dev). **Purpose:** Provides `executeAction(command, params)` which calls Python via the pywebview JS-to-Python bridge. Falls back to a mock when running outside pywebview (browser dev).
#### [`blockly_app/ui/blockly/core/async-procedures.js`](blockly_app/ui/blockly/core/async-procedures.js) — Async Code Generation
**Purpose:** Overrides Blockly's built-in procedure generators for async/await support, overrides `variables_set` for debug highlight, and provides `generateCode(ws)` which replaces `workspaceToCode()`.
| Component | Description |
|---|---|
| `procedures_defreturn` override | Generates `async function` instead of `function` |
| `procedures_callreturn` override | Wraps in async IIFE with `enterFunction()` / `exitFunction()` for call depth tracking |
| `procedures_callnoreturn` override | Adds `await highlightBlock()` + call depth tracking |
| `variables_set` override | Adds `await highlightBlock()` before variable assignment for debug support |
| `generateCode(ws)` | Main-block-aware code generation. Returns plain string or `{ definitions, mainCode, hmiCode }` for concurrent execution. |
#### [`blockly_app/ui/blockly/core/debug-engine.js`](blockly_app/ui/blockly/core/debug-engine.js) — Debug Engine #### [`blockly_app/ui/blockly/core/debug-engine.js`](blockly_app/ui/blockly/core/debug-engine.js) — Debug Engine
**Purpose:** Implements Run, Debug, Step Over, Step Into, and Stop functionality. **Purpose:** Implements Run, Debug, Step Over, Step Into, and Stop functionality. Supports single-program and concurrent (main+HMI) execution modes.
| Function | Description | | Function | Description |
|---|---| |---|---|
| `runProgram()` | Non-debug execution: wraps generated code in async function and eval()s it | | `runProgram()` | Non-debug execution: detects single vs concurrent mode, dispatches accordingly |
| `runDebug()` | Debug execution: wraps executeAction to check breakpoints and add delays | | `_runSingle(code)` | Single program execution (no HMI block present) |
| `stepOver()` | Resumes from pause, executes current block, pauses at next block | | `_runConcurrent(codeResult)` | Concurrent execution: main + HMI via `Promise.all()` with shared variable scope |
| `runDebug()` | Debug execution: detects single vs concurrent mode |
| `_runDebugSingle(code)` | Single program debug with breakpoints and stepping |
| `_runDebugConcurrent(codeResult)` | Concurrent debug: main has stepping, HMI runs uninterrupted |
| `stepOver()` | Resumes from pause, executes current block, pauses at next block at same call depth |
| `stepInto()` | Resumes from pause, pauses at very next highlightBlock call | | `stepInto()` | Resumes from pause, pauses at very next highlightBlock call |
| `stopExecution()` | Sets stopRequested flag, resolves any pending pause Promise | | `continueExecution()` | Resumes and runs until next breakpoint |
| `stopExecution()` | Sets stopRequested flag, cancels in-flight actions, resolves pending pause Promise |
| `highlightBlock(blockId)` | Highlights the currently executing block in the workspace | | `highlightBlock(blockId)` | Highlights the currently executing block in the workspace |
#### [`blockly_app/ui/blockly/blocks/manifest.js`](blockly_app/ui/blockly/blocks/manifest.js) — Block Manifest #### [`blockly_app/ui/blockly/blocks/manifest.js`](blockly_app/ui/blockly/blocks/manifest.js) — Block Manifest
@ -94,8 +111,63 @@ Both functions use `async/await` — they return after the file dialog closes an
const BLOCK_FILES = ['digitalOut.js', 'digitalIn.js', 'delay.js']; const BLOCK_FILES = ['digitalOut.js', 'digitalIn.js', 'delay.js'];
``` ```
#### [`blockly_app/ui/blockly/blocks/digitalOut.js`](blockly_app/ui/blockly/blocks/digitalOut.js) — Example Block #### [`blockly_app/ui/blockly/core/hmi-manager.js`](blockly_app/ui/blockly/core/hmi-manager.js) — HMI State Manager
**Purpose:** Self-contained block definition. Contains both the visual appearance AND the code generator. **Purpose:** Global `HMI` object providing LabVIEW-style Front Panel with gridstack.js grid layout. Manages widget lifecycle, mode switching (design/runtime), and layout serialization.
Each block file calls `BlockRegistry.register()` with all metadata, so the toolbox is automatically generated. | Method | Description |
|---|---|
| `HMI.init()` | Initialize GridStack instance. Called once after DOM is ready. |
| `HMI.addWidget(name, type, config, blockId)` | Add widget to grid. Updates if exists. |
| `HMI.removeWidget(name)` | Remove widget from grid and DOM. |
| `HMI.setMode(mode)` | Switch between `'design'` (grid unlocked) and `'runtime'` (grid locked). |
| `HMI.setLED(name, state, color)` | Set LED on/off with color. Called from generated code. |
| `HMI.setNumber(name, value, unit)` | Display numeric value with unit label. |
| `HMI.setText(name, text)` | Display text string. |
| `HMI.setGauge(name, value, min, max)` | Display gauge bar with range. |
| `HMI.getLayout()` / `HMI.loadLayout(items)` | Serialize/restore grid positions for save/load. |
| `HMI.clearAll()` | Remove all widgets and reset grid. |
#### [`blockly_app/ui/blockly/core/hmi-preview.js`](blockly_app/ui/blockly/core/hmi-preview.js) — Design-time Widget Preview
**Purpose:** Workspace change listener that creates/removes HMI widgets as blocks are placed/deleted. Widgets appear in the HMI panel at design time, not just at runtime.
| Function | Description |
|---|---|
| `initHMIPreview(ws)` | Sets up workspace change listener. Tracks block-to-widget mapping. |
| `_handleCreate(event)` | Creates widget when HMI block is placed. |
| `_handleDelete(event)` | Removes widget when HMI block is deleted. |
| `_handleChange(event)` | Updates widget when block field changes (NAME, COLOR, UNIT, MIN, MAX). |
| `_reconcile()` | Debounced (100ms) full sync — catches undo/redo edge cases. |
| `window._hmiPreviewScan()` | Scans existing blocks after workspace import to recreate previews. |
#### [`blockly_app/ui/blockly/core/resizable-panels.js`](blockly_app/ui/blockly/core/resizable-panels.js) — Resizable Split Panels
**Purpose:** Drag-to-resize dividers between Blockly workspace, HMI panel, and console. Updates `#blockly-div` dimensions and calls `Blockly.svgResize()` on each move.
| Divider | Direction | Range |
|---------|-----------|-------|
| Vertical (Blockly↔HMI) | Horizontal drag | HMI 200px–50% viewport |
| Horizontal (workspace↔console) | Vertical drag | Console 80px–40% viewport |
#### [`blockly_app/ui/blockly/blocks/`](blockly_app/ui/blockly/blocks/) — Block Definitions
All block files registered in [`manifest.js`](blockly_app/ui/blockly/blocks/manifest.js):
| File | Category | Type | Execution Model |
|------|----------|------|-----------------|
| `mainProgram.js` | Program | Hat block | N/A (entry point) |
| `mainHmiProgram.js` | Program | Hat block | N/A (HMI entry point) |
| `print.js` | Program | Statement | Client-side (`consoleLog()`) |
| `digitalOut.js` | Robot | Statement | ROS2 action (`digital_out`) |
| `digitalIn.js` | Robot | Output | ROS2 action (`digital_in`) |
| `delay.js` | Robot | Statement | ROS2 action (`delay`) |
| `pwmWrite.js` | Robot | Statement | ROS2 action (`pwm_write`) |
| `odometryRead.js` | Robot | Output | ROS2 action (`odometry_read`) |
| `odometryGet.js` | Robot | Output | Client-side (JSON field extract) |
| `hmiSetLed.js` | HMI | Statement | Client-side (`HMI.setLED()`) |
| `hmiSetNumber.js` | HMI | Statement | Client-side (`HMI.setNumber()`) |
| `hmiSetText.js` | HMI | Statement | Client-side (`HMI.setText()`) |
| `hmiSetGauge.js` | HMI | Statement | Client-side (`HMI.setGauge()`) |
Each block file calls `BlockRegistry.register()` with all metadata, so the toolbox is automatically generated. See [BLOCKS.md](BLOCKS.md) for the full guide to creating blocks.

411
workspace.json
View File

@ -48,156 +48,172 @@
}, },
"next": { "next": {
"block": { "block": {
"type": "digitalOut", "type": "variables_set",
"id": "Cz1MB5`}~cPRiZhh$/P9", "id": "qzDOA-}@MU,]iqj`er}D",
"fields": { "fields": {
"GPIO": 17 "VAR": {
"id": "[g,f6Mp!O$eZPCFs0U[H"
}
}, },
"inputs": { "inputs": {
"digitalOut": { "VALUE": {
"block": { "block": {
"type": "procedures_callreturn", "type": "math_number",
"id": "U|7!ynVp5Z.nwD_`4=Z,", "id": "39KE0jn61fTR1Qp4)d57",
"extraState": { "fields": {
"name": "foo", "NUM": 500
"params": [
"logic"
]
},
"inputs": {
"ARG0": {
"block": {
"type": "logic_boolean",
"id": "HXRaHRaPE)[G3WGmOi-T",
"fields": {
"BOOL": "TRUE"
}
}
}
} }
} }
} }
}, },
"next": { "next": {
"block": { "block": {
"type": "print", "type": "digitalOut",
"id": "#iq/$E*oB0V`%5O}O+2!", "id": "Cz1MB5`}~cPRiZhh$/P9",
"fields": {
"GPIO": 17
},
"inputs": { "inputs": {
"TEXT": { "digitalOut": {
"block": { "block": {
"type": "text", "type": "procedures_callreturn",
"id": "^ZnZat6;hW,I`H7H15^^", "id": "U|7!ynVp5Z.nwD_`4=Z,",
"fields": { "extraState": {
"TEXT": "cek" "name": "foo",
"params": [
"logic"
]
},
"inputs": {
"ARG0": {
"block": {
"type": "logic_boolean",
"id": "HXRaHRaPE)[G3WGmOi-T",
"fields": {
"BOOL": "TRUE"
}
}
}
} }
} }
} }
}, },
"next": { "next": {
"block": { "block": {
"type": "delay", "type": "variables_set",
"id": "IXp?_lac7+V*GG!lW{]0", "id": "5WG|J40M0!L^mM5x_2@N",
"fields": { "fields": {
"DURATION_MS": 1000 "VAR": {
"id": "xjE5n-FetBK*,)qj?pyn"
}
},
"inputs": {
"VALUE": {
"block": {
"type": "text",
"id": "^ZnZat6;hW,I`H7H15^^",
"fields": {
"TEXT": "cek"
}
}
}
}, },
"next": { "next": {
"block": { "block": {
"type": "variables_set", "type": "print",
"id": "1@OTYSIBwU0xvg(2?-y;", "id": "#iq/$E*oB0V`%5O}O+2!",
"fields": {
"VAR": {
"id": "C_:{ED@bJimgLzEmC6(`"
}
},
"inputs": { "inputs": {
"VALUE": { "TEXT": {
"block": { "block": {
"type": "digitalIn", "type": "variables_get",
"id": "DJpFh.6H~L9fX2V4SDJd", "id": "4[hYi)09%}I%Pj4}fhdb",
"fields": { "fields": {
"GPIO": 16 "VAR": {
"id": "xjE5n-FetBK*,)qj?pyn"
}
} }
} }
} }
}, },
"next": { "next": {
"block": { "block": {
"type": "variables_set", "type": "delay",
"id": "7v:u)fVh$GAwIhdwtH-i", "id": "IXp?_lac7+V*GG!lW{]0",
"fields": { "fields": {
"VAR": { "DURATION_MS": 1000
"id": "C_:{ED@bJimgLzEmC6(`"
}
},
"inputs": {
"VALUE": {
"block": {
"type": "math_number",
"id": "x.-GBy*1d`n}m-;O;Nre",
"fields": {
"NUM": 1
}
}
}
}, },
"next": { "next": {
"block": { "block": {
"type": "print", "type": "variables_set",
"id": "iB`R-:UTF#c3EXAU.Qh-", "id": "1@OTYSIBwU0xvg(2?-y;",
"fields": {
"VAR": {
"id": "C_:{ED@bJimgLzEmC6(`"
}
},
"inputs": { "inputs": {
"TEXT": { "VALUE": {
"block": { "block": {
"type": "text", "type": "digitalIn",
"id": "oA52{ydq^ZsJF:O@NU,d", "id": "DJpFh.6H~L9fX2V4SDJd",
"fields": { "fields": {
"TEXT": "cok" "GPIO": 16
} }
} }
} }
}, },
"next": { "next": {
"block": { "block": {
"type": "delay", "type": "variables_set",
"id": "VQy`Sl3]ey49sP%+N6$R", "id": "7v:u)fVh$GAwIhdwtH-i",
"fields": { "fields": {
"DURATION_MS": 1000 "VAR": {
"id": "C_:{ED@bJimgLzEmC6(`"
}
},
"inputs": {
"VALUE": {
"block": {
"type": "math_number",
"id": "x.-GBy*1d`n}m-;O;Nre",
"fields": {
"NUM": 1
}
}
}
}, },
"next": { "next": {
"block": { "block": {
"type": "variables_set", "type": "variables_set",
"id": "i|LkDgVjImZd2}owndlz", "id": "toVbALp.{C5C+{9GMK%e",
"fields": { "fields": {
"VAR": { "VAR": {
"id": "[g,f6Mp!O$eZPCFs0U[H" "id": "xjE5n-FetBK*,)qj?pyn"
} }
}, },
"inputs": { "inputs": {
"VALUE": { "VALUE": {
"block": { "block": {
"type": "math_number", "type": "text",
"id": "[C@fwlekugl(`pi1b;1(", "id": "oA52{ydq^ZsJF:O@NU,d",
"fields": { "fields": {
"NUM": 100 "TEXT": "cok"
} }
} }
} }
}, },
"next": { "next": {
"block": { "block": {
"type": "pwmWrite", "type": "print",
"id": "Ezn#r.|lvDj5{Q1-C:E$", "id": "iB`R-:UTF#c3EXAU.Qh-",
"fields": {
"ADDRESS": "64",
"CHANNEL": 0
},
"inputs": { "inputs": {
"PWM_VALUE": { "TEXT": {
"block": { "block": {
"type": "variables_get", "type": "variables_get",
"id": "OkA-}PRPzgi;[I)@vcG$", "id": "+wF~Y65u+aNA.ZX3!^Ra",
"fields": { "fields": {
"VAR": { "VAR": {
"id": "[g,f6Mp!O$eZPCFs0U[H" "id": "xjE5n-FetBK*,)qj?pyn"
} }
} }
} }
@ -206,49 +222,125 @@
"next": { "next": {
"block": { "block": {
"type": "variables_set", "type": "variables_set",
"id": "m{+MhlXz-1tpl`mPPBh5", "id": "E8`?q[Z3*IpN3,52Y=jA",
"fields": { "fields": {
"VAR": { "VAR": {
"id": "ju{xs[rjZumqS87$0nhu" "id": "[g,f6Mp!O$eZPCFs0U[H"
} }
}, },
"inputs": { "inputs": {
"VALUE": { "VALUE": {
"block": { "block": {
"type": "odometryRead", "type": "math_number",
"id": "v=Js89HC8D0UUA.-pN[q", "id": "iOJP=T-{O5h_^;{Yz?%(",
"fields": { "fields": {
"SOURCE": "encoder" "NUM": 3000
} }
} }
} }
}, },
"next": { "next": {
"block": { "block": {
"type": "variables_set", "type": "delay",
"id": "]LcUOwlc-y=`.e?EVgTa", "id": "VQy`Sl3]ey49sP%+N6$R",
"fields": { "fields": {
"VAR": { "DURATION_MS": 1000
"id": "Ug!mIa*[PnsL?H#9Ar*G"
}
}, },
"inputs": { "next": {
"VALUE": { "block": {
"block": { "type": "variables_set",
"type": "odometryGet", "id": "i|LkDgVjImZd2}owndlz",
"id": "VG2Q/8?zcyU}s4!W;V/M", "fields": {
"fields": { "VAR": {
"FIELD": "x" "id": "[g,f6Mp!O$eZPCFs0U[H"
}, }
"inputs": { },
"VAR": { "inputs": {
"VALUE": {
"block": {
"type": "math_number",
"id": "[C@fwlekugl(`pi1b;1(",
"fields": {
"NUM": 100
}
}
}
},
"next": {
"block": {
"type": "pwmWrite",
"id": "Ezn#r.|lvDj5{Q1-C:E$",
"fields": {
"ADDRESS": "64",
"CHANNEL": 0
},
"inputs": {
"PWM_VALUE": {
"block": {
"type": "variables_get",
"id": "OkA-}PRPzgi;[I)@vcG$",
"fields": {
"VAR": {
"id": "[g,f6Mp!O$eZPCFs0U[H"
}
}
}
}
},
"next": {
"block": { "block": {
"type": "variables_get", "type": "variables_set",
"id": "^AW6|z21?ycRyzJ2y5u9", "id": "m{+MhlXz-1tpl`mPPBh5",
"fields": { "fields": {
"VAR": { "VAR": {
"id": "ju{xs[rjZumqS87$0nhu" "id": "ju{xs[rjZumqS87$0nhu"
} }
},
"inputs": {
"VALUE": {
"block": {
"type": "odometryRead",
"id": "v=Js89HC8D0UUA.-pN[q",
"fields": {
"SOURCE": "encoder"
}
}
}
},
"next": {
"block": {
"type": "variables_set",
"id": "]LcUOwlc-y=`.e?EVgTa",
"fields": {
"VAR": {
"id": "Ug!mIa*[PnsL?H#9Ar*G"
}
},
"inputs": {
"VALUE": {
"block": {
"type": "odometryGet",
"id": "VG2Q/8?zcyU}s4!W;V/M",
"fields": {
"FIELD": "x"
},
"inputs": {
"VAR": {
"block": {
"type": "variables_get",
"id": "^AW6|z21?ycRyzJ2y5u9",
"fields": {
"VAR": {
"id": "ju{xs[rjZumqS87$0nhu"
}
}
}
}
}
}
}
}
}
} }
} }
} }
@ -355,8 +447,8 @@
{ {
"type": "main_hmi_program", "type": "main_hmi_program",
"id": "l!A!vr#-Z*nShL9rf[fa", "id": "l!A!vr#-Z*nShL9rf[fa",
"x": 690, "x": 370,
"y": 30, "y": 130,
"inputs": { "inputs": {
"BODY": { "BODY": {
"block": { "block": {
@ -398,6 +490,75 @@
} }
} }
} }
},
"next": {
"block": {
"type": "hmiSetGauge",
"id": "FeWSZv_C@ci}5(#:EV.A",
"fields": {
"NAME": "Gauge1",
"MIN": 0,
"MAX": 4069
},
"inputs": {
"VALUE": {
"block": {
"type": "variables_get",
"id": "LIIgG+UMkthq!S.K}dX8",
"fields": {
"VAR": {
"id": "[g,f6Mp!O$eZPCFs0U[H"
}
}
}
}
},
"next": {
"block": {
"type": "hmiSetText",
"id": "1d7kB$!lQpY@szAnVaLx",
"fields": {
"NAME": "Status"
},
"inputs": {
"TEXT": {
"block": {
"type": "variables_get",
"id": "`usj,%r!VHQt}Ucr]Bxv",
"fields": {
"VAR": {
"id": "xjE5n-FetBK*,)qj?pyn"
}
}
}
}
},
"next": {
"block": {
"type": "hmiSetNumber",
"id": "-^@C9g@boq7MU?u2E}f9",
"fields": {
"NAME": "Value1",
"UNIT": "pin"
},
"inputs": {
"VALUE": {
"block": {
"type": "variables_get",
"id": "Cpu4;(,q`6WS@(f@L,|6",
"fields": {
"VAR": {
"id": "[g,f6Mp!O$eZPCFs0U[H"
}
}
}
}
}
}
}
}
}
}
} }
} }
} }
@ -425,6 +586,10 @@
{ {
"name": "valX", "name": "valX",
"id": "Ug!mIa*[PnsL?H#9Ar*G" "id": "Ug!mIa*[PnsL?H#9Ar*G"
},
{
"name": "text",
"id": "xjE5n-FetBK*,)qj?pyn"
} }
] ]
}, },
@ -433,12 +598,44 @@
"name": "LED1", "name": "LED1",
"type": "led", "type": "led",
"x": 1, "x": 1,
"y": 0, "y": 1,
"w": 2, "w": 2,
"h": 1, "h": 1,
"config": { "config": {
"color": "#4caf50" "color": "#4caf50"
} }
},
{
"name": "Gauge1",
"type": "gauge",
"x": 1,
"y": 2,
"w": 2,
"h": 2,
"config": {
"min": 0,
"max": 4069
}
},
{
"name": "Status",
"type": "text",
"x": 0,
"y": 0,
"w": 6,
"h": 1,
"config": {}
},
{
"name": "Value1",
"type": "number",
"x": 3,
"y": 1,
"w": 2,
"h": 1,
"config": {
"unit": "pin"
}
} }
] ]
} }