#92 Feature request: GUI hot-reload from disk, MCP update_pipeline, and external AI API support

[Anonymous]

Originally created by: cola3k
Originally owned by: SouravRoy-ETL

Summary

Three related improvements to bridge the MCP ↔ GUI gap and make the built-in AI configurable.

Background

Duckle ships with a local Qwen Coder 1.5B model that can generate pipeline JSON in the GUI — a great idea, but in practice 1.5B is too weak for non-trivial pipelines. I connected Claude Code via duckle-mcp, which works beautifully for creating complex pipelines (multi-stage unpivot → regex extract → pivot chains, etc.), but then I hit the GUI sync wall described below.

Problem 1: No bidirectional GUI ↔ JSON sync

Current behavior:

• When the GUI edits a pipeline, it saves to the JSON file immediately ✓
• When the JSON file is modified externally (by MCP, git pull, hand-editing), the GUI does not pick up the change — even after switching to another pipeline and back ✗
• If the GUI has a pipeline open in memory and the JSON file is changed, the next GUI save silently overwrites the external changes ✗

Steps to reproduce:

1. Open a pipeline in the Duckle GUI (e.g. dsd_drug_export)
2. Use duckle-mcp create_pipeline --overwrite to modify the same pipeline's output path
3. Switch to another pipeline in the GUI, then switch back
4. The GUI still shows the old output path
5. Make any edit in the GUI and save — the MCP changes are overwritten

Expected behavior:

• At minimum: File → Reload from Disk menu item
• Ideally: a file-system watcher that detects external JSON changes and prompts "This file was modified externally. Reload?"

This is the single biggest friction point for a "git-friendly, file-based" workspace — the files should be the source of truth, not the GUI's in-memory state.

Problem 2: MCP lacks a proper update_pipeline tool

Current behavior:

• create_pipeline can overwrite, but requires the entire pipeline object
• This means the caller must first read_pipeline, merge changes, then create_pipeline --overwrite
• MCP-created pipelines do not auto-register in repository.json or duckle.json, so they are invisible to the GUI until the user manually edits those index files

Expected behavior:

• An update_pipeline tool that accepts a partial pipeline diff (e.g. only the changed node property) and merges it
• Or at minimum, create_pipeline should also write the corresponding entries into repository.json and duckle.json so new pipelines appear in the GUI automatically

Problem 3: Built-in AI only supports the local Qwen model

Current behavior:

• The "Duckie" AI assistant in the GUI is hard-wired to the local Qwen 2.5 Coder 1.5B via llama.cpp
• There is no UI to configure an external AI endpoint

Suggested improvement:

• Add an AI settings panel in the GUI that lets users choose between:
• Local model (default, current behavior)
• External API endpoint:
  • URL field (e.g. https://api.anthropic.com/v1/messages)
  • API key field
  • Model name field (e.g. claude-sonnet-4-6, gpt-4o)
• This way, users who have access to more capable models (Claude, GPT-4, etc.) can get high-quality pipeline generation directly in the GUI, while users who prefer fully-local keep the current behavior

Why this matters

The combination of Duckle (visual designer + MCP tooling + git-friendly files) is incredibly promising. The MCP server lets powerful external AI drive Duckle, which is a far better experience than struggling with the local 1.5B model. But without file-based sync between MCP and GUI, the workflow breaks down — you end up fighting the tool instead of being productive. These three improvements would make the dual MCP + GUI workflow seamless.

[Anonymous]

• assigned_to: SouravRoy-ETL

[Anonymous]

Originally posted by: SouravRoy-ETL

That's a nice idea. I will implement it.