Amir Farhadi

Windows background daemon and Model Context Protocol (MCP) server providing real-time desktop topology, active window envelopes, Virtual Desktop GUIDs, and container-aware tab extraction in under 15 ms with 0.0% idle CPU.

• Dual-Engine Topology: Sub-millisecond Win32 shallow gating (< 0.5 ms) paired with FlaUI.UIA3 batch caching to replace CPU-heavy OCR screenshot polling.
• Real-World Downstream Integration: Streams live UI focus zones to Caster Voice OS to enable dynamic IDE terminal voice grammars with zero speech engine audio latency.
• Browser Extraction: Container-aware tab and URL extraction with strict DOM tree pruning (10.17 ms for 30 tabs) without browser extensions.
• Time-Series Store: Embedded SQLite WAL database recording window state transitions for temporal agent reasoning.
• System Tray Host: Per-Monitor V2 DPI-aware host with non-activating live DevTools HUD overlay (WS_EX_NOACTIVATE | WS_EX_TOPMOST).

▸ System Tray Host & Non-Activating Live DevTools HUD

▸ Zero DOM Crawling: Sub-15ms Browser Tab & URL Extraction

→ View GitHub Repository (opens in a new tab) → View Real-World Use Case: Caster Dynamic IDE Grammars (opens in a new tab) → View Architecture & Engineering Guide (opens in a new tab) → View Empirical Telemetry Benchmarks (opens in a new tab)

Native C++ Windhawk modification injecting real-time speech recognition hypotheses, active Dragonfly grammars, and ADCE semantic interaction zones directly into the Windows 11 taskbar (Shell_TrayWnd) adjacent to the system tray.

• In-Process Shell XAML Injection: Hooks taskbar.dll symbols (CTaskBand, TaskbarHost, TrayUI) to mount native WinRT controls within SystemTrayFrameGrid across primary and secondary displays.
• Overlapped Named Pipe IPC: Integrates an asynchronous pipe server (\\.\pipe\CasterTaskbarHud) ingesting JSON telemetry with sub-0.5ms deserialization marshaled via WH_CALLWNDPROC.
• Unified Single Command Strip: Implements a compact 160px command strip to eliminate horizontal space starvation and prevent taskbar button clipping in TaskListButtonPanel.
• In-Situ Context Menu: Hooks XAML RightTapped events to present a Win32 popup menu (TrackPopupMenuEx) for runtime display mode toggling and HKCU registry persistence.

▸ Real-Time Windows 11 Taskbar Telemetry, Voice Hypotheses, & ADCE Semantic Context

→ View GitHub Repository (opens in a new tab) → View Official Windhawk Mods Catalog (opens in a new tab)

Clean-room, stateless Windows Virtual Desktop automation engine replacing legacy pyvda across Caster production. Eliminates Explorer restart crashes and resolves synthetic sub-AUMID isolation for Windows Terminal.

• Zero-Cached-State Direct COM Dispatch: Acquires fresh interface pointers per call directly from explorer.exe ALPC endpoints via explicit vtable offsets, guaranteeing crash resilience.
• Transient MTA Apartments: Automatically joins MTA or preserves caller STA state upon encountering RPC_E_CHANGED_MODE (0x80010106).
• Sub-AUMID Normalization: Resolves modern Windows Shell XAML Island sub-AUMIDs (~Wh~w<HEX_HWND>) to canonical package IDs, achieving Task View parity for multi-window pinning.
• Active Workspace Synchronization: Executes sub-millisecond window synchronization during desktop switches to guarantee sibling windows carry across workspaces.

▸ WinVDA Clean-Room Zero-Cached-State Transient COM Architecture

→ View GitHub Repository (opens in a new tab) → View Technical Specification & Architecture (opens in a new tab)

Independent distribution repository for modular Caster voice programming plugins, themed visual overlays, and decoupled context resolvers.

• Universal AST Rule Catalog: Parses get_rule() AST syntax trees across 160+ grammar modules at startup in under 100ms without initializing speech engines.
• Dynamic Configuration Sync: Monitors rules.toml file timestamps to refresh enabled rule maps on the fly without process reboots.
• Themed Qt HUD: Packages the advanced PyQt overlay with 10+ accessible QSS themes, opacity sliders, and frameless drag modes.
• Strict Safety CI: Includes automated repository safety scanners, path hygiene checks, and GitHub Actions CI test suites.

Upstream-targeted architectural enhancements to Caster core providing cross-platform process containment, synchronous microphone state observation, and extensible plugin lifecycles.

• Cross-Platform Process Strategy Pattern: Encapsulates child process containment via WindowsProcessStrategy (Win32 Job Objects with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE), LinuxProcessStrategy (prctl(PR_SET_PDEATHSIG)), and DarwinProcessStrategy.
• Self-Healing HUD Auto-Recovery: Detects communication drops in show_hud() and respawns the display process cleanly without restarting speech engines.
• Speech Recognition Decoupling: Replaces synchronous XML-RPC logging with an in-memory queue (queue.Queue) and background daemon worker, preventing audio stutter.
• Engine Microphone Observer Pattern: Implements register_mic_mode_listener on EngineModesManager, providing zero-overhead synchronous callbacks for mic transitions (sleeping, listening, off).

A 27-month, 969-commit personal voice computing operating system enabling hands-free programming, terminal navigation, and desktop orchestration.

• 3-Tier Window Switcher: Deterministic OS focus engine with Win32 thread attach, Taskbar UIA fallback, and synthetic keystroke routing.
• Kaldi ASR Engine Patches: Resolved compiler race conditions and memory leaks during deferred grammar reloads.
• Multimodal IPC: Integrated Olympus RS31H foot pedal controls via AutoHotkey v2 and thread-safe XML-RPC bridges.
• CCR Language Grammars: Modular command suites for Python, C#, PowerShell, Git, and AI IDE workflows (Windsurf, Cursor).

Multi-agent orchestration middleware connecting private Azure cloud infrastructure, Microsoft AI Foundry, and Azure Speech Services with the Google Gemini SDK ecosystem.

• Cross-Vendor Orchestration: Custom Python middleware managing concurrent execution across independent LLM agent processes.
• Speech & AI Integration: Synthesized multimodal agent reasoning using Microsoft AI Foundry models and Azure Speech Services.
• Context Synchronization: Shared state store coordinating multi-step planning, task delegation, and execution logging.
• Live Telemetry: Real-time event streaming and execution monitoring across cloud endpoints.

Low-level Windows desktop focus management and voice window switching engine for the Caster and Dragonfly speech recognition frameworks.

• 3-Tier Failsafe Architecture: Primary AttachThreadInput + SetForegroundWindow Win32 API bypass, secondary Taskbar UIA automation, and tertiary synthetic keystroke dispatch.
• Virtual Desktop Filtering: Interfaces with Windows Virtual Desktop APIs to filter application targets by active workspace.
• In-App Tab Cycling: Direct voice commands to switch between open tabs across browsers and code editors.

Upstream contributions merged into Caster, an open-source voice dictation and accessibility platform.

• PR #975 (opens in a new tab) (Numeric Engine Refactor): Rebuilt number grammar rules using NumberRef, resolving Caster Issue #938 (opens in a new tab) and Issue #857 (opens in a new tab) to remove the legacy 4-digit dictation ceiling.
• PR #965 (opens in a new tab) (HUD Startup Synchronization): Added polling retry logic to resolve thread initialization race conditions during startup.
• PR #962 (opens in a new tab) (Parser Corrections): Fixed title-case contraction and possessive casing bugs in continuous dictation output.
• PR #960 (opens in a new tab) (Windows 11 Desktops): Restored virtual desktop voice navigation by integrating updated pyvda COM interfaces.

→ View Merged Commits on GitHub (opens in a new tab)

Upstream patch resolving digit sequence matching conflicts, arbitrary length boundaries, and formatting loss in Dragonfly's core Number grammar class.

• Sequence Matching: Corrected greedy matching conflicts between single digits, compound numbers, and formatted number strings.
• Unbounded Digit Sequences: Removed hardcoded length restrictions to support arbitrary numeric input across speech engines.
• Zero Formatting & Upstream Link: Preserved leading and embedded zeros across speech backends, unblocking Caster Issue #938 (opens in a new tab) and Issue #857 (opens in a new tab).

→ View Pull Request #409 on GitHub (opens in a new tab)

Engine patch fixing fatal KaldiError crashes during deferred mid-phrase grammar reloads.

• Root Cause Fix: Captured stale rules scheduled for destruction at deferral time rather than querying modified global dictionaries post-phrase.
• State Isolation: Prevented the Kaldi grammar compiler from attempting to unload newly compiled replacement rules.
• Caster Stability: Resolves Caster Issue #943 (opens in a new tab), eliminating fatal process crashes during background rule compilation while user speech is in progress.

→ View Pull Request #408 on GitHub (opens in a new tab)

Stability patch for pyvda (Python Windows Virtual Desktop API) to recover from destroyed COM interface pointers and RPC server unavailability.

• Explorer Restart Resilience: Intercepts RPC_S_SERVER_UNAVAILABLE (0x800706BA) when Windows Explorer restarts.
• Re-initialization Hook: Automatically re-binds the internal IVirtualDesktopManagerInternal interface on fault detection.
• Sleep/Resume Protection: Prevents unhandled exceptions when systems resume from low-power sleep states.

→ View Pull Request #57 on GitHub (opens in a new tab)

Standalone CLI utility to snapshot and restore window coordinates, dimensions, and virtual desktop assignments on Windows, permanently resolving application drift. Publicly featured on Tiny Tool Town.

• State Persistence: Serializes window geometry (X, Y, Width, Height, Z-order) and desktop GUIDs to JSON configuration files.
• Drift Correction: Restores multi-monitor window layouts following display disconnects, sleep cycles, or resolution changes.
• Process Filter: Ignores transient tooltips, invisible utility frames, and background daemon windows.

→ View GitHub Repository for WinStasis (opens in a new tab) → View on Tiny Tool Town for WinStasis (opens in a new tab)

Standalone CLI utility acting like 'ls' for Windows Virtual Desktops. Maps out all active virtual desktops and open application windows in a clean terminal table.

• Desktop Hierarchy: Traverses the active desktop array and prints a structured tree view to the terminal.
• Window Attributes: Reports window handle (HWND), executable process name, window title, coordinates, and bounding dimensions.
• Zero Overhead: Lightweight single-binary executable with sub-50 ms execution time.

→ View GitHub Repository for vdtree (opens in a new tab) → View on Tiny Tool Town for vdtree (opens in a new tab)