Orchestrating Multiple Heterogeneous LLMs via Hermes

In the current era of artificial intelligence, the bottleneck for local on-site AI is no longer just the model’s capability, but the tension between compute density and latency. As a hobbyist interested in all things technological, I was seeking to build an AI with a massive 70B+ parameter model, but I couldn’t afford top-of-the-line new GPUs. I also don’t want to wait for multi-second “thought” latency when a model is bottlenecked by VRAM or sluggish hardware. Furthermore, running every minor task—like summarizing a file name or compressing a chat history—on a high-end model wastes resources. So what does one do? They cobble together a hybrid layered platform from gear they have and what they can reasonably pick up on eBay. 

In April, I purchased an AMD Radeon 7600 with 8GB of VRAM to create my starter LLM system. Experimenting with Ollama, OpenCLAW, and Hermes back then was painful. Things worked, but the results were underwhelming. A month or so later, I added an AMD Radeon 7900 with 24GB of VRAM, and LLM performance and results remarkably improved. The agentic platforms were still pretty rough, and this system remained autonomous from my normal daily computing platforms that held all my personal data. I was waiting for something more: the new Mac Mini, specifically the M5 Pro with 64 GB of system RAM. With this system, I could finally bridge the chasm between an agentic platform and my real-world data (files, email, and calendar). 

At that point, I stepped back and asked Gemini to suggest the best way to build a hybrid orchestration solution using the gear already on hand. By using the Hermes Agent as a centralized intelligence gateway, I built a tiered compute architecture that intelligently routes tasks across a heterogeneous hardware stack. This approach maximizes the utility of a local powerhouse, like the Mac Mini, by offloading specialized workloads to the high-performance remote EPYC server over a fast, high-bandwidth 10 GbE link.

The Hardware Stack: A Symphony of Silicon

To understand the power of this architecture, we must first look at the three distinct compute tiers that form the system’s backbone.

1. The Local Anchor: Apple Mac Mini M5 Pro (64GB RAM)

At the center of my workspace sits the Mac Mini. This machine hosts Ollama with the Gemma4:26b model. In this architecture, Gemma4 serves as the “Local Anchor.”

Because it resides on the same machine as the Hermes Agent and the developer’s IDE, Gemma4 provides near-instantaneous responses for the “Main Loop”—the primary chat interface where the developer interacts with the agent. It handles general queries, runs local shell commands, and manages the immediate conversation context. The 64GB of unified memory enables high-speed inference and ensures that even as the conversation grows, the local experience stays fluid and responsive. The Gemma4:26b model also leaves enough free space for other tasks so that, outside of Hermes, everything still functions as expected. 

2. The Utility Worker: AMD Radeon 7600 (8GB VRAM)

The Mac Mini is connected via a 10 GbE (10 Gigabit Ethernet) link to a central AMD EPYC ROME server as the first specialized compute node: an AMD Radeon 7600. This GPU is optimized for the Qwen3.5:9b model.

In the Hermes ecosystem, this model is designated as the “Fast Tier.” It is not intended for deep reasoning, but rather for the “dirty work” of an autonomous agent. When Hermes needs to perform background tasks—such as compressing long conversation histories, generating session titles, or running regex-based text extractions—it routes those calls to this fast, low-VRAM node. By offloading these “auxiliary” tasks to the Radeon 7600, we ensure that the heavy-duty GPU and the local Mac Mini are never interrupted by trivial computations.

3. The Reasoning Specialist: AMD Radeon 7900 (24GB VRAM)

The crown jewel of the remote server is the AMD Radeon 7900, boasting 24GB of VRAM. This card hosts the Qwen3.8:27b model, the “Heavy Tier.”

The Radeon 7900 is designed for high-stakes inference. When I ask Hermes to “Refactor this entire micro-service architecture” or “Debug this complex race condition in a distributed system,” Hermes utilizes its delegation capability to spawn a sub-agent. This sub-agent is routed specifically to the EPYC server’s 7900 GPU. The 24GB of VRAM enables a much larger context window and more sophisticated attention mechanisms, letting the agent “think” through complex, multi-step problems that would overwhelm smaller models.

At this point, I’m also considering adding a second used AMD Radeon 7900 and moving from Ollama to vLLM to utilize GPU resources better and increase the model size from 27B to 70B or 72B. This should result in stronger multi-step architectural reasoning, fewer edge-case logic errors, and better synthesis across large repositories.

The 10 GbE Backbone: Eliminating the Remote Penalty

The “Achilles’ Heel” of remote compute, especially for hobbyists, has traditionally been network latency. If a developer waits for an agent to execute a tool that requires three sequential calls to a remote LLM, a standard Wi-Fi or 1GbE connection can introduce unacceptable delays. Here is where a wired 10 GbE network comes in handy. 

I purchased the Mac Mini with a 10 GbE interface specifically for this usage model. Using a 10 GbE link, I effectively treat the remote EPYC server as if it were on an internal PCIe bus. Model-request latency is reduced to the physical limits of the software stack, making the transition between the local Gemma4 and the remote Qwen models feel seamless. The high bandwidth ensures that massive context windows (which can be several megabytes of text) are transmitted in milliseconds, allowing the “Heavy Tier” to receive the full project state without stalling the developer’s workflow.

Orchestration via Hermes: The Intelligence Gateway

The “glue” that makes this hardware-software ecosystem possible is the Hermes Agent. Through its config.yaml architecture, Hermes acts as a sophisticated traffic controller. It doesn’t just “pick a model”; it assigns intent to compute.

• Direct Interaction: I interact with the local Gemma4:26b model.
• Auxiliary Routing: Hermes’ internal routines (compression, title generation) are automatically hard-coded to the EPYC-FAST (Radeon 7600) provider.
• Delegation Routing: Complex tasks are explicitly routed to the EPYC-HEAVY (Radeon 7900) provider via delegated sub-agents.

This creates a “Brain-and-Limbs” dynamic. The Mac Mini is the brain, making decisions and interacting with the human, while the EPYC server provides the specialized muscle (the GPUs) needed to execute high-intensity tasks.

High-Impact Use Cases

This architecture is uniquely suited for workflows that require a mixture of high-speed interaction and deep, intensive reasoning such as:

1. Marketing Content Development (The “Agentic Marketer”)

In a marketing workflow, I use Gemma4 to explain content objectives and layouts or generate quick text snippets. When I hit a “Blocker,” I command Hermes to “Solve this.”

• Gemma4 maintains the session and manages the file system.
• Qwen3.5 (Fast) runs the sanity tests and lints the content in the background.
• Qwen3.8 (Heavy) performs the deep architectural reasoning, writes the new implementation, and plans the refactoring steps.

2. Stock screening

For research that handles large datasets pulled from market data feeds, the workflow is highly iterative. For example, it’s not uncommon to ask for the last 30 minutes of 1-minute candles for a specific stock and then request that this data be analyzed to determine whether the stock is trending up or down. 

• Gemma4 handles the natural-language interface and queries market data from the TradingView gateway.
• Qwen3.5 (Fast) performs data cleaning, handles formatting, and manages metadata.
• Qwen3.8 (Heavy) runs complex statistical analyses, generates deep insights from large text corpora, and synthesizes findings into a comprehensive response.

3. Real-Time Workflow & Knowledge Synthesis

From managing calendars and email to creating documents, the system acts as an automated person-Friday.

• Gemma4 manages the daily ingestion of new information and user queries.
• Qwen3.5 (Fast) handles the summarization of incoming articles, emails, and calendar entries along with the generation of search tags.
• Qwen3.8 (Heavy) performs “Cross-Document Reasoning,” identifying connections between disparate topics across thousands of files to build a cohesive mental model of a subject.

Conclusion

The future of AI productivity is not found in a single, massive monolithic model, but in the intelligent orchestration of specialized models across a high-performance, heterogeneous network. By combining the local responsiveness of Apple Silicon with the raw, multi-GPU power of an EPYC server—bridged by a 10 GbE link and managed by the Hermes Agent I’ve turned a collection of hardware into a unified, thinking machine. This is more than a workstation; it is a complete, scalable intelligence platform.

In my next post, I’ll talk about how I added Slack, Telegram, and iMessage gateways so I can reach Hermes from anywhere. I’ll also cover integrating email and calendar support and connecting a gateway to TradingView for market data.