Mixture-of-Experts (MoE) AI architectures require massive memory bandwidth to activate sparse expert networks, bottlenecking conventional GPU clusters; ThAME integrates monolithic 3D memory to deliver high-throughput, energy-efficient MoE inference.
Frontier language models are increasingly adopting Mixture-of-Experts (MoE) architectures, where different neural network 'experts' are dynamically activated for specific tokens, enabling trillions of parameters with sparse computation.
However, serving MoE models in production triggers the infamous 'memory wall': GPUs spend more energy moving billions of expert weights across off-chip memory buses than actually performing matrix multiplications.
ThAME (3D Memory-Enabled Heterogeneous Accelerator) solves this memory bottleneck by stacking high-density 3D memory layers directly on top of specialized compute logic. The close physical vertical integration delivers massive memory bandwidth and near-zero interconnect latency tailored for dynamic MoE expert routing.
ThAME slashes generative AI inference energy consumption while accelerating token generation speeds by multiples, offering an architectural blueprint for serving trillions-parameter AI models sustainably.
ThAME: 3D Memory-Enabled Heterogeneous Accelerator for LLM Mixture of Experts
Mixture of Experts (MoE) architectures have emerged as a dominant paradigm for scaling Large Language Models (LLMs). However, MoE inference on conventional hardware is constrained by three fundamental bottlenecks. These encompass the massive memory bandwidth required to fetch non-contiguous expert weights, the non-deterministic scatter-gather traffic generated by input-dependent token routing, and the tail-latency dependency imposed by synchronous expert output aggregation. To address these challenges, we propose ThAME, a three-dimensional (3D) heterogeneous multi-chiplet architecture for MoE inference. ThAME employs Ferroelectric Field-Effect Transistor (FeFET)-based non-volatile and DRAM-based volatile memory chiplets with a co-designed compute mapping strategy that aligns the distinct computational profiles of attention mechanisms and expert routing. Furthermore, we design a specialized Network-on-Chip communication backbone optimized to mitigate the bottlenecks associated with non-deterministic token routing traffic across the combinatorial space of input-dependent MoE traffic patterns. Experimental results demonstrate that ThAME outperforms state-of-the-art counterparts by up to 15.7x in terms of speedup and improves energy efficiency by up to 9.8x.
Telecommunication engineers believed that static noise permanently corrupted electrical signals; Claude Shannon proved that information can be transmitted with zero errors across noisy channels by converting meaning into binary bits. Published in 1948, Shannon’s masterwork founded information theory and created the digital language of the modern world.
Error-correcting codes were invented to clean static noise from radio broadcasts; Elwyn Berlekamp and Robert McEliece proved that deliberately scrambling error-correcting codes creates an NP-complete cryptographic lock. Published in 1978 and sidelined for decades due to large key sizes, the McEliece code-based cryptosystem has never been broken in nearly fifty years, standing today as NIST’s premier quantum-resistant encryption standard (Classic McEliece).
Early computer networks were isolated corporate islands that could not speak to one another; Vint Cerf and Bob Kahn designed an open universal protocol that connects any computer on Earth regardless of hardware. Published in 1974, TCP/IP created the architecture of the Internet, allowing billions of devices to communicate seamlessly across a unified global network.
Early unmanned aerial vehicles were restricted to rigid, radio-tethered military reconnaissance; modern autonomous UAV architectures integrate micro-avionics, sensor fusion, and swarm intelligence to revolutionize civilian skies.
Smart city initiatives frequently imported glossy foreign technological blueprints that failed in regional municipal governments; empirical ICT auditing maps local technological capabilities to design realistic e-government roadmaps.
Computer scientists spent forty-four years convinced that a 1976 algorithm was the absolute mathematical limit for finding the shortest delivery route; Anna Karlin, Nathan Klein, and Shayan Oveis Gharan broke the 3/2 barrier using high-dimensional geometry and random spanning trees. Awarded the Best Paper at STOC, this historic breakthrough shattered the most famous stagnation in theoretical computer science.
Isolated security benchmarks test whether AI can write single malicious scripts; CyberBench measures whether autonomous software agents can execute multi-day corporate cyber attacks from start to finish. By stress-testing AI models on complex, realistic enterprise penetration scenarios, cybersecurity researchers established the first realistic yardstick for autonomous digital threats.
Centralized drone fleets crash whenever their central controller signal is jammed; bio-inspired drone swarms coordinate locally like flocks of birds without any central leader. By synthesizing breakthroughs in decentralized mesh communication and obstacle avoidance, robotics engineers have created resilient aerial swarms capable of mapping disaster zones and locating wildfire survivors autonomously.
Ask this paper your own questions, or keep browsing the verified research catalogue.
