We’ve deployed 2,000+ AMD GPUs in customers’ AI data centers

Moreh offers a fully integrated cluster system from individual GPUs to servers, networking, and software.

Building Blocks

GPU

Instinct MI300 Series

AMD’s Instinct MI300 series GPUs, powered by Moreh software, delivers industry-leading compute and memory performance.

Peak Performance

1.3x

NVIDIA H200: 989 TFLOPS FP16

AMD MI325X: 1307 TFLOPS FP16

Memory Capacity

1.8x

NVIDIA H200: 141 GB

AMD MI325X: 256 GB

Peak Memory Bandwidth

1.25x

NVIDIA H200: 4.8 TB/s

AMD MI325X: 6.0 TB/s

Intra-node networking

Infinity Fabric

AMD’s Infinity Fabric technology enables high-bandwidth communication between GPUs within a single server, similar to NVIDIA’s NVLink. AMD’s MI300 series GPUs provide 896 GB/s of Infinity Fabric bandwidth, which is on par with the 900 GB/s delivered by NVIDIA’s NVLink 4.0.

Inter-node networking

Moreh’s RoCE Networking

Moreh designs and installs efficient and scalable networks tailored to cluster size and customer needs, based on the standard RoCE (RDMA over Converged Ethernet). Instead of using more hardware, our software applies various optimizations to minimize communication overhead.

Middleware

MoAI Framework and Moreh vLLM

Moreh’s software stack ensures the best performance of LLM inference and training on an AMD GPU cluster. We also provide comprehensive technical support to address various issues related to AMD GPUs that customers may encounter.

Platform

MoAI Platform

MoAI Platform provides all the necessary features for operating and managing AI infrastructure based on AMD GPUs. Built on Kubernetes, it offers a flexible design that allows customers to easily integrate it with existing cloud or MLOps platforms if desired.