AI Infrastructure – Marco Santucci

Test Lustre Cluster Throughput and High Availability options

Post published:7 January 2025
Post category:Cloud

https://www.youtube.com/watch?v=DdlIhfU2VAU In this video (English subtitled), I demonstrate the aggregate performance of a 4-node Lustre cluster (3 OSS + 1 MDS/MGS) and the block volumes provided by Oracle OCI. Of course, this is a functionality test and not a production POC. In addition to understanding and observing the aggregate performance that only 3 active OSS nodes can deliver, I wanted to test 2 failover systems between OSS1 and OSS2 nodes (OSS3 is unpaired and works without failover). 1 - The…

Optimizing HPC Storage: Lustre Design Strategies for Oracle OCI

Post published:29 October 2024
Post category:Cloud Technology

In today’s high-performance computing (HPC) landscape, storage solutions that offer speed, scalability, and reliability are essential. Lustre, an open-source parallel file system, is a popular choice for HPC environments due to its capacity to handle large datasets and intensive workloads. When deploying Lustre on Oracle Cloud Infrastructure (OCI), achieving an optimized design that balances performance with cost-efficiency requires careful planning. Let's explore key Lustre infrastructure design patterns tailored for OCI, focusing on strategies to meet diverse storage needs and performance…

Understanding Lustre Performance: Throughput in High-Demand Scenarios

Post published:22 October 2024
Post category:Technology

When it comes to AI and machine learning applications, data throughput can become the deciding factor between success and failure. Lustre’s architecture is designed to offer unmatched performance across large-scale clusters, and the achievable throughput can be optimized based on the hardware configuration—specifically, the type of storage drives used and the network infrastructure in place. Throughput on Different Storage Drives 1. HDDs (Hard Disk Drives) For installations that use traditional HDDs, throughput generally peaks between 100 to 200 MB/s…