Tuesday, January 25, 2011

Multi Core CPU and NUMA systems

With introduction of  multi core CPU's,  performance engineers are being challenged to fine tune systems to get maximum mileage out of them. Its definitely worth to see whether your Operating System supports following important features.

1) Kernel Thread/Process affinity between local processor and memory. NUMA systems means non-uniform memory access wherein the latency to fetch memory on remote processor is much higher. Also processor would spend considerable time generating/servicing TLB flush on such situations resulting in lower performance. Solaris supports Memory Placement Optimization (MPO) for some time now. The kernel is aware of system memory map and allocates/creates kernel thread(s) closer to CPU

2) Make sure NIC and HBA drivers can dynamically bind interrupts to any processor on the system. Modern PCI-Express supports multiple in-band interrupts (MSI) per device. But Operating System need to take advantage by supporting large number of MSI interrupt vectors

3) If you're running Virtual Machines,  make sure your Operating System and HW are SRIOV capable. Most Virtual Guests tunnel Network and Storage requests through Control or domain 0. This was OK when system had only few CPU's . But with multi core systems, Operating System performance will suffer heavily without SRIOV.  PCI Express fabric errors are handled by Domain 0, while VM Guests can perform IO by manipulating HW directly. Hypervisor typically ignores PCI Config writes when executed by the VM Guests. Watch out for potential Denial of Service (DoS) attack when using SRIOV. This technology allows VM Guests to access piece of HW directly. It may be possible for a errand VM Guests to perform excessive IO potentially pinning PCI Express Bus

No comments:

Post a Comment