[atlas-devel] Fwd: Re: Compiling Atlas with hyperthreading
Brought to you by:
rwhaley,
tonyc040457
Menu
▾
▴
[atlas-devel] Fwd: Re: Compiling Atlas with hyperthreading
|
From: R. C. W. <rcw...@ls...> - 2017-06-29 23:14:53
|
just realized my reply only went to Jeff. -------- Forwarded Message -------- Subject: Re: [atlas-devel] Compiling Atlas with hyperthreading Date: Thu, 29 Jun 2017 17:22:05 -0500 From: R. Clint Whaley <rcw...@ls...> To: Jeff Hammond <jef...@gm...> Jeff, Have you run a thread monitor to see if MKL is simply not using the hyperthreading regardless of whether it is on or off in BIOS? You also may want to try something like LU. Cheers, Clint On 06/29/2017 05:15 PM, Jeff Hammond wrote: > I don't see any negative impact from using HT relative to not using HT, at > least with MKL DGEMM on E5-2699v3 (Haswell). The 0.1-0.5% gain here is > irrelevant and may be due to thermal effects (this box is in my cubicle, > not an air-conditioned machine room). > > $ OMP_NUM_THREADS=36 KMP_AFFINITY=scatter,granularity=fine > ./dgemm_perf_PMKL.x $((384*40)) $((384*40)) $((384*4)) > BLAS_NAME dim1 dim2 dim3 seconds Gflop/s > Intel MKL (parallel) 15360 15360 1536 0.8582699 844.4612765 > Intel MKL (parallel) 15360 15360 1536 0.8627163 840.1089930 > > HT on > > $ OMP_NUM_THREADS=72 KMP_AFFINITY=scatter,granularity=fine > ./dgemm_perf_PMKL.x $((384*40)) $((384*40)) $((384*4)) > BLAS_NAME dim1 dim2 dim3 seconds Gflop/s > Intel MKL (parallel) 15360 15360 1536 0.8636520 839.1988073 > Intel MKL (parallel) 15360 15360 1536 0.8644268 838.4465853 > > I would be interested to see folks post data to support the argument > against HT. > > Jeff > > On Thu, Jun 29, 2017 at 7:57 AM, lixin chu via Math-atlas-devel < > mat...@li...> wrote: >> >> Thank you very much for quick response. Just to check if my understanding > is correct : >> >> 1. By turning off cpuid in bios, I only need to use -t N to build Atlas > right? >> >> 2. The N in -t N is the total number of threads on the machine, not per > Cpu right ? >> >> 3. One more question I have is, how to set the correct -t N for mpi based > application. >> Let's say on the 2-cpu machine with 4 cores per CPU, should I use -t > 4 or -t 8 if I rum my application with 2 mpi processes : >> mpirun -n 2 myprogram >> >> Many thanks ! >> >> Sent from Yahoo Mail on Android >> >> On Thu, Jun 29, 2017 at 22:20, R. Clint Whaley >> <wh...@my...> wrote: >> Hyperthreading is an optimization aimed at addressing poorly optimized >> code. The idea is that most codes cannot drive the backend hardware >> (ALU/FPU, etc) at the maximal rate, so if you duplicate registers you >> can, amongst several threads, find enough work to keep the backend busy. >> >> ATLAS (or any optimized linear algebra library) already runs the FPU at >> its maximal rate supported by the cache architecture after cache blocking. >> >> If you can already drive the backend at >90% of peak, then >> hyperthreading can actually *lose* you performance, as the threads bring >> conflicting data in the cache. >> >> It's usually not a night and day difference, but I haven't measured it >> in the huge blocking era used by recent developer releases (it may be >> worse there). >> >> My general recommendation is turn off hyperthreading for highly >> optimized codes, and turn it on for relatively unoptimized codes. >> >> As to which core IDs correspond to the physical cores, that varies by >> machine. On x86, you can use CPUID to determine that if you are >> super-knowledgeable. I usually just turn it off in the BIOS, because I >> don't like something that may thrash my cache running, even if it might >> occasionally help :) >> >> Cheers, >> Clint >> >> On 06/28/2017 10:32 PM, lixin chu via Math-atlas-devel wrote: >>> Hello,Would like go check if my understanding is correct for compiling > Atlas on a machine that has multiple CPUs and hyperthreading. >>> I have two types of machine: >>> - 2 CPU, each with 4 Core, hyperthreaded, 2 threads per core- 2 CPU, > each with 8 Cores, hyperthreaded, 2 threads per core >>> So when I compile Atlas, is it correct that I should use: >>> -tl 8 0,1,2,3,4,5,6,7 and -tl 16 0,1,....15 (assuming the affinity ID > is from 0-7 and 0-15). >>> That means the number 8 or 16 is the total cores on the machine, not > number of cores per CPU. Am I correct ? >>> I also read somewhere saying that Atlas supports Hyperthreading. What > does this mean ? >>> Does this mean:1. I do not need to disable hyperthreading in BIOS (no > performance difference whether it is enabled or disabled, as long as the > number of threads and affinity IDs are set correctly when compiling > Atlas)2. Or I can make use of the hyperthread, that is, -tl 16 and -tl 32 ? >>> Thank you very much, >>> lixin >>> >>> >>> >>> > ------------------------------------------------------------------------------ >>> Check out the vibrant tech community on one of the world's most >>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>> >>> >>> >>> _______________________________________________ >>> Math-atlas-devel mailing list >>> Mat...@li... >>> https://lists.sourceforge.net/lists/listinfo/math-atlas-devel >> >>> >> >> >> > ------------------------------------------------------------------------------ >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >> _______________________________________________ >> Math-atlas-devel mailing list >> Mat...@li... >> https://lists.sourceforge.net/lists/listinfo/math-atlas-devel >> >> >> > ------------------------------------------------------------------------------ >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >> _______________________________________________ >> Math-atlas-devel mailing list >> Mat...@li... >> https://lists.sourceforge.net/lists/listinfo/math-atlas-devel >> > > > > -- > Jeff Hammond > jef...@gm... > http://jeffhammond.github.io/ > > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > > > _______________________________________________ > Math-atlas-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/math-atlas-devel > -- ********************************************************************** ** R. Clint Whaley, PhD * Assoc Prof, LSU * www.csc.lsu.edu/~whaley ** ********************************************************************** |
