ftq/ 775 0 0 0 10777501713 76065ustar00johnjohnftq/README.ftqviz 664 0 0 2041 10777435266 12016ustar00johnjohnFrom Darren Hart The attached version overlays the samples from the input files onto a higher resolution array and interpolates between known datapoints, then passes that through a low pass filter, and uses the resulting signal to calculate the fft with a known fixed sampling rate. The attached script now uses proper getopt() argument parsing with the following usage: usage: ./ftqviz.py -t times-file -c counts-file [-s SAMPLING_HZ] [-w WINDOW_LEN] SAMPLING_HZ defaults to 10 kHZ which works with 1M sample files. You can take it up to 100kHZ on smaller files, but I am not convinced that adds much useful information. WLEN defaults to 25 and should be increased by roughly the same factor as the SAMPLING_HZ. This is left as an option since it is usually a rather manual process to fine tune how much smoothing you want to do. The larger the number, the more attenuated the high frequencies will be in the FFT. Please see the attached PNG illustrating the results. This version of the script can be shared publically. ftq/README.plan9 664 0 0 174 10777435266 11503ustar00johnjohnFTQ for Plan 9 =============== To compile, simply do a 'mk'. Use the included python program to analyze the data collected. ftq/README.txt 664 0 0 21211 10777435266 11332ustar00johnjohn------------------------------------------ FTQ: The Fixed Time Quantum microbenchmark ------------------------------------------ This directory contains FTQ, a useful benchmark for probing the interference characteristics of a particular computer (standalone, cluster node, etc...) as experienced by user applications. It is discussed extensively in two places: Sottile/Minnich : Cluster 2004 paper Sottile : Univ. of New Mexico, PhD dissertation Both of these should be available at some point in downloadable form from the chamatools web page. For the time being, the following is a brief summary of the code, what it does, and how you should run it. For further information, contact Matt Sottile at matt@lanl.gov. 1. What is FTQ? ---------------- FTQ is a simple benchmark that probes interference within a system. It does so as follows. For a given sampling period (say, 1ms), we would like to know how many units of work can be achieved by a user application. The unit of work is defined below. We can iteratively sample the system, checking for a series of 1ms sample intervals, how much work is actually achieved in each. For a high interference system, one will see a great deal of variability in they output data. For a low interference system, one will see less variability. Of course, there is the distinct possibility that one may be experiencing constant interference, which would be identical to the low interference case, except with a lower than ideal mean value. A method for investigating whether or not this is the case is discussed below. Traditionally, a simple benchmark to quantify this effect was what we call the Fixed Work Quantum benchmark. That method basically fixes a work quantum (say, 10000 integer operations), and samples the amount of time it takes to execute over a series of samples. The problem with this method is that although interference is revealed, the lack of a firm time axis (think about it for a second -- instead of sampling intervals being defined in terms of time, they're defined in terms of work!) makes sophisticated analysis in things like the frequency domain difficult and at times, impossible. FTQ fixes this with two subtle tweaks to FWQ. First off, instead of performing a single work quantum that requires on the order of the time we wish the sampling period to be, FTQ uses many samples of work quanta that are significantly smaller than the desired sampling period. Second, due to the fact that FTQ is a self sampling system, it is entirely possible (actually, it's almost guaranteed) that a sample may exceed the sampling interval by some amount. FTQ is constructed to compensate for that by shrinking subsequent samples to get the time series back on track. In the end, FTQ provides data that both reveals interference and is in a form where sophisticated analysis is possible depending on the desires of the user. 2. Building FTQ --------------- FTQ is simple to build. It has been tested on MacOSX, Linux, and even at one time under Windows XP via cygwin (note: Windows hasn't been tested in a long time). There is no configuration step at the present time, although it is likely that one may emerge if the FTQ configuration options get sophisticated enough. In the meantime, if one just says: % make You should observe 4 executables being created. They are ftq, ftq15, ftq31, and ftq63. Each of these represents increasing granularity work quanta used for sampling. For example, ftq uses a single integer increment operation as the work quantum, while ftq31 uses 16 increments followed by 15 decrements (effectively equivalent to the single ftq count++). The reason for this is that one will observe, especially for tiny work quanta, jitter in the data due to the fact that the majority (approx 70-80%) of the instructions executed per work quantum are actually branch and conditional operations. This causes some level of fluctuation in how long a work quantum takes to execute simply due to the structure of the instruction stream. As the work quantum increases in granularity, the proportion of work quantum instructions to control flow and conditional structures becomes more even, and the fluctuations within the data begin to be due to interference external to FTQ itself (other than apparent cache related, high frequency low level perturbations). Some users may be interested in profiling FTQ to observe hardware counters in order to see, say, the cache behavior of FTQ itself. This is useful in understanding what sources of interference in the output data are internal to FTQ due to cache misses or instruction mix characteristics, versus those caused by the hardware or operating system. One can build FTQ against the TAU (Tuning and Analysis Utilities) package, which in turn can provide profiling support using the performance counters from the PAPI (Performance API) library. To enable this in FTQ, first download and build TAU and PAPI. They can be found at: TAU : http://www.cs.uoregon.edu/research/tau/home.php PAPI: http://icl.cs.utk.edu/papi/ Then uncomment the TAU related lines in the Makefile, fixing the paths to match the location and version of TAU that was installed. Finally, type: % make tau Details regarding running TAU with hardware counter measurements is discussed below. 3. Running FTQ -------------- The simplest way to get FTQ going is to just try it out with a small sample count and observe what the data looks like. FTQ has just a few parameters. Note that it might sounds strange to have the work quantum granularity a compile time option, but this is done so that the instruction stream is not polluted with conditionals enforcing a runtime determined work quantum. The parameters are as follows: -s : Dump output to the STDOUT -o : Prefix for output data files -i : Bits in sampling interval limits -n : Number of samples to take -h : Usage Let's consider a simple run like this: % ./ftq63 -o testrun -i 20 -n 1000 What does this produce? If we look at the current directory, we should see two new files: testrun_counts.dat and testrun_times.dat. These are the outputs from FTQ. The times file contains the end times of each sample as reported by the high precision timers in cycle.h. We should see 1000 lines for the above case, corresponding to the end times for each of the 1000 samples executed. Note that if one subtracts time k from time k+1, one can observe the actual sampling interval that the k+1'th sample required. Note that the first sample has no start time, and is generally thrown out without losing any really valuable information. The counts file contains the real data from FTQ. Each of the 1000 entries represents the number of work quanta (in this case, 32 increment operations followed by 31 decrements per work quantum) executed in the time period ending at the corresponding time in the testrun_times.dat file. That's about it for the output data without getting into analysis techniques and tips. The final argument that is unexplained is the "-i 20" argument. What does this mean? Easy. The sampling intervals that FTQ uses are determined using a bitmask and some simple tricks to deal with long samples that must be compensated for. The "20" is the number of bits that the desired sampling period requires in terms of processor cycles (so even though it may seem large, it isn't). In other words, for the case above, "-i 20", a single sample period will span at most (2^20)-1 samples. On a 2.8GHz CPU, this corresponds to approximately 0.5ms. ** (**: note - the exponent may be off by one here, and my time computation may be off too. Writing this fast, so please forgive errors until I get a chance to proofread this. -m) 4. Simple data analysis ----------------------- So you've made data from FTQ, and want to actually do something with it. This is pretty easy with tools you likely already have on your desktop. Here is a brief overview of some things you can try. You likely have GNU Octave on your machine, or available to install without much effort. Let's try looking at the data that was produced above (testrun_counts.dat). Here is the sequence of Octave commands to run: octave:1> load testrun_counts.dat octave:2> mean(testrun_counts) ans = 1721.7 octave:3> var(testrun_counts) ans = 1.0718e+04 octave:4> sqrt(var(testrun_counts)) ans = 103.53 octave:5> plot(testrun_counts) In the above sequence of commands, we loaded the data into the Octave array "testrun_counts", looked at it's mean, variance, and standard deviation. Finally, we plotted it. Easy, eh? 5. Baselining your machine -------------------------- (This will be written soon - I want to make sure the "bootchart" code used to gather the FTQ pre-boot measurements is fine to redistribute without much headache.) 0777435266 11503ustar00johnjohnftq/ftq 775 0 0 120533 10777435266 10403ustar00johnjohnd hK$\ n|hx8`8 Hi8bH 8!N ad8'\8P<8aH9ёH 8aH8ߐH ՁAh@|@@|@9Jd9d|A|t,-A\<`Ƅ|0@@08`8 H 8`84H <`Ƅ\P,AP,A9P}(0}pXTHLTTPD.P|+x|+x|+x,@PpiA!)}!|*|cJ| P|&q@,})9C8j BT?A ||*aN aH%|@8!N |x,@@aH|@8!N |xH59,A|x,@@aH|@8!N 8`dH送9,AȀaHm|A8`HaHU|@8!N H1|A|x8!N 8`8!N N AP| A8`8!HN aa aL|8$(,0X48aTH a L|gP8!HN |ix}F(QA0i!$AA H!$|@@  8`8! N |x8! N |ix8a !9  8H\<8a XH|ixa|@8a K5!|A}#Kx8!PN 8`8!PN aK|A8`8!N ac8!N A CC|* <`!ܐ|xN ȁ@|kx|A`KA0a< ,|;x|A , %@9}&PP}&p|2K| @ k|eP8!8N }c[x H&Qa<|lx|@d8`8!8N (, @t9@8|p@@8}c[xA8 H!(@a<|jx|A ,}ESx9JA0K8`8!8N 8a6@K,A0a<@}@6|#x(KdKA$a< @|#x|A , %@L9 |PP|"K| @ k|fP8!8N K(, @p8|p@@8}c[xA8 H(@a<|jx|A$ }GSx9JA$9K<8`8!8N 8a6K @a>(@9JA08@| @@8aLa8 K$A0|kx|AL @}g[x9kaD>9$| ALL|XP|p|:Ld(pA$aL T| PK|A|x8!HN 8`8!HN 8`8!HN ,|0P,@,}CSx,} @PKEaDA0|@K<8a>AK݁$aD0}HA0K8`8!HN (p@(aL !T|(PK A0|ALfa8L 4T$$|@>>(@d9JA084|8@@(a>Kсa8A0|lx|4|@@A>(@}f[x9ka8>$8$$|A|LL|XP|"Ld(pADaL T| PK|A|x8!HN 8`8!HN >}c[xa88KeA08}ga8KDaLa ḰA0|kx|AL 4K8`8!HN ,| P,@,}CSx,|0PKa8A0|@K8a>AKꝁa80}FA0KT8`8!HN L(TK|kxP@$$,((pA (!T| @AD|A\(p@0}c[xaL$!T|(PK5@aL|AK TU<}r<@<|(@@d8|`@@4}c[xaLA9 KM@aL|jx|A }HSx9J.(9@<|(@A}&PP}&p|2K(qA$}c[x$T}@PK]|A|x8!HN 8`8!HN 8`8!HN 8`8!HN (p@0}c[xaL$T| PK@aL|AKaL ,TT<|r<<| @@@T>A08,| @@8U>FTc>K݁@FaLA0|lx} ,|(@AU>(@}HSx9JU>(9<| @A|PP }2 K(pA}c[x$T|0PK|A|x8!HN 8`8!HN U>"}CSxA08"K聁@aL0}DKX}c[xA K@FaL|jx|A ,K8`8!HN 8`8!HN (TK899(88 8K8! N 8888 K8! N Ё8|mx| @4}kx98 8K8!0N a4,pA| x| x (a(| @@|R|t|@(}kxa AK8!0N 8a&A,} RK}48,}D(9eK}cxHaa8HIaa48 K98!0N 耣88K8!N ؁A0|lx| @4}cx88 8K8!(N ,qAPl(| x| X@U'<|R|@$}cxA! K8!(N 9)K}ISx|A9)|@|HP|pK88K8!N  8(88 KQ8! N X|mxD,|xPT| x ,p@88E}LSx8x a:"!X ,dAA,XA,bA8`8!N 94q@<~x|A P|A@T|@A||@ | A\8b9gEa\| x8|AЀP|@T|@| @}f[x9k809@(| `@8b8| 0@AD|@pq0@hqA,@(9Jq@A qAqA,,@}d[x9k ,A,AD|AP}d[x9k8-,?c09,}kx8A AK8!N q A}h[x9k9 +(KqA}g[x9k9 K}f[x9k80K`$| `@8b8| 0@A $K }e[x9k809JK,@܈|t,0@?cp9K9JK9JK}e[x9k809JKT8a TP @}pH M$ <8a 8P<`? 8@|pH a\A8@4qA9CA|PP,A4}h[x9ka\!X<}):)})t(9JA8KP}d[x9k8,9JK| AH{y~`P}{qA9CA|PP,A$}h[x9k}1)})t(9JK}d[x9k8,9JK}8DKTP||TP8DKx8|;x:"%!XK 9 K,oA(,xA8`8!N 9K9KqA`:qA, 9 8TPKd88TPKK88}tKqALqA$88U>Kp88}4KPqA 8 9K,89K8!,98GpA |p|x8!N pA |x8!N pA|x8!N pA|x8!N |x8!N |ix ,-AA|, A`,#AD,+A(,,A 8`N ,`,8`N ,` ,8`N ,`,8`N ,`,8`N ,a ,8`N ,hA\,lA(,uA 8`N ,`,8`N ,qA,`, ,a ,8`N ,pA,`,,`,8`N 9% 8%8(98 8KI|x8! N a8`DN a8`DN (, 898Hq8! N (, 898HA8! N ||dx<ޭ`ޭ| ||` (|@ |!-@|L,N 8!!H|nx!88?c89d<8Te>p83}$(P| x| x| A, @p| @L}kx}cx8tƀA| A}k}JPNn8!@N }JБNn8!@N }L08 |8P}80|Cx}L0K, @ }cxK, @@8}<0K|, A } 0} 08 |8P}<0|CxK\AKL(KՃ8! N 8!= C0tA |At |А }@!lAB("r !l!!(@ Ȃ! **(8!N |АK!l!!(Ȃ2 !lac(@ Bc**8!N 98K8!N |ix(|@0|AP98(8 80!H8! N ,4|+8! N |ix(|@0|AP98(8 80!H8! N ,4+)|8P8! N |@|АN |А}@N |ixa4<|p8|(@D|p@|(A8U,|A8a8Ky!4@T(|A8a@KY!48889 8@!H݀,(|0A a4K8!0N <D}3֑ |p8!0N |jxa4<|p8|(@D|p@|(A8U,|A8a8KqA4@TA8a@KYA48889 8@AH݀,|A a4K8!0N <D31}@P |p8!0N ЁA8!9JA(Ug> Uf>,3AA,0AA`|AD,*A,.AdUc>Ka a$ |N!,$A(|@|x8! N |@d}CSx8! N ,`,|@D8aA(AKy,$(}EA(T>K$88| @<,pA ,?c(9, (K}Ё ,a ,,pA,`,(K,`,$K|x8! N ,q@,`@,KP| xUe>(0AUd>(9@|A 9JKt9JKlU:} JU}"9|AT>9JK|tT>9JK,1A,2A|K,7A$Al,4Ad,5A\,6ATK,8AH,9A@K8!!A|kx| Ah, @`}#J| X@A@|t}J:, 9AI|x8!N 90 9@9)| X@@818`8!N |x8!N 8!A|kx}#:| X@A@|t}J8P, 0AI|x8!N 99 9@9)| X@@898`8!N p9@a,pAC(A0, @ 80AAHi|A$a8hH8!N ȁ؁8H|A$a8lH8!N aa8HA|A$a9qHu8!N 8@\(A98< 44T>9|BpA8U$:|R?c(99)| A|Z8!N ؁A0a,| x, @,U&:8 |*U':|R9), A| x, @9E0 ?c(9 }t|@x, 0@8`8!(N |AKX9 , @,U':9 |B|AK09), A|x8!(N |P@8`8!(N |P@8`8!(N 8,K8! a$|mx|lx| x}H^1@}kx9}t|@x| AUE:|*T<}^1AT| A@UD:|"T<}F^0|X0}GPP}cx9809| @Ș 8!N }kxKUE:|*T<|r9DЀ8}H^1AP}H^0}X0}DPP}cx990}kx9}t|@8K@UF:|2T<|r9EKA !$|kx!$, @D}c[xaA8 KaA } :$9$!$, A| @}c[xA! K]8!N |jx4', A9 ! U$<|"T:8ˆ|2$}CSxA, H сA$! |@9)4} @P8|Ra,0A Kك8!(N 8! aA|mx| x 9k| h@@D| A4}+Kx8 });U':|:T<}XP9J80| @8!N |t8|p0}J8 }(+U%:|*T<|@P9J80K||jx<}(, A9 !(U&<|2T:9 |B$}CSxA4 H u4$a8!(|@9)}M2@} @P<|J|J}kx|JA K8!0N |ix 8 |t|A`}'Kx9)!|tT>8†|2pA|;xK!|gx|8A8`8!N |x8!N A(|ixa}#Kx!$A(A 9'H E|@$9$'(9G'KȀa8! N |@p|,@(T89|B$8!(N 8|#֐$|f8PKa$K!r$8!(N ,@$Td89|B8!(N 9|C֐$|ePKMa$KAa8!(N 8!<88!N !8!((t<|8A|x8!0N !He|@8`8!0N |x8!0N 8!=|@dƀ8!N 8!(! !|A|x8!N <|0@$|A8`8!N |x8!N <|0@$|A8`8!N |x8!N |x8!N = C0!lgAB(@ B* 0!lŀȁ(@ b*!24!lAB(@ bB*!$! A K! ؁a8!(N ؁A0!4|hx| x$|@| @8|;x$}| @| @p8|;x$}J| @| @H8$})}CxA! K|hx$|A}}Cx8!(N |xK|xK|xKl8!! !(@ 8!N !Ue>q9 ?c 9d?8!N !8!@@8!0N ! Te>p| @X} J,@L}I, @8}DPP9`| @ , A| @}ISx, @9 $|L0$8|H08 | 89 } @P|@0$|#x$ ?c(9 t |L0 };x }IPP| A ?c(9 Uh |Cx  8!0N 9J| x$U~$ pT$|3x$ ?c@9 t U~d |+x K}kJ, A`@8`K8!0N 8`K8!0N ȁAH!@@`@@|(aAKՁ!H0ȉ (ةA0(8!8N ۊ8!8N !00Te>p8,AD<`|&0|80}8804ȁ0؊a0(8!8N ,5A8=a}>0}@4}884K8bK}8!N 8bK8!N a8`)DN a! |jx| AAa, @}J}J| X@A}gRxp@qGA }'HP|~\*}k:~U*}J:}.&q@D}9k9J +Kk *JjBU)?A9k9J}!~\*~U*aN }.&q@$}~ 9k~ 9JBU)?A}!~\*~U*aN , A}bxp@xqA 9)q@}.&q@4} -Mm ,LlBU)?AX}.q@} BU)A8| X@@0 K8aN KXa |@,@|xq)@piA(!)}!}@*|cJ},*|J| P| p@@|&q@d})8c8Cc| `@@C| h@@|c| `@@l| h@@dB8c8T?AH|q@,})}C$8c}e$8| X@@,BTA|}@*}`,*| X@@ |xN 8`@8`N a|A$|8c8|(AB|xN 8cN 0123456789abcdef0123456789ABCDEF#c/pid?DP{RNaN+Inf-Infe%de%de%dmain@$AAp?@$@Y@@@È@jA.AcAׄAeB_ B7HvBmB0@BּC k&4CAy7Cv4WؠCmgNCX`=Dx@DKPDMՒD-JDxCyٝE *,(ET25EqE'/E1!F)>Y9F_$FnFȦ"FɜFЛ|G3BarMGhy GCWiG*HJH=c)\Hr]7HxaHܲyIY}LIFkC\I|T3I'\I1sJ^~:JQ~dJavJG}0Jlln>K$ᇈKZjAKP1P=Kd>ZMK}M`WL/ܡnmLcDL^LupMrifaM8O?MnbrM9GM׽)zN tF:NBȫdNw.=Ni9 NCOuԔOLLIOO̧O뢿[P!EⅿPU%'/PQ_P6PDbQ*[U{Q`xUmQV4*Qɼ߫5zR XlR4.Ri"rm9Rk )YRӢeS(N$S>aSs-W} S]\OSubسcT ]pTGh9L%T}B.TI%}Tn\UqsUQFUR`UxUpd+?V%=5V[@=̃NV&VJ0GƔVY:=W0$fWds&WW>)±XMs3]X9a!5kXoiBX8)XƠ4*YxHA5YC\-(Yx4%rYA/vYiZ?ZMOZwH02Z7|$~Zb[-["}CX[V/;[-C; [JS[\ =\+4"M\a2I0\HA|\R[]pKsy]4MPW]j=Hm]f-]]ԀW8u^ ma^@D$M|^tU-`^x^HUW_5.P_Hpy[_~]r_:'_k _`WE˷`RVR`N.'`"W:(`5vYnaninfinityinf7635118191655355242878388607671088631342177271251562519531251220703125152587890625190734863281251192092895507812514901161193847656257450580596923828125 ((((( H usage: %s [-n samples] [-i bits] [-h] [-o outname] [-s] failureftq_times.datftq_counts.dat%s_times.dat%s_counts.datWARNING: sample count exceeds maximum. setting count to maximum. WARNING: interval bits invalid. set to %d. %llud %llud %llud %llud success     # .;J\p ; #; %3+; ,; -; C1HEWGWS3X4(b4(c0d4(eWfWgWh; l; n:to4(p4(rWs2u; x4(tetext/9strtod.c@fmtlock.catol.c,utflen.c.string;pow10.cxendDmemccpy.s0nowargv0.sbdata7utfecpy.chedata libc.a libc.h8argv04pread.stabhfmtalloc.stringclose.s5rerrstr.cpsamples lock.cu.hfmtftq srcusr sysgetpid.c&sleep.s)read.csetSB%dofmt.cfrexp.c#fmtdef.hmain9.sendinterval_exitstr_exits.sinclude_nprivatessprint.c$0.40240000/fmt.conexlockftq.c_f2v .framef>_v2d.frame.safex?_v2f.framex?_divvu .frame dn@|_modvu .frame dnAvneg.frameAP_divv0.framednegnneg dnqBX_modv0.framenneg dnrC\_vasop0.frame typervturetfnlv ,WX  "6GLutflen .frames runen -WX  ")Gstrlen.frames .WX  "4Gstrchr.framec /WX  "# #x?Hp_fmtinstall.framefI(fmtinstall.frameretfcIdfmtfmt .framepcJT_fmtdispatch .frame.saferuneffmtisrunes 0WX  "# 1@A #Lxadd.framevnM|xsub.framenvMxdtoa.frameHc2Pc4Lc3Dc1WX  "nLfrexp.framex epdnldexp0.framex deltaedpmodf8.framexd ip @WX  "0q_fmtlock.frameq_fmtunlock.frame Aqerrstr.frame Bqmemmove.framesmemcpy.frames2ns1 Cosmemcmp.frames2ns1 Dtmemccpy.frame cs2 ns1RAAJASAA cfTdVAQ J M AOBGAFsBI* ^[XM I C EFHBfCATDAP KKK KIBADA%eEABACCnHDDJہCCAcJBAVW lGC XCABBF ACACH UEDEKH BEHDEKV+AODD BBEBENAAHFEBHJCAHCKCAYHHYGU RBMEpVUQPL K GEB_ IGEB\VSPYBM EJDB݁D_ROIC ADD LLBAv3}:o,B.h%L B'aD'GOC~MEDD6wL(w:Uh*k P BNDG&BBhG KG BB C$ VDDD QHGQ KHC|RLn J r[DIz D  kX6SBAAIE"nfiABECAZAErBVAVAXw}|rEdF"b"n/q* }8yF"f&eZV NBDRCAAK N VEYDLDFC ELH"BgpFDDI I VV$CP;uEBbJfmt create.frame close.frame _exits.frameftq/ftq.c 664 0 0 11306 10777501642 10566ustar00johnjohn/** * ftq.c : Fixed Time Quantum microbenchmark * * Written by Matthew Sottile (matt@lanl.gov) * * This is a complete rewrite of the original tscbase code by * Ron and Matt, in order to use a better set of portable timers, * and more flexible argument handling and parameterization. * * Licensed under the terms of the GNU Public Licence. See LICENCE_GPL * for details. */ #include #include /** * macros and defines */ #define MAX_SAMPLES 2000000 #define DEFAULT_COUNT 10000 #define DEFAULT_BITS 20 #define MAX_BITS 30 #define MIN_BITS 3 #define CORE63 #ifdef CORE15 #define MULTIITER #define ITERCOUNT 8 #endif #ifdef CORE31 #define MULTIITER #define ITERCOUNT 16 #endif #ifdef CORE63 #define MULTIITER #define ITERCOUNT 32 #endif /** * global variables */ /* samples: each sample has a timestamp and a work count. */ static unsigned long long samples[MAX_SAMPLES*2]; static uvlong now, last, endinterval; static unsigned long done; /** * usage() */ void usage(char *av0) { fprint(2,"usage: %s [-n samples] [-i bits] [-h] [-o outname] [-s]\n", av0); exits("failure"); } /** * main() */ void main(int argc, char **argv) { /* local variables */ unsigned long numsamples = DEFAULT_COUNT; unsigned long long interval_length; int interval_bits = DEFAULT_BITS; unsigned long long count; char fname_times[1024], fname_counts[1024]; int i; int fp; int use_stdout = 0; char* tmp; #ifdef MULTIITER register int k; #endif #ifdef _WITH_TAU_ TAU_PROFILE("main()", "int (int, char **)", TAU_DEFAULT); TAU_PROFILE_TIMER(mtimer,"main","void (void)", TAU_USER); TAU_PROFILE_TIMER(t1,"coretimer","void (void)", TAU_USER); TAU_PROFILE_TIMER(looptimer,"looptimer","void (void)", TAU_USER); TAU_PROFILE_TIMER(datasave,"datasave","void (void)", TAU_USER); TAU_PROFILE_TIMER(io,"io","void (void)", TAU_USER); TAU_PROFILE_INIT(argc, argv); TAU_PROFILE_SET_NODE(0); TAU_PROFILE_START(mtimer); #endif /* _WITH_TAU_ */ /* default filenames */ sprint(fname_times,"ftq_times.dat"); sprint(fname_counts,"ftq_counts.dat"); ARGBEGIN{ case 's': use_stdout = 1; break; case 'o': tmp = EARGF(usage(argv[0])); sprint(fname_times,"%s_times.dat",tmp); sprint(fname_counts,"%s_counts.dat",tmp); break; case 'i': interval_bits = atoi(EARGF(usage(argv[0]))); break; case 'n': numsamples = atoi(EARGF(usage(argv[0]))); break; case 'h': default: usage(argv[0]); break; }ARGEND fp = create(fname_times,OWRITE,0666); if (fp < 0) { fprint(2, "couldn't open file...\n"); exits("failure"); } /* sanity check */ if (numsamples > MAX_SAMPLES) { fprint(2,"WARNING: sample count exceeds maximum.\n"); fprint(2," setting count to maximum.\n"); numsamples = MAX_SAMPLES; } if (interval_bits > MAX_BITS || interval_bits < MIN_BITS) { fprint(2,"WARNING: interval bits invalid. set to %d.\n", MAX_BITS); interval_bits = MAX_BITS; } /* set up sampling. first, take a few bogus samples to warm up the cache and pipeline */ interval_length = 1 << interval_bits; done = 0; count = 0; samples[done*2] = 0; cycles(&last); endinterval = (last + interval_length) & (~(interval_length - 1)); #ifdef _WITH_TAU_ TAU_PROFILE_START(t1); #endif /* _WITH_TAU_ */ while (1) { count = 0; #ifdef _WITH_TAU_ TAU_PROFILE_START(looptimer); #endif /* _WITH_TAU_ */ for (now = last; now < endinterval; ) { #ifdef MULTIITER for (k=0;k numsamples) break; cycles(&last); endinterval = (last + interval_length) & (~(interval_length - 1)); } #ifdef _WITH_TAU_ TAU_PROFILE_STOP(t1); #endif /* _WITH_TAU_ */ #ifdef _WITH_TAU_ TAU_PROFILE_START(io); #endif /* _WITH_TAU_ */ if (use_stdout == 1) { for (i=0;i # Description: Plot the time and frequency domain plots of a times and # counts log file pair. # Prerequisites: numpy, scipy, and pylab packages. For debian/ubuntu: # o python-numeric # o python-scipy # o python-matplotlib # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # Copyright (C) IBM Corporation, 2007 # # 2007-Aug-30: Initial version by Darren Hart from numpy import * from numpy.fft import * from scipy import * from pylab import * from sys import * from getopt import * NS_PER_S = 1000000000 NS_PER_MS = 1000000 NS_PER_US = 1000 def smooth(x, wlen): if x.size < wlen: raise ValueError, "Input vector needs to be bigger than window size." # reflect the signal to avoid transients... ? s = r_[2*x[0]-x[wlen:1:-1], x, 2*x[-1]-x[-1:-wlen:-1]] w = hamming(wlen) # generate the smoothed signal y = convolve(w/w.sum(), s, mode='same') # recenter the the smoothed signal over the originals (slide along x) y1 = y[wlen-1:-wlen+1] return y1 def my_fft(x, sample_hz): X = abs(fftshift(fft(x))) freq = fftshift(fftfreq(len(x), 1.0/sample_hz)) return array([freq, abs(X)/len(x)]) def smooth_fft(timefile, countfile, sample_hz, wlen): # The higher the sample_hz, the larger the required wlen (used to generate # the hamming window). It seems that each should be adjusted by roughly the # same factor ns_per_sample = NS_PER_S / sample_hz print "Interpolated Sample Rate: ", sample_hz, " HZ" print "Hamming Window Length: ", wlen t = fromfile(timefile, dtype=int64, sep='\n') x = fromfile(countfile, dtype=int64, sep='\n') # interpolate the data to achieve a uniform sample rate for use in the fft xi_len = (t[len(t)-1] - t[0])/ns_per_sample xi = zeros(xi_len) last_j = 0 for i in range(0, len(t)-1): j = (t[i] - t[0])/ns_per_sample xi[j] = x[i] m = (xi[j]-xi[last_j])/(j-last_j) for k in range(last_j + 1, j): xi[k] = m * (k - last_j) + xi[last_j] last_j = j # smooth the signal (low pass filter) y = smooth(xi, wlen) # generate the fft X = my_fft(xi, sample_hz) Y = my_fft(y, sample_hz) # plot the hamming window subplot(311) plot(hamming(wlen)) axis([0,wlen-1,0,1.1]) title(str(wlen)+" Point Hamming Window") # plot the signals subplot(312) ts = arange(0, len(xi), dtype=float)/sample_hz # time signal in units of seconds plot(ts, xi, alpha=0.2) plot(ts, y) legend(['interpolated', 'smoothed']) title("Counts (interpolated sample rate: "+str(sample_hz)+" HZ)") # plot the fft subplot(313) plot(X[0], X[1], ls='steps', alpha=0.2) plot(Y[0], Y[1], ls='steps') ylim(ymax=20) xlim(xmin=-3000, xmax=3000) legend(['interpolated', 'smoothed']) title("FFT") show() def usage(): print "usage: "+argv[0]+" -t times-file -c counts-file [-s SAMPLING_HZ] [-w WINDOW_LEN]" if __name__=='__main__': try: opts, args = getopt(argv[1:], "c:hs:t:w:") except GetoptError: usage() exit(2) sample_hz = 10000 wlen = 25 times_file = None counts_file = None for o, a in opts: if o == "-c": counts_file = a if o == "-h": usage() exit() if o == "-s": sample_hz = long(a) if o == "-t": times_file = a if o == "-w": wlen = int(a) if not times_file or not counts_file: usage() exit(1) smooth_fft(times_file, counts_file, sample_hz, wlen) 42 10566ustar00johnjohnftq/mkfile 664 0 0 207 10777501672 10763ustar00johnjohn