[Assorted-commits] SF.net SVN: assorted:[1902] sandbox/trunk/src/py/pandasvsdataframe.py

[<yan...@us...>]

Revision: 1902
          http://sourceforge.net/p/assorted/svn/1902
Author:   yangzhang
Date:     2017-05-12 22:00:12 +0000 (Fri, 12 May 2017)
Log Message:
-----------
Add pandas vs pydataframe benchmark

Added Paths:
-----------
    sandbox/trunk/src/py/pandasvsdataframe.py

Added: sandbox/trunk/src/py/pandasvsdataframe.py
===================================================================
--- sandbox/trunk/src/py/pandasvsdataframe.py	                        (rev 0)
+++ sandbox/trunk/src/py/pandasvsdataframe.py	2017-05-12 22:00:12 UTC (rev 1902)
@@ -0,0 +1,38 @@
+import numpy as np
+import pandas
+import pydataframe as pdf
+import random
+import timeit
+
+r = random.Random()
+r.seed(0)
+
+words=set(open('/usr/share/dict/words').read().split('\n')[:-1])
+xs = r.sample(words,10)
+vals = np.array([r.choice(xs) for i in xrange(100000)])
+
+cat = pandas.Categorical(vals)
+df = pandas.DataFrame(dict(name=cat),index=range(len(cat)))
+df['name2'] = cat.labels
+
+fac = pdf.Factor(vals)
+DF = pdf.DataFrame({'name':fac})
+
+print timeit.repeat(lambda: pandas.Categorical(vals), number=100)
+print timeit.repeat(lambda: pdf.Factor(vals), number=100)
+# [1.7276630401611328, 1.7099509239196777, 1.699409008026123]
+# [5.793086051940918, 5.864365100860596, 5.855109930038452]
+
+print timeit.repeat(lambda: (vals == 'Kane').sum(), number=100)
+print timeit.repeat(lambda: (cat == 'Kane').sum(), number=100)
+print timeit.repeat(lambda: (fac == 'Kane').sum(), number=100)
+# [0.17582988739013672, 0.1752328872680664, 0.1806349754333496]
+# [0.0377810001373291, 0.038002967834472656, 0.03856086730957031]
+# [0.038198232650756836, 0.0375368595123291, 0.03746795654296875]
+
+print timeit.repeat(lambda: (df['name'] == 'Kane').sum(), number=100)
+print timeit.repeat(lambda: (df['name2'] == 1).sum(), number=100)
+print timeit.repeat(lambda: (DF.get_column('name') == 'Kane').sum(), number=100)
+# [1.3420658111572266, 1.3421308994293213, 1.3467140197753906]
+# [0.22314882278442383, 0.22048115730285645, 0.22372817993164062]
+# [0.03833794593811035, 0.03797793388366699, 0.03791189193725586]

This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.