python - Perfomance of map vs starmap? -


i trying make pure-python (without external dependencies) element-wise comparison of 2 sequences. first solution was:

list(map(operator.eq, seq1, seq2))

then found starmap function itertools, seemed pretty similar me. turned out 37% faster on computer in worst case. not obvious me, measured time necessary retrieve 1 element generator (don't know if way correct):

from operator import eq itertools import starmap  seq1 = [1,2,3]*10000 seq2 = [1,2,3]*10000 seq2[-1] = 5  gen1 = map(eq, seq1, seq2)) gen2 = starmap(eq, zip(seq1, seq2))  %timeit -n1000 -r10 next(gen1) %timeit -n1000 -r10 next(gen2)  271 ns ± 1.26 ns per loop (mean ± std. dev. of 10 runs, 1000 loops each) 208 ns ± 1.72 ns per loop (mean ± std. dev. of 10 runs, 1000 loops each)

in retrieving elements second solution 24% more perfomant. after both produce same results list. somewhere gain 13% in time:

%timeit list(map(eq, seq1, seq2)) %timeit list(starmap(eq, zip(seq1, seq2)))  5.24 ms ± 29.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) 3.34 ms ± 84.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

i don't know how dig deeper in profiling of such nested code? question why first generator faster in retrieving , gain 13% in list function?

edit: first intention perform element-wise comparison instead of all, all function replaced list, replacement not affect timing ratio.

python 3.6.2 on windows 10 (64bit)

there several factors contribute (in conjunction) observed performance difference:

  • zip re-uses returned tuple if has reference count of 1 when next __next__ call made.
  • map builds new tuple passed "mapped function" every time __next__ call made. won't create new tuple scratch because python maintains storage unused tuples. in case map has find unused tuple of right size.
  • starmap checks if next item in iterable of type tuple , if passes on.
  • calling c function within c code pyobject_call won't create new tuple passed callee.

so starmap zip use 1 tuple on , on again passed operator.eq reducing function call overhead immensely. map on other hand create new tuple (or fill c array cpython 3.6 on) every time operator.eq called. speed difference tuple creation overhead.

instead of linking source code i'll provide cython code can used verify this:

in [1]: %load_ext cython  in [2]: %%cython    ...:    ...: cpython.ref cimport py_decref    ...:    ...: cpdef func(zipper):    ...:     = next(zipper)    ...:     print('a', a)    ...:     py_decref(a)    ...:     b = next(zipper)    ...:     print('a', a)  in [3]: func(zip([1, 2], [1, 2])) (1, 1) (2, 2)

yes, tuples aren't immutable, simple py_decref sufficient "trick" zip believing noone else holds reference returned tuple!

as "tuple-pass-thru":

in [4]: %%cython    ...:    ...: def func_inner(*args):    ...:     print(id(args))    ...:    ...: def func(*args):    ...:     print(id(args))    ...:     func_inner(*args)  in [5]: func(1, 2) 1404350461320 1404350461320

so tuple passed right through (just because these defined c functions!) doesn't happen pure python functions:

in [6]: def func_inner(*args):    ...:     print(id(args))    ...:    ...: def func(*args):    ...:     print(id(args))    ...:     func_inner(*args)    ...:  in [7]: func(1, 2) 1404350436488 1404352833800

note doesn't happen if called function isn't c function if called c function:

in [8]: %%cython    ...:     ...: def func_inner_c(*args):    ...:     print(id(args))    ...:     ...: def func(inner, *args):    ...:     print(id(args))    ...:     inner(*args)    ...:  in [9]: def func_inner_py(*args):     ...:     print(id(args))     ...:     ...:  in [10]: func(func_inner_py, 1, 2) 1404350471944 1404353010184  in [11]: func(func_inner_c, 1, 2) 1404344354824 1404344354824

so there lot of "coincidences" leading point starmap zip faster calling map multiple arguments when called function c function...


Comments

Post a Comment

Popular posts from this blog

angular - Ionic slides - dynamically add slides before and after -

hi i'm using ngfor create set of 3 slides while starting in middle i'm guaranteed able slide left or right on start. when slide right can simple listen reachedend , push slide array i'm looping. but have problem adding slide beginning. if same above , use e.g. array.unshift() or spread add item beginning, view think it's on position 0 , snaps view new slide. the code below work animates slide change index 1. slide = [0,1,2] //example loop slidechanged(event) { if(this.slides.isbeginning()){ this.slide = [this.slide[0]-1, ...this.slide]; this.slides.update(); this.slides.slideto(1) } } <ion-slides [initialslide]="1" (ionslidedidchange)="slidechanged($event)"> <ion-slide *ngfor="let item of slide"> <h1>slide {{item}}</h1> </ion-slide> </ion-slides> any appreciated! you can using ionslidenextend , ionslideprevend events slides . please...
Read more

Add a dynamic header in angular 2 http provider -

i have http interceptor built via tutorial https://scotch.io/@kashyapmukkamala/using-http-interceptor-with-angular2 the problem need have dynamic header changes automatically once changes 'mode' in app. service able tell interceptor change header, or have interceptor pull header (which string)from service each request. able throwing away patch method, call http.patch('new header string') set header string. call in service controls 'mode' setting. wondering if there way interceptor pull recent value service each request or if can add method http provider via interceptor? i've tried call service in interceptor each request returns undefined. because creating new instance of service or copying value before service fired , loaded/changed data. i don't want pass header string every http provider call because defeats purpose of having interceptor, id able set once changes or have provider fetch service before makes request. tl;dr : need add autom...
Read more

minify - Minimizing css files -

i want know how css files can minimized. currently, using external sites css minifier compress css files. however, after removing whitespaces, comments, etc, file sizes reduced 10-15%. how frameworks bootstrap manage make minimized versions kilobytes. as example, css is: body { background-color: #ff0; } the reduction goes 28 bytes 22 bytes other css files have lot more characters still smaller this. actually minifying code means it's remove thing code white space characters new line characters comments block delimiters it reduces amount of code has transferred on web , speed site load fastly.but unreadable format.
Read more