python multiprocessing pool 遇到问题

from functools import wraps
from pathos.multiprocessing import ProcessingPool as Pool # to tackle pickle problem, using pathos.multiprocessing rather than multiprocessing

def parallel(func):
    @wraps(func)
    def apply(values):
        pool = Pool()
        result =  pool.map(func, values)
        pool.close()
        pool.join()
        return result
    return apply

程序中我这样使用

@parallel
def square(x):
    return x ** 2

@parallel
def cubic(x):
    return x ** 3

if __name__ == '__main__':
    x = range(10)
    ret1 = square(x)
    ret2 = cubic(ret1)
    print(ret2)

会出现

result =  pool.map(func, values)
  File "/home/phil/anaconda2/lib/python2.7/site-packages/pathos/multiprocessing.py", line 133, in map
    return _pool.map(star(f), zip(*args)) # chunksize
  File "/home/phil/anaconda2/lib/python2.7/site-packages/multiprocess/pool.py", line 250, in map
    assert self._state == RUN
AssertionError

问题是，如果注释掉 square 或者 cubic 任意一个的装饰器，程序是可以运行的。

请教该如何做？

1 条回复 • 2016-01-19 15:52:15 +08:00

billgreen1

2016-01-19 15:52:15 +08:00

注释掉
pool.close()
pool.join()
可行。

但是大多数程序给出的代码都有这两句的。