python multiprocessing.Pool

284354749

示例1：



import multiprocessing
def run(msg):
print '%s, it works!' % msg
def start_process():
print 'Starting',multiprocessing.current_process().name
if __name__=='__main__':
pool = multiprocessing.Pool(processes=2, initializer=start_process,)
pool.apply_async(run('congbo'))
pool.close()
pool.join()
　　运行结果：


　　Starting PoolWorker-1
Starting PoolWorker-2
congbo, it works!
　　pool.apply_async(run('congbo')) 这种使用方式也行？见示例2。
　　

示例2：



import multiprocessing
def run(msg):
print '%s, it works!' % msg
while True:
pass
def start_process():
print 'Starting',multiprocessing.current_process().name
if __name__=='__main__':
pool = multiprocessing.Pool(processes=2, initializer=start_process,)
pool.apply_async(run('congbo'))
　  print "It's my turn!"
pool.close()
pool.join()
　　运行结果：



Starting PoolWorker-1
Starting PoolWorker-2
congbo, it works!
...
...
　　没有打印出“It's my turn!”，程序没有切换到主进程接着执行，子进程死循环。
　　说明这种使用方式是不对的，只能像 apply_async 原型那样使用。见示例3。
　　

示例3：



import multiprocessing
def run(msg):
print '%s, it works!' % msg
def start_process():
print 'Starting',multiprocessing.current_process().name
if __name__=='__main__':
pool = multiprocessing.Pool(processes=2, initializer=start_process,)
pool.apply_async(run, args=('congbo',))
pool.close()
pool.join()
　　运行结果：


　　Starting PoolWorker-1
Starting PoolWorker-2
congbo, it works!
　　tips：apply_async 中 args 应为元组，故单个参数后面需加逗号。如果参数错误，不成功执行，不报异常！
　　

示例4：



import multiprocessing
class Worker:
def run(self, msg):
print '%s, it works!' % msg
def start_process():
print 'Starting',multiprocessing.current_process().name
if __name__=='__main__':
pool = multiprocessing.Pool(processes=2, initializer=start_process,)
pool.apply_async(Worker().run, args=('congbo',))
pool.close()
pool.join()
　　运行结果：



Starting PoolWorker-1
Starting PoolWorker-2
Exception in thread Thread-2:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 504, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/lib/python2.7/multiprocessing/pool.py", line 319, in _handle_tasks
put(task)
PicklingError: Can't pickle : attribute lookup __builtin__.instancemethod failed
　　

示例5：



import multiprocessing
class Worker:
def run(self, msg):
print '%s, it works!' % msg
def wrap(worker, msg):
worker.run(msg)
def start_process():
print 'Starting',multiprocessing.current_process().name
if __name__=='__main__':
pool = multiprocessing.Pool(processes=2, initializer=start_process,)
    pool.apply_async(wrap, args=(Worker(), 'congbo'))
pool.close()
pool.join()
　　不直接传入类实例的方法，用另一个函数包装一下，将类的实例作为参数传入即可。
　　为什么要这样？有没有更好的办法？
　　
　　参考：http://stackoverflow.com/questions/1816958/cant-pickle-type-instancemethod-when-using-pythons-multiprocessing-pool-ma
　　
　　原文：http://www.iyunv.com/congbo/archive/2012/08/23/2652322.html