我试图加快一些python代码只能运行单线程。我在for循环中运行了许多这些并且想要并行化它并将结果保存在字典中。
我搜索了堆栈溢出并阅读了multiprocessing文档,但找不到一个好的解决方案。
未并行化的示例:%%time
# This only uses one thread! It's slow
mydict = {}
for i in range(20000000):
mydict[i] = i**2
返回:CPU times: user 8.13 s, sys: 1.04 s, total: 9.17 s
Wall time: 9.21 s
我的字典是正确的print([mydict[i] for i in range(10)])
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
我尝试并行化:%%time
import multiprocessing as mp
from multiprocessing import Process, Manager
def square(d, i):
d[i] = i**2
with mp.Manager() as manager:
d = manager.dict()
with manager.Pool(processes=4) as pool:
pool.map(square, (d, range(20000000)))
返回:TypeError: square() missing 1 required positional argument: 'i'
预期结果是正确的字典,但时间大约是9.21s的1/4。