I would like to parallelise the for loop in fun2() of the following code but I cannot figure out how. I would like to keep the function inside of class2 and that seems to be why mapping to multiprocessing.Pool does not work. But I was also unable to make it work with multiprocessing.pool.ThreadPool.
class class1():
def __init__(self, x):
self.var1 = x
def fun1(self):
self.result = self.var1**2
class class2():
def __init__(self, y):
self.var2 = y
def fun2(self):
for v in self.var2:
obj = class1(v)
obj.fun1()
setattr(self, 'obj_{}'.format(v), obj)
obj2 = class2(range(5))
obj2.fun2()
This code works but is very slow in my case as fun1 is very expensive in my real programm. Any help is very much appreciated. Thank you!
Here's a way to do that. The change is in class2
class class1():
def __init__(self, x):
self.var1 = x
def fun1(self):
time.sleep(1) #simulate a long-running job
self.result = self.var1**2
print (self.result)
class class2():
def __init__(self, y):
self.var2 = y
def _sub_process(self, v):
obj = class1(v)
obj.fun1()
setattr(self, 'obj_{}'.format(v), obj)
print(f"in _subprocess, v is {v}")
def fun2(self):
with multiprocessing.Pool() as p:
print(p.map(self._sub_process, self.var2))
if __name__ == "__main__":
obj2 = class2(range(5))
obj2.fun2()
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments