尝试运行docplex示例时出现BrokenProcessPool错误

萨那

我正在尝试与python进程池并行运行一些cplex模型。我尝试将作为Windows 10 -spyder 3.6.9上带有docplex的进程池的示例运行运行时出现此错误:

  File "C:/Users/.spyder-py3/docplex_contribs-master/docplex_contribs/src/zoomontecarlo2.py", line 43, in <module>
    main()

  File "C:/Users/.spyder-py3/docplex_contribs-master/docplex_contribs/src/zoomontecarlo2.py", line 36, in main
    allres = run_model_process_pool(model_build_fn=build_sampled_model, nb_process=nb_samples, verbose=True)

  File "C:\Users\.spyder-py3\docplex_contribs-master\docplex_contribs\src\process_pool.py", line 108, in run_model_process_pool
    res = future.result()

  File "C:\Users\AppData\Local\Continuum\anaconda3\lib\concurrent\futures\_base.py", line 425, in result
    return self.__get_result()

  File "C:\Users\AppData\Local\Continuum\anaconda3\lib\concurrent\futures\_base.py", line 384, in __get_result
    raise self._exception

BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

我尝试使用其他机器并将max_worker设置为1,但没有帮助。

编辑:我把我正在使用的代码,使其更加清晰。这是我的process_pool.py:

import concurrent.futures
from concurrent.futures import ProcessPoolExecutor
class ModelRunner(object):
    run_kw = 'run'
    @staticmethod
    def make_result(result, sol):
        # temporary, for now we cannot pickle solutions.
        if sol:
            if result == 'solution':
                return sol
            elif result == 'dict':
                sol_d = sol.as_name_dict()
                sol_d['_objective_value'] = sol.objective_value
                return sol_d
            else:
                # default is objective
                return sol.objective_value
        else:
            return None

    def __init__(self, buildfn, result="objective", verbose=True):
        self.buildfn = buildfn
        self._result = result
        self.verbose = bool(verbose)

    def __call__(self, **kwargs):
        try:
            nrun_arg = kwargs.get(self.run_kw, -1)
            nrun = int(nrun_arg)

        except (KeyError, TypeError):
            print(f"warning: no run number was found in kwargs")
            nrun = -1

        # use the model build function to create one instance
        m = self.buildfn(**kwargs)
        assert m is not None
        mname = m.name
        if self.verbose:
            print('--> begin run #{0} for model {1}'.format(nrun, mname))
        m.name = '%s_%d' % (mname, nrun)

        sol = m.solve()
        if sol:
            timed = m.solve_details.time
            if self.verbose:
                print(
                    '<-- end run #{0} for model {1}, obj={2}, time={3:.2f}s'.format(nrun, m.name, sol.objective_value, timed))
            return self.make_result(self._result, sol)
        else:
            print("*** model {0} has no solution".format(m.name))
            return None


def run_model_process_pool(model_build_fn, nb_process, max_workers=3,
                           result='objective', verbose=True):
    if nb_process <= 2:
        raise ValueError(f"Expecting a number of processes >= 2, {nb_process} was passed")
    pool_runner = ModelRunner(model_build_fn, result=result, verbose=verbose)
    allres = []
    with ProcessPoolExecutor(max_workers=max_workers) as executor:
        import psutil

        future_to_i = {executor.submit(pool_runner, run=i): i for i in range(nb_process)}  
        # executor.shutdown(wait=False)
        for future in concurrent.futures.as_completed(future_to_i):
            print(psutil.virtual_memory())
            res = future.result()
            if res is not None:
                allres.append(res)
            else:
                return None
    return allres

其中一个是zoomontecarlo2.py,它内部具有cplex模型并使用process_pool:

import random
from docplex.mp.model import Model
def build_zoo_mincost_model(nbKids):
    mdl = Model(name='buses')
    nbbus40 = mdl.integer_var(name='nbBus40')
    nbbus30 = mdl.integer_var(name='nbBus30')
    costBus40 = 500.0
    costBus30 = 400.0
    mdl.add_constraint(nbbus40 * 40 + nbbus30 * 30 >= nbKids, 'kids')
    mdl.minimize(nbbus40 * costBus40 + nbbus30 * costBus30)
    return mdl

nb_kids = 300
max_floating = 30
nb_samples = 50
samples = [random.randint(-max_floating, max_floating) for _ in range(nb_samples)]
def build_sampled_model(**kwargs):
    nrun = kwargs.pop('run', -1)
    nb_floating = samples[nrun % nb_samples]
    print(f"-- running kids model with {nb_floating} floating kids")
    return build_zoo_mincost_model(300 + nb_floating)
def main():
    from process_pool import run_model_process_pool

    samples = [random.randint(-max_floating, max_floating) for _ in range(nb_samples)]
    allres = run_model_process_pool(model_build_fn=build_sampled_model, nb_process=nb_samples, verbose=True)
    mean_cost = sum(allres) / nb_samples
    print(f"* monte carlo, #samples={nb_samples}, max. absents={max_floating}, mean cost is {mean_cost}")
    print(allres)
   
if __name__ == "__main__":
    main()

当引擎进入“未来的并发.futures.as_completed(future_to_i)”循环时,内存信息为:

svmem(总数= 17091981312,可用= 9288286208,百分比= 45.7,已使用= 7803695104,空闲= 9288286208)

当到达“ res = future.result()”时,它因上面的错误而崩溃。

丹尼尔·荣格拉斯

将我的评论之一变成答案:您必须弄清楚该过程突然被杀死的原因是什么。一个潜在的原因是它用尽了内存。如果进程内存不足,则操作系统可能会杀死该进程,请进一步注意。

根据您的评论,这确实可能是这里发生的事情。

本文收集自互联网,转载请注明来源。

如有侵权,请联系 [email protected] 删除。

编辑于
0

我来说两句

0 条评论
登录 后参与评论

相关文章

尝试运行Arduino时出现错误

尝试运行Selenium Webdriver(WebdriverJS)的示例测试时出现错误

尝试运行Envoy前代理示例时出现的问题

尝试运行python脚本时出现错误的剧本错误

尝试运行 IF EXISTS 查询时出现语法错误

尝试运行emacsclient时出现“未知终端类型”错误

尝试运行站点时出现 NodeJs 错误

当我尝试运行测试时出现错误

尝试运行Powershell脚本时出现Java ProcessBuilder错误

尝试运行命令时出现Enable-Migrations错误

尝试运行Java代码时程序顶部出现错误

尝试运行统一游戏时出现此错误

尝试运行简单的Spark Streaming Kafka示例时遇到错误

尝试运行服务器时出现Django运行时错误

每当尝试运行此LinkedList删除功能时,为什么会出现分段错误错误?

尝试运行 Scala Play 示例时出错

React App:为什么在尝试运行npm start脚本时出现错误?

尝试运行Shell脚本时出现错误“:命令未找到”

尝试运行 sub 时出现错误“对象变量或未设置块变量”

尝试运行我的机器人命令时出现此错误

尝试运行JS代码时出现未定义的错误

尝试运行spec.js时出现量角器错误105

尝试运行Python脚本时出现令人困惑的错误

尝试运行Angular Universal时出现错误“意外的令牌导入”

出现IOError:[Errno 21]尝试运行PyInstaller时是目录错误

尝试运行龙卷风main.py时出现语法错误

尝试运行Jasper报告时,出现错误“未知的超链接目标0”

尝试运行 PowerShell 脚本时出现“浏览文件夹”错误

当我尝试运行Xcode模拟器时,出现错误“ Stop“(null)”?