在 C++ 中嵌入使用 Numpy 的 Python 在运行时动态加载的库中不起作用

灯头

我想通过动态加载的库 ( dlopen/ dlsym)在 Ubuntu Linux 的 C++ 中嵌入 Python 当我嵌入一个numpy通过单个 C++ 静态链接的可执行文件导入的 Python 模块时,一切正常。当我使用动态加载的库而不尝试导入时,numpy一切正常。但是,当我尝试numpy从动态加载的库中导入时,它失败了。


首先,这是一个没有动态库的简单示例

test_static.cpp

#include <iostream>
#include <Python.h>

int main(){

  Py_Initialize();

  PyObject* pName = PyUnicode_FromString("mymodule");
  PyObject* pModule = PyImport_Import(pName);
  if (!pModule){
    std::cout << "PyImport_Import() failed for " << "test" << std::endl;
    PyErr_Print();
    exit(-1);
  }

  PyObject* pDict = PyModule_GetDict(pModule);
  if (!pDict){
    std::cout << "PyModule_GetDict() failed" << std::endl;
    exit(-1);
  }
  PyObject* pClass = PyDict_GetItemString(pDict,"myclass");

  if (!PyCallable_Check(pClass)) {
    std::cout << "PyCallable_Check(pClass) failed" << std::endl;
    exit(-1);
  }
  PyObject* pInstance = PyObject_CallObject(pClass, NULL);

  pName = PyUnicode_FromString("mymethod");
  PyObject* result=PyObject_CallMethodObjArgs(pInstance, pName,NULL);
  if (!result){
    std::cout << "Error calling PyObject_CallMethodObjArgs" << std::endl;
    PyErr_Print();
    exit(-1);
  }
  Py_DECREF(pName);
  Py_DECREF(pModule);
  Py_Finalize();

  return 0;
}

这是用

g++ -g -O0 -o test_embed_python_static test_static.cpp -I/usr/include/python3.7m -lpython3.7m

我的 Python 模块是:

我的模块.py

import numpy as np

class myclass:
  def __init__(self):
    self.something="asdf"
    self.a=np.zeros( (5),dtype=float)
  def mymethod(self):
    print("Called myclass.mymethod()")
    print('%d'%(self.a.size))

这个 Python 模块位于本地目录中,我设置了环境变量 export PYTHONPATH=.

输出是:

user@host:~/test_embed_python$ ./test_embed_python_static
Called myclass.mymethod()
5

这是使用动态库的示例:

test_dynamic.cpp

#include <dlfcn.h>
typedef void MyCppFunctionHandle();
int main(){
  void* soHandle = dlopen("/home/user/lib/libtestlib.so", RTLD_NOW );
  MyCppFunctionHandle* functionHandle=(MyCppFunctionHandle*)dlsym(soHandle,"MyCppFunction");
  functionHandle();
  return 0;
}

test_lib.cpp

#include <iostream>
#include <Python.h>

extern "C"
void MyCppFunction(){

  Py_Initialize();

  PyObject* pName = PyUnicode_FromString("mymodule");
  PyObject* pModule = PyImport_Import(pName);
  if (!pModule){
    std::cout << "PyImport_Import() failed for " << "mymodule" << std::endl;
    PyErr_Print();
    exit(-1);
  }

  PyObject* pDict = PyModule_GetDict(pModule);
  if (!pDict){
    std::cout << "PyModule_GetDict() failed" << std::endl;
    exit(-1);
  }

  PyObject* pClass = PyDict_GetItemString(pDict,"myclass");

  if (!PyCallable_Check(pClass)) {
    std::cout << "PyCallable_Check(pClass) failed" << std::endl;
    exit(-1);
  }
  PyObject* pInstance = PyObject_CallObject(pClass, NULL);

  pName = PyUnicode_FromString("mymethod");
  PyObject* result=PyObject_CallMethodObjArgs(pInstance, pName,NULL);
  if (!result){
    std::cout << "Error calling PyObject_CallMethodObjArgs" << std::endl;
    PyErr_Print();
    exit(-1);
  }
  Py_DECREF(pName);
  Py_DECREF(pModule);
  Py_Finalize();
}

构建库和主要可执行文件:

g++ -g -O0 -fPIC -shared -o libtestlib.so test_lib.cpp -I/usr/include/python3.7m -lpython3.7m
mv libtestlib.so ~/lib/
g++ -g -O0 -o test_embed_python_dynamic test_dynamic.cpp -ldl

使用与mymodule.py上述相同再次PYTHONPATH设置.

输出:

user@host:~/test_embed_python$ ./test_embed_python_dynamic 
Dynamic library opened successfully
Function loaded successfully
Calling function
PyImport_Import() failed for mymodule
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/numpy/core/__init__.py", line 40, in <module>
    from . import multiarray
  File "/usr/lib/python3/dist-packages/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/usr/lib/python3/dist-packages/numpy/core/overrides.py", line 6, in <module>
    from numpy.core._multiarray_umath import (
ImportError: /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so: undefined symbol: PyExc_ImportError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/code/test/test_embed_python/mymodule.py", line 1, in <module>
    import numpy as np
  File "/usr/lib/python3/dist-packages/numpy/__init__.py", line 142, in <module>
    from . import core
  File "/usr/lib/python3/dist-packages/numpy/core/__init__.py", line 71, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
Here is how to proceed:
- If you're working with a numpy git repository, try `git clean -xdf`
  (removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed:
  your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
  1. Check that you are using the Python you expect (you're using /usr/bin/python3),
     and that you have no directories in your PATH or PYTHONPATH that can
     interfere with the Python and numpy versions you're trying to use.
  2. If (1) looks fine, you can open a new issue at
     https://github.com/numpy/numpy/issues.  Please include details on:
     - how you installed Python
     - how you installed numpy
     - your operating system
     - whether or not you have multiple versions of Python installed
     - if you built from source, your compiler versions and ideally a build log

     Note: this error has many possible causes, so please don't comment on
     an existing issue about this - open a new one instead.

Original error was: /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so: undefined symbol: PyExc_ImportError

Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
    from apport.report import Report
  File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in <module>
    import apport.fileutils
  File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in <module>
    from apport.packaging_impl import impl as packaging
  File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 23, in <module>
    import apt
  File "/usr/lib/python3/dist-packages/apt/__init__.py", line 23, in <module>
    import apt_pkg
ImportError: /usr/lib/python3/dist-packages/apt_pkg.cpython-37m-x86_64-linux-gnu.so: undefined symbol: PyExc_ValueError

Original exception was:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/numpy/core/__init__.py", line 40, in <module>
    from . import multiarray
  File "/usr/lib/python3/dist-packages/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/usr/lib/python3/dist-packages/numpy/core/overrides.py", line 6, in <module>
    from numpy.core._multiarray_umath import (
ImportError: /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so: undefined symbol: PyExc_ImportError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/test_embed_python/mymodule.py", line 1, in <module>
    import numpy as np
  File "/usr/lib/python3/dist-packages/numpy/__init__.py", line 142, in <module>
    from . import core
  File "/usr/lib/python3/dist-packages/numpy/core/__init__.py", line 71, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
Here is how to proceed:
- If you're working with a numpy git repository, try `git clean -xdf`
  (removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed:
  your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
  1. Check that you are using the Python you expect (you're using /usr/bin/python3),
     and that you have no directories in your PATH or PYTHONPATH that can
     interfere with the Python and numpy versions you're trying to use.
  2. If (1) looks fine, you can open a new issue at
     https://github.com/numpy/numpy/issues.  Please include details on:
     - how you installed Python
     - how you installed numpy
     - your operating system
     - whether or not you have multiple versions of Python installed
     - if you built from source, your compiler versions and ideally a build log


     Note: this error has many possible causes, so please don't comment on
     an existing issue about this - open a new one instead.

Original error was: /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so: undefined symbol: PyExc_ImportError

Numpy Github 上的这个问题看起来很相似,但从未发布过实际的解决方案:

https://github.com/numpy/numpy/issues/14480

灯头

我在这里偶然发现了解决方案我仍然不清楚问题是什么以及为什么要解决这个问题,但添加了这一行

void*const libpython_handle = dlopen("libpython3.7m.so", RTLD_LAZY | RTLD_GLOBAL);

之前Py_Initialize();test_lib.cpp解决了这个问题。

本文收集自互联网,转载请注明来源。

如有侵权,请联系 [email protected] 删除。

编辑于
0

我来说两句

0 条评论
登录 后参与评论

相关文章

在C ++中嵌入python / numpy

在运行时创建dll文件在C#中不起作用

为什么从 C# 运行时 python 中的相对路径不起作用,但在 C# 之外使用相同的解释器时却起作用?

在C#中在运行时加载DLL

在C / C ++中嵌入Python

在运行时在Objective-C中检测并使用可选的外部C库

从C ++应用程序中的嵌入式Python调用时,Numpy导入在多数组扩展库上失败

在运行时渲染DOM在Dojox中不起作用

当地图正在运行时,React中的过滤器方法不起作用

在python中复制numpy数组后加法不起作用

使用 python.net 在 C# 中嵌入 python 时的内存管理

使用&在后台运行时,Nohup for Python脚本不起作用

是否应该为可以在运行时在cython中创建的numpy数组动态分配内存?

将日期插入表变量在SSMS中不起作用,但在运行时起作用

在C ++中嵌入python:奇怪的分段错误

在C应用程序中嵌入Python

使用pybind11将Python解释器嵌入到多线程C ++程序中

在C ++程序中嵌入的python脚本中使用matplotlib时出错

使用pyinstaller在Python中嵌入.wav文件

使用 Python 中的嵌入属性解析 XML

使用Visual Studio在C ++中嵌入Julia

Python 中的 C++ 模块 - 获取运行时路径?

在运行时从python Literal类型中获取文字?

在 Python 中在运行时附加列表

如何在运行时在C中检测OS

在pycharm中运行时子进程不起作用

运行时 eval() 中的 jQuery 不起作用

C#复制到剪贴板在运行时不起作用

尝试在Winamp的in_midi.dll中错误地加载C运行时库