ModuleNotFoundError when packaging a Python project

I've developed a Python-based simulator and want to package it for the use of public.

Here is my setup.py:

import os

import setuptools
from setuptools import setup

from mypackage.version import __name__, __version__, __author__, __email__, __license__, __url__, \
    __description__, __keywords__, __classifiers__, __python_requires__
# ---------------------------------------------------------------------------------------------------------
# GENERAL
# ---------------------------------------------------------------------------------------------------------

thelibFolder = os.path.dirname(os.path.realpath(__file__))
requirementPath = thelibFolder + '/requirements.txt'
install_requires = []
if os.path.isfile(requirementPath):
    with open(requirementPath) as f:
        install_requires = f.read().splitlines()

data = dict(
    name=__name__,
    version=__version__,
    author=__author__,
    url=__url__,
    python_requires=__python_requires__,
    author_email=__email__,
    description=__description__,
    license=__license__,
    keywords=__keywords__,
    install_requires=install_requires,
    platforms='any',
    classifiers=__classifiers__
    # py_modules=['ipython']
)


# ---------------------------------------------------------------------------------------------------------
# OTHER METADATA
# ---------------------------------------------------------------------------------------------------------


# update the readme.rst to be part of setup
def readme():
    with open('README.md') as f:
        return f.read()


def packages():
    return ["mypackage"] + ["mypackage." + e for e in setuptools.find_packages(where='mypackage')]


data['long_description'] = readme()
data['long_description_content_type'] = 'text/x-rst'
data['packages'] = packages()
data['include_package_data'] = True


# ============================================================
# SETUP
# ============================================================


def run_setup(setup_args):
    # retrieve the original input arguments and execute default setup
    kwargs = setup_args
    setup(**kwargs)

    print('*' * 74)
    print("Python installation succeeded.")
    print('*' * 74)


run_setup(data)

And here is my requirements.txt:

ipython==7.29.0
ipython-genutils==0.2.0
attrs==21.2.0
cachetools==4.2.4
certifi==2021.10.8
chardet==4.0.0
cloudpickle==2.0.0
cycler==0.11.0
decorator==5.1.0
future==0.18.2
google-auth==2.3.3
gym==0.21.0
idna==3.3
iniconfig==1.1.1
kiwisolver==1.3.2
kubernetes==19.15.0
matplotlib==3.4.3
networkx==2.6.3
numpy==1.21.4
oauthlib==3.1.1
overloading==0.5.0
packaging==21.2
pandas==1.3.4
Pillow==8.4.0
pluggy==1.0.0
py==1.11.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pyglet==1.5.21
pyparsing==3.0.6
pytest==6.2.5
python-dateutil==2.8.2
pytz==2021.3
PyYAML==6.0
requests==2.26.0
requests-oauthlib==1.3.0
rsa==4.7.2
scipy==1.7.2
seaborn==0.11.2
simpy==4.0.1
six==1.16.0
sortedcontainers==2.4.0
toml==0.10.2
typing==3.7.4.3
urllib3==1.26.7
websocket-client==1.2.1
prettytable==2.4.0

Then, I run the following to make the package: python3 setup.py sdist

It creates the package for me under dist, and then I can successfully upload it to a hosted repository (in my case gemfury.com). Of course, I can then pull it in a local project on my machine and can use my own package.

Then I try to create another python project, to basically use this package:

In requirements.txt:

--extra-index-url https://<my-token>@pypi.fury.io/<my-username>/
mypackage==0.1.0

In main.py:

from mypackage import MyServer

MyServer.run()

MyServer has a from IPython.display import Image, display in the beginning.

Then I make a Dockerfile to containerize this project:

FROM python:3.10
ADD main.py /
ADD requirements.txt /
RUN pip install -r requirements.txt
CMD [ "python", "./main.py"]

I then run the Dockerfile, it successfully pull mypackage, but then I get the following error:

...
    from IPython.display import Image, display
ModuleNotFoundError: No module named 'IPython'

I still get the same error even if I manually add ipython to requirements.txt file in my dockerized project (in addition to having it requirements.txt in mypackage),

Any ideas?

Update 1: Here is the output of my docker build (when I just have mypackage in requirements):

> Preparing build context archive...
> [==================================================>]1153/1153 files
> Done
> 
> Sending build context to Docker daemon...
> [==================================================>] 3.788MB Done
> 
> Step 1/5 : FROM python:3.10  ---> 4246fb19839f 
> Step 2/5 : ADD main.py
> /  ---> Using cache  ---> 8dab230b6201 
> Step 3/5 : ADD requirements.txt
> /  ---> Using cache  ---> 2c56cb36982b 
> Step 4/5 : RUN pip install -r
> requirements.txt  ---> Running in e5ec7c02aaea Looking in indexes:
> https://pypi.org/simple, https://****@pypi.fury.io/<user-name>/
> Collecting mypackage==0.1.0   Downloading
> https://pypi.fury.io/<user-name>/-/ver_SqO4f/mypackage-0.1.0.tar.gz
> (62 kB)
>     ERROR: Command errored out with exit status 1:
>      command: /usr/local/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] =
> '"'"'/tmp/pip-install-z_ofzy_s/mypackage_<some_number_here>/setup.py'"'"';
> __file__='"'"'/tmp/pip-install-z_ofzy_s/mypackage_<some_number_here>/setup.py'"'"';f
> = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import
> setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"',
> '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))'
> egg_info --egg-base /tmp/pip-pip-egg-info-fz0vejus
>          cwd: /tmp/pip-install-z_ofzy_s/mypackage_<some_number_here>/
>     Complete output (9 lines):
>     Traceback (most recent call last):
>       File "<string>", line 1, in <module>
>       File "/tmp/pip-install-z_ofzy_s/mypackage_<some_number_here>/setup.py",
> line 6, in <module>
>         from mypackage.version import __name__, __version__, __author__, __email__, __license__, __url__, \
>       File "/tmp/pip-install-z_ofzy_s/mypackage_<some_number_here>/mypackage/__init__.py",
> line 16, in <module>
>         from .helpers.utils import Utils
>       File "/tmp/pip-install-z_ofzy_s/mypackage_<some_number_here>/mypackage/helpers/utils.py",
> line 11, in <module>
>         from IPython.display import Image, display
>     ModuleNotFoundError: No module named 'IPython'
>     ---------------------------------------- WARNING: Discarding https://pypi.fury.io/username/-/<somthing>/mypackage-0.1.0.tar.gz#sha256=<some-token>
> (from https://pypi.fury.io/<user-name>/mypackage/)
> (requires-python:>=3.9). Command errored out with exit status 1:
> python setup.py egg_info Check the logs for full command output.
> ERROR: Could not find a version that satisfies the requirement
> mypackage==0.1.0 (from versions: 0.1.0)

and this is when I have copy paste the contents of mypackage requirements.txt to this new dockerized project's requirements.txt:

> Preparing build context archive...
> [==================================================>]1153/1153 files
> Done
> 
> Sending build context to Docker daemon...
> [==================================================>] 3.788MB Done
> 
> Step 1/5 : FROM python:3.10  ---> 4246fb19839f 
> Step 2/5 : ADD main.py
> /  ---> Using cache  ---> 8dab230b6201 
> Step 3/5 : ADD requirements.txt
> /  ---> Using cache  ---> 050eebba6079 
> Step 4/5 : RUN pip install -r
> requirements.txt  ---> Running in edd3d9d868a1 Looking in indexes:
> https://pypi.org/simple, https://****@pypi.fury.io/<user-name>/
> Collecting IPython==7.29.0   Downloading
> ipython-7.29.0-py3-none-any.whl (790 kB) Collecting
> ipython-genutils==0.2.0   Downloading
> ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB) Collecting
> attrs==21.2.0   Downloading attrs-21.2.0-py2.py3-none-any.whl (53 kB)
> Collecting cachetools==4.2.4   Downloading
> cachetools-4.2.4-py3-none-any.whl (10 kB) Collecting
> certifi==2021.10.8   Downloading
> certifi-2021.10.8-py2.py3-none-any.whl (149 kB) Collecting
> chardet==4.0.0   Downloading chardet-4.0.0-py2.py3-none-any.whl (178
> kB) Collecting cloudpickle==2.0.0   Downloading
> cloudpickle-2.0.0-py3-none-any.whl (25 kB) Collecting cycler==0.11.0  
> Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB) Collecting
> decorator==5.1.0   Downloading decorator-5.1.0-py3-none-any.whl (9.1
> kB) Collecting future==0.18.2   Downloading future-0.18.2.tar.gz (829
> kB) Collecting google-auth==2.3.3   Downloading
> google_auth-2.3.3-py2.py3-none-any.whl (155 kB) Collecting gym==0.21.0
> Downloading gym-0.21.0.tar.gz (1.5 MB) Collecting idna==3.3  
> Downloading idna-3.3-py3-none-any.whl (61 kB) Collecting
> iniconfig==1.1.1   Downloading iniconfig-1.1.1-py2.py3-none-any.whl
> (5.0 kB) Collecting kiwisolver==1.3.2   Downloading
> kiwisolver-1.3.2-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
> (1.6 MB) Collecting kubernetes==19.15.0   Downloading
> kubernetes-19.15.0-py2.py3-none-any.whl (1.7 MB) Collecting
> matplotlib==3.4.3   Downloading matplotlib-3.4.3.tar.gz (37.9 MB)
> Collecting networkx==2.6.3   Downloading
> networkx-2.6.3-py3-none-any.whl (1.9 MB) Collecting numpy==1.21.4  
> Using cached
> numpy-1.21.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
> (15.9 MB) Collecting oauthlib==3.1.1   Downloading
> oauthlib-3.1.1-py2.py3-none-any.whl (146 kB) Collecting
> overloading==0.5.0   Downloading overloading-0.5.0-py3-none-any.whl
> (10 kB) Collecting packaging==21.2   Downloading
> packaging-21.2-py3-none-any.whl (40 kB) Collecting pandas==1.3.4  
> Downloading
> pandas-1.3.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
> (11.5 MB) Collecting Pillow==8.4.0   Downloading
> Pillow-8.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
> (3.1 MB) Collecting pluggy==1.0.0   Downloading
> pluggy-1.0.0-py2.py3-none-any.whl (13 kB) Collecting py==1.11.0  
> Downloading py-1.11.0-py2.py3-none-any.whl (98 kB) Collecting
> pyasn1==0.4.8   Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
> Collecting pyasn1-modules==0.2.8   Downloading
> pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB) Collecting
> pyglet==1.5.21   Downloading pyglet-1.5.21-py3-none-any.whl (1.1 MB)
> Collecting pyparsing==3.0.6   Downloading
> pyparsing-3.0.6-py3-none-any.whl (97 kB) Collecting pytest==6.2.5  
> Downloading pytest-6.2.5-py3-none-any.whl (280 kB) Collecting
> python-dateutil==2.8.2   Downloading
> python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB) Collecting
> pytz==2021.3   Downloading pytz-2021.3-py2.py3-none-any.whl (503 kB)
> Collecting PyYAML==6.0   Downloading
> PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
> (682 kB) Collecting requests==2.26.0   Downloading
> requests-2.26.0-py2.py3-none-any.whl (62 kB) Collecting
> requests-oauthlib==1.3.0   Downloading
> requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB) Collecting
> rsa==4.7.2   Downloading rsa-4.7.2-py3-none-any.whl (34 kB) Collecting
> scipy==1.7.2   Downloading
> scipy-1.7.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
> (39.9 MB) Collecting seaborn==0.11.2   Downloading
> seaborn-0.11.2-py3-none-any.whl (292 kB) Collecting simpy==4.0.1  
> Downloading simpy-4.0.1-py2.py3-none-any.whl (29 kB) Collecting
> six==1.16.0   Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
> Collecting sortedcontainers==2.4.0   Downloading
> sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB) Collecting
> toml==0.10.2   Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
> Collecting typing==3.7.4.3   Downloading typing-3.7.4.3.tar.gz (78 kB)
> Collecting urllib3==1.26.7   Downloading
> urllib3-1.26.7-py2.py3-none-any.whl (138 kB) Collecting
> websocket-client==1.2.1   Downloading
> websocket_client-1.2.1-py2.py3-none-any.whl (52 kB) Collecting
> prettytable==2.4.0   Downloading prettytable-2.4.0-py3-none-any.whl
> (24 kB) Collecting mypackage==0.1.0   Downloading
> https://pypi.fury.io/<user-name>/-/ver_SqO4f/mypackage-0.1.0.tar.gz (62
> kB)
>     ERROR: Command errored out with exit status 1:
>      command: /usr/local/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] =
> '"'"'/tmp/pip-install-0c2n4ub7/mypackage_<some_number>/setup.py'"'"';
> __file__='"'"'/tmp/pip-install-0c2n4ub7/mypackage_<some_number>/setup.py'"'"';f
> = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import
> setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"',
> '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))'
> egg_info --egg-base /tmp/pip-pip-egg-info-sogtdamf
>          cwd: /tmp/pip-install-0c2n4ub7/mypackage_<some_number>/
>     Complete output (9 lines):
>     Traceback (most recent call last):
>       File "<string>", line 1, in <module>
>       File "/tmp/pip-install-0c2n4ub7/mypackage_<some_number>/setup.py",
> line 6, in <module>
>         from mypackage.version import __name__, __version__, __author__, __email__, __license__, __url__, \
>       File "/tmp/pip-install-0c2n4ub7/mypackage_<some_number>/mypackage/__init__.py",
> line 16, in <module>
>         from .helpers.utils import Utils
>       File "/tmp/pip-install-0c2n4ub7/mypackage_<some_number>/mypackage/helpers/utils.py",
> line 11, in <module>
>         from IPython.display import Image, display
>     ModuleNotFoundError: No module named 'IPython'

Update 2: After applying the fix that @AKX provided, I successfully managed to create the package. However, after uploading it to Dockerhub (and successfully building it), when trying to pull the image in Kubernetes, I'm still getting the same error:

Traceback (most recent call last):
File "//./main.py", line 1, in <module>
from mypackage import MyServer
File "/usr/local/lib/python3.10/site-packages/mypackage/__init__.py", line 16, in <module>
from .helpers.utils import Utils
File "/usr/local/lib/python3.10/site-packages/mypackage/helpers/utils.py", line 11, in <module>
from IPython.display import Image, display
ModuleNotFoundError: No module named 'IPython'

Here is my DockerHub build output:

    Cloning into '.'...
Warning: Permanently added the RSA host key for IP address '140.82.113.3' to the list of known hosts.
Reset branch 'main'
Your branch is up-to-date with 'origin/main'.
Pulling cache layers for index.docker.io/<user-name>/mypackage-server:latest...
Done!
KernelVersion: 4.4.0-1060-aws
Components: [{u'Version': u'20.10.7', u'Name': u'Engine', u'Details': {u'KernelVersion': u'4.4.0-1060-aws', u'Os': u'linux', u'BuildTime': u'2021-06-02T11:54:58.000000000+00:00', u'ApiVersion': u'1.41', u'MinAPIVersion': u'1.12', u'GitCommit': u'b0f5bc3', u'Arch': u'amd64', u'Experimental': u'false', u'GoVersion': u'go1.13.15'}}, {u'Version': u'1.4.6', u'Name': u'containerd', u'Details': {u'GitCommit': u'd71fcd7d8303cbf684402823e425e9dd2e99285d'}}, {u'Version': u'1.0.0-rc95', u'Name': u'runc', u'Details': {u'GitCommit': u'b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7'}}, {u'Version': u'0.19.0', u'Name': u'docker-init', u'Details': {u'GitCommit': u'de40ad0'}}]
Arch: amd64
BuildTime: 2021-06-02T11:54:58.000000000+00:00
ApiVersion: 1.41
Platform: {u'Name': u'Docker Engine - Community'}
Version: 20.10.7
MinAPIVersion: 1.12
GitCommit: b0f5bc3
Os: linux
GoVersion: go1.13.15
Buildkit: Starting build for index.docker.io/<user-name>/mypackage-server:latest...
WARNING: Support for the legacy ~/.dockercfg configuration file and file-format is deprecated and will be removed in an upcoming release
#1 [internal] load build definition from Dockerfile
#1 sha256:6077744f93da04808b4551164241f0dff8ca03ee31a1632f8850de00d3dbda4f
#1 transferring dockerfile: 177B done
#1 DONE 0.1s
#2 [internal] load .dockerignore
#2 sha256:44ac79ccbd1c88949b7ee0d7437a85339527a18b284b1e72cb8eda89cc88c888
#2 transferring context: 2B done
#2 DONE 0.1s
#3 [internal] load metadata for docker.io/library/python:3.10
#3 sha256:c787ec5cc33e1cbee663ba2529b5d51f0293f2c277b40d9bd37129383a68d5ac
#3 ...
#4 [auth] library/python:pull token for registry-1.docker.io
#4 sha256:c2895d77dca196320af7cf22ec0920eabacf3daa8200c13b1cd2d44d11ff37da
#4 DONE 0.0s
#3 [internal] load metadata for docker.io/library/python:3.10
#3 sha256:c787ec5cc33e1cbee663ba2529b5d51f0293f2c277b40d9bd37129383a68d5ac
#3 DONE 0.6s
#5 importing cache manifest from index.docker.io/<user-name>/mypackage-server:latest
#5 sha256:ddd8fdc8cba780689d29ed78f506612d5adf5f05d571bd20543af43a7b813269
#5 DONE 0.0s
#6 [1/4] FROM docker.io/library/python:3.10@sha256:437585501d11ef4b4b831cf8a6d6611eb526e327006d506bcedcfdea3fde442a
#6 sha256:cc58830b68a0fc6cdd31a0a33aa733dd524565698a34e22761295b380ee99ae3
#6 DONE 0.0s
#7 [internal] load build context
#7 sha256:86ced93d97622264db6635c82c06781baf8b5ec79412cb539264d35babd321dc
#7 transferring context: 236B done
#7 DONE 0.0s
#8 [2/4] ADD main.py /
#8 sha256:0eb04817a5dd4564d1d3ff2c5dd48b3c006b04fd21b8e57d875bcd8a4e1afcb5
#8 CACHED
#9 [3/4] ADD requirements.txt /
#9 sha256:9682f2bb47fc6347109ac84fea0fc94b4d64795a42ca5f2b829f83012425283f
#9 DONE 0.1s
#10 [4/4] RUN pip install --ignore-installed -r requirements.txt
#10 sha256:33b4307e5b034d63fa3cf9de96c09c5364f3654d2dba2c03e27cb862f1aab1e7
#10 2.877 Looking in indexes: https://pypi.org/simple, https://****@pypi.fury.io/<user-name>/
#10 3.208 Collecting mypackage==0.1.1
#10 3.362 Downloading https://pypi.fury.io/<user-name>/-/ver_caosL/mypackage-0.1.1.tar.gz (62 kB)
#10 4.023 Building wheels for collected packages: mypackage
#10 4.024 Building wheel for mypackage (setup.py): started
#10 4.592 Building wheel for mypackage (setup.py): finished with status 'done'
#10 4.593 Created wheel for mypackage: filename=mypackage-0.1.1-py3-none-any.whl size=92482 sha256=f503bba9118c834100339a6f797deed1d7886fde84b1a4104584bd8740deaccf
#10 4.594 Stored in directory: /root/.cache/pip/wheels/80/ec/0c/96f83e1d06f1dfc7d5e39dc619c0aef4a6e293254795eeab78
#10 4.601 Successfully built mypackage
#10 4.669 Installing collected packages: mypackage
#10 4.886 Successfully installed mypackage-0.1.1
#10 4.887 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#10 5.066 WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
#10 5.066 You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
#10 DONE 5.2s
#11 exporting to image
#11 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#11 exporting layers done
#11 writing image sha256:800942f8026a92502f76c9794cddb13f71409c4497de6e85fd86220cc6a906fb done
#11 naming to docker.io/<user-name>/mypackage-server:latest
#11 naming to docker.io/<user-name>/mypackage-server:latest done
#11 DONE 0.0s
#12 exporting cache
#12 sha256:2700d4ef94dee473593c5c614b55b2dedcca7893909811a8f2b48291a1f581e4
#12 preparing build cache for export done
#12 DONE 0.0s
Pushing index.docker.io/<user-name>/mypackage-server:latest...
Done!
Build finished

Update 3: I've updated the log from update 2, and here I will also include the YAML file I'm using for deploying mypackage-server on Kubernetes:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mypackage-server
  namespace: ingress-nginx
  labels:
    app: mypackage-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mypackage-server
  template:
    metadata:
      labels:
        app: mypackage-server
        namespace: ingress-nginx
    spec:
      containers:
        - name: mypackage-server
          image: <user-name>/mypackage-server
          imagePullPolicy: Always
          ports:
            - containerPort: 8081
      imagePullSecrets:
      - name: regcred