我有一个脚本,运行似乎很慢,并且使用cProfile(和可视化工具KCacheGrind)进行了概要分析
似乎占用了几乎90%的运行时间的是导入顺序,尤其是_ _ init _ _.py
文件的运行...
这是KCacheGrind输出的屏幕截图(很抱歉,附加图片...)
我对导入序列在python中的工作方式不是很熟悉,所以也许我感到有些困惑...我也将_ _ init _ _.py
文件放置在我的所有自定义软件包中,不确定是否应该这样做。
无论如何,如果有人有任何暗示,不胜感激!
EDIT: additional picture when function are sorted by self:
EDIT2:
here the code attached, for more clarity for the answerers:
from strategy.strategies.gradient_stop_and_target import make_one_trade
from datetime import timedelta, datetime
import pandas as pd
from data.db import get_df, mongo_read_only, save_one, mongo_read_write, save_many
from data.get import get_symbols
from strategy.trades import make_trade, make_mae, get_prices, get_signals, \
get_prices_subset
#from profilehooks import profile
mongo = mongo_read_only()
dollar_stop = 200
dollar_target = 400
period_change = 3
signal = get_df(mongo.signals.signals, strategy = {'$regex' : '^indicators_group'}).iloc[0]
symbol = get_symbols(mongo, description = signal['symbol'])[0]
prices = get_prices(
signal['datetime'],
signal['datetime'].replace(hour = 23, minute = 59),
symbol,
mongo)
make_one_trade(
signal,
prices,
symbol,
dollar_stop,
dollar_target,
period_change)
The function get_prices simply get data from a mongo db database, and make_one_trade does simple calculation with pandas. This never poses problem anywhere else in my project.
EDIT3:
Here the Kcache grind screen when i select 'detect cycle' option in View tab:
Could that actually mean that there are indeed circular imports in my self created packages that takes all that time to resolve?
No. You are conflating cumulative time with time spent in the top-level code of the __init__.py
file itself. The top-level code calls other methods, and those together take a lot of time.
Look at the self column instead to find where all that time is being spent. Also see What is the difference between tottime and cumtime in a python script profiled with cProfile?, the incl. column is the cumulative time, self is the total time.
I'd just filter out all the <frozen importlib.*>
entries; the Python project has already made sure those paths are optimised.
但是,您的第二个屏幕快照确实表明,在性能分析运行中,您自己忙碌的所有Python代码都正在加载字节码以供模块导入(marshal
模块提供了Python字节码序列化实现)。Python程序只执行了导入模块的工作,而没有做其他工作,或者正在使用某种形式的动态导入来装载大量的模块,或者忽略了正常的模块缓存并反复重新装载了相同的模块。
您可以使用Python 3.7的新-X importtime
命令行开关来分析导入时间,也可以使用专用的import-profiler来找出为什么导入需要这么长时间。
本文收集自互联网,转载请注明来源。
如有侵权,请联系 [email protected] 删除。
我来说两句