RandomForestClassifier .fit在ec2上因内存错误而失败，但在本地运行时没有错误

我正在包含tfidf和尺寸为（42238，155085）的其他要素的熊猫数据框上拟合随机森林分类器。大小为26GB。在本地训练模型时，代码运行没有错误（尽管速度很慢），但是，在内存为4倍的ec2实例上，当内存利用率达到37％时，训练过程会因内存错误而终止。这在6个运行时执行中是一致的。一切都与anaconda 2.7，代码和数据相同，为什么此过程在ec2上失败？

我正在16GB macbook pro上本地运行模型。ec2实例具有64GB内存。

我尝试过的

1. n_jobs = 1
2. n_estimators = 10
3. max_depth = 10
4. 增加EC2上的交换空间
5. 将最大锁定内存设置为无限制

ec2实例上的ulimit -a。

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 251728
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 251728
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

码

model = RandomForestClassifier(max_depth=None, 
                       max_features=0.2, 
                       min_samples_leaf=1, 
                       min_samples_split=2,
                       n_estimators=10, 
                       n_jobs=-1,
                       verbose=1)
model.fit(X_train,y_train)

堆栈跟踪

model.fit(X_train,y_train)
  File "/opt/anaconda2/lib/python2.7/site-packages/sklearn/ensemble/forest.py", line 247, in fit
    X = check_array(X, accept_sparse="csc", dtype=DTYPE)
  File "/opt/anaconda2/lib/python2.7/site-packages/sklearn/utils/validation.py", line 433, in check_array
    array = np.array(array, dtype=dtype, order=order, copy=copy)
  File "/opt/anaconda2/lib/python2.7/site-packages/pandas/core/generic.py", line 1603, in __array__
    return com._values_from_object(self)
  File "pandas/_libs/lib.pyx", line 47, in pandas._libs.lib.values_from_object
  File "/opt/anaconda2/lib/python2.7/site-packages/pandas/core/generic.py", line 4684, in get_values
    return self.values
  File "/opt/anaconda2/lib/python2.7/site-packages/pandas/core/generic.py", line 4629, in values
    return self._data.as_array(transpose=self._AXIS_REVERSED)
  File "/opt/anaconda2/lib/python2.7/site-packages/pandas/core/internals.py", line 3949, in as_array
    arr = mgr._interleave()
  File "/opt/anaconda2/lib/python2.7/site-packages/pandas/core/internals.py", line 3960, in _interleave
    result = np.empty(self.shape, dtype=dtype)
MemoryError