我正在尝试为CNN制作大量（〜170万）图像（烛光图和音量）。但是，据我所知，我目前拥有的脚本在每次迭代之后一直在增加其内存使用量，每次迭代大约2-5mb。无论我正在运行脚本的多少个实例，这都会增加直到我的内存完全填满。（其中16gb脚本最终使用11-12gb）。

目标是同时运行脚本的多个实例。我尝试了并行处理，但结果并没有那么好。因此，我只是在使用多个内核。我尝试了很多减少内存使用的方法，但是似乎没有任何效果。

我在VS代码中使用Jupyter笔记本（Python 3.8.5）（anaconda），具有64位Windows系统。16GB RAM和Intel i7 8th gen。

First Cell调用程序包，加载数据并设置参数。

# import required packages 
import matplotlib.dates as mpdates 
import matplotlib.pyplot as plt 
import mplfinance as mpf
import matplotlib as mpl
from PIL import Image
import pandas as pd 
import math as math
import numpy as np
import io   as io
import gc   as gc
import os as os


#set run instance number
run=1

#timeframe
tf = 20

#set_pixels
img_size=56

#colors
col_up = '#00FF00'
col_down = '#FF0000'
col_vol = "#0000FF"

#set directory
direct = "C:/Users/robin/1 - Scriptie/images/"

#loading the data
data1 = pd.read_csv(r'D:\1 - School\Econometrics\2020 - 2021\Scriptie\Explainable AI\Scripts\Data\test_data.csv',header=[0, 1] , index_col = 0 )
data1.index=pd.to_datetime(data1.index)

#subsetting the data
total_symbols = math.floor(len(data1.columns.unique(level=0))/6)
symbols1 = data1.columns.unique(level=0)[(run-1)*total_symbols:run*total_symbols]

#set the plot parameters
mc = mpf.make_marketcolors(up = col_up ,down = col_down, edge='inherit', volume= col_vol, wick='inherit')
s  = mpf.make_mpf_style(marketcolors=mc)   

第二个单元格定义用于绘制图表的函数：

# creating candlestick chart with volume
def plot_candle(i,j,data,symbols,s,mc,direct,img_size, tf):
     
    #slicing data into 30 trading day windows
    data_temp=data[symbols[j]][i-tf:i]  

    #creating and saving the candlestick charts
    buf = io.BytesIO()
    save = dict(fname= buf, rc = (["boxplot.whiskerprops.linewidth",10]), 
                    pad_inches=0,bbox_inches='tight')
    mpf.plot(data_temp,savefig=save, type='candle',style=s, volume=True, axisoff=True,figratio=(1,1),closefig=True)
    buf.seek(0)
    im = Image.open(buf).resize((img_size,img_size))
    im.save(direct+"/"+str(symbols[j])+"/"+str(i-tf+1)+".png", "PNG")
    buf.close()
    plt.close("all")

第三个单元循环遍历数据并调用第二个单元中的函数。

#check if images folder excists, if not, create it. 
if not os.path.exists(direct):
    os.mkdir("C:/Users/robin/1 - Scriptie/images")

for j in range(0,len(symbols1)):

    #Check if symbol folder excists, if not, create it 
    if not os.path.exists(direct+"/"+symbols1[j]):
             os.mkdir(direct + "/"+symbols1[j])

    for i in range(tf,len(data1)) :

        #check if the file has already been created
        if not os.path.exists(direct+"/"+str(symbols1[j])+"/"   +str(i-tf+1)+".png"):
            #call the functions and create the 
            plot_candle(i , j , data1 , symbols1 ,s ,mc ,direct , img_size, tf)
            gc.collect()