matplotlib / matplotlib/mplfinance

Tricks to make saving images quicker?

Open
#636 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
4.4k
Forks
678
PR merge metrics
No merged PRs in 30d

Description

I am using a friends script (so I don't know all the details in this script), but I wonder if there are simple tricks I can do here to make it save quicker.
I want to be able to save around 10k images.

I plan to incorporate multiprocessing to make it even quicker.

Thank you.

```python
def pre_outcome_chart(month, save_dir, sample_df, test = True):
if test:
sample_df = sample_df.sample(100)

for index, row in sample_df.iterrows():
ticker = row['ticker']
input_date = row['entry_date']

#nput_date_string = input_date.strftime('%Y-%m-%d')
startDate = pd.to_datetime(input_date).date() - relativedelta(months=month)
endDate = pd.to_datetime(input_date).date() + relativedelta(days =1)
startDateString = startDate.strftime('%Y%m%d')
endDateString = endDate.strftime('%Y%m%d')

filename = os.path.join(DATA_DIR, ticker + '.csv')
df = pd.read_csv(filename, index_col=0, parse_dates=True)
dt_range = pd.date_range(start=startDate, end=endDate)
df = df[df.index.isin(dt_range)]
# Drop any rows with missing data
df.dropna(inplace=True)

# Compute moving averages
df['SMA50'] = df['Close'].rolling(window=50, min_periods=1).mean()
df['SMA100'] = df['Close'].rolling(window=100, min_periods=1).mean()
df['SMA200'] = df['Close'].rolling(window=200, min_periods=1).mean()

# Define addplots
ema10 = fplt.make_addplot(df['Close'].ewm(span=10, min_periods=1).mean(), color='#CBC3E3')
ema20 = fplt.make_addplot(df['Close'].ewm(span=20, min_periods=1).mean(), color='#87CEEB')
sma50 = fplt.make_addplot(df['SMA50'], color='red')
sma100 = fplt.make_addplot(df['SMA100'], color='yellow')
sma200 = fplt.make_addplot(df['SMA200'], color='white')

# Define market colors and style
mc = fplt.make_marketcolors(up='black',down='#f76757',
edge={'up': '#13eda4', 'down': '#f76757'},
wick={'up': '#13eda4', 'down': '#f76757'},
volume={'up': '#13eda4', 'down': '#f76757'},
)

s = fplt.make_mpf_style(marketcolors=mc,facecolor='black',figcolor='black',
gridcolor='gray',
gridstyle='dotted',
rc={'xtick.color':'white',
'ytick.color':'white',
'axes.labelcolor':'white',
'text.color':'white',
'axes.edgecolor': 'gray',
'grid.alpha': 0.7,
'grid.linewidth': 0.5,
})

# Plot the data
fig, axlist = fplt.plot(
df,
type='candle',
addplot=[ema10, ema20, sma50, sma100, sma200],
style=s,
figsize=(12,6),
update_width_config={'candle_linewidth':1.0, 'candle_width':0.525, 'volume_width': 0.525},
tight_layout=True,
volume=True,
ylabel='Price',
xrotation=0,
returnfig=True
)

# Add legend
ax = axlist[0]
legend_handles = [ax.lines[i] for i in range(len(ax.lines))]
legend_labels = ['EMA10', 'EMA20', 'SMA50', 'SMA100', 'SMA200']
ax.legend(legend_handles, legend_labels)

# Get the index of the input_date candle
input_date_index = df.index.get_loc(pd.to_datetime(input_date, format='%Y-%m-%d').floor('D'))

# Get the coordinates of the input_date candle
x_coord = input_date_index
y_coord = df['High'].iloc[input_date_index]

# Calculate the maximum value of the chart
chart_max = max(df['High'].max(), df[['SMA50', 'SMA100', 'SMA200']].max().max())

# Calculate the maximum range of the chart
chart_range = df['High'].max() - df['Low'].min()

# Calculate the desired arrow length as a fraction of the maximum range
arrow_length_fraction = 0.1 # Adjust this value to control the arrow length
arrow_length = arrow_length_fraction * chart_range

# Calculate the desired gap size
gap_size = arrow_length

# Adjust the ylim to create the gap
axlist[0].set_ylim(df['Low'].min(), 1.2 * df['High'].max() + gap_size)

# Annotate the input_date above the input_date candle
axlist[0].annotate('',
xy=(x_coord, y_coord),
xytext=(x_coord, y_coord+arrow_length),
arrowprops=dict(arrowstyle='->', color='yellow', linewidth=3),
color='yellow', ha='center', va='bottom')

# Add the date tag at the top of the chart
ax.annotate(ticker + ' ' + pd.to_datetime(input_date).date().strftime('%Y-%m-%d'),
xy=(x_coord, y_coord),
xytext=(x_coord, chart_max+arrow_length),
color='yellow', ha='center', va='bottom')
# Save chart

plt.savefig(
f"{save_dir}/"
+ f"{ticker}_"
+ input_date
+ 'M_'
+ '.png', dpi=300, bbox_inches='tight')
plt.close(fig)`
```

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the pre_outcome_chart function and measure the time spent in pd.read_csv, fplt.plot, and plt.savefig for the 10k-image workload. Compare the current loop with the proposed multiprocessing approach and define completion as a measured, reproducible reduction in total save time without changing the generated charts.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-visualization, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.