google / google/meridian

"ValueError: `times` coords already set" when `n_media_times = n_times + max_lag`

Open
#1,502 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.5k
Forks
294
Avg merge
1d 21h
Merged PRs (30d)
47

Description

Hi!

I am trying to incorporate media "burn-in" effects by having `n_media_times = n_times + max_lag`, as described in the [API reference](https://developers.google.com/meridian/reference/api/meridian/data/input_data/InputData) (media attribute). I suspect that there is an issue in the code though as the [DataFrameInputDataBuilder.with_media](https://developers.google.com/meridian/reference/api/meridian/data/data_frame_input_data_builder/DataFrameInputDataBuilder#with_media) method sets `time_col`. In contrast, the [DataFrameInputDataBuilder.with_organic_media](https://developers.google.com/meridian/reference/api/meridian/data/data_frame_input_data_builder/DataFrameInputDataBuilder#with_organic_media) sets `media_time_col`.

Thus when I have my input dataframe that has `max_lag` more rows in the beginning for my media data than for my kpi I get the following error:

---------------------------------------------------------------------------

ValueError Traceback (most recent call last)
Cell In[107], line 13
1 data_loader = DataFrameDataLoader(
2 df=df,
3 coord_to_columns=coord_to_columns,
(...)
11 #organic_frequency_to_channel=organic_frequency_to_channel,
12 )
---> 13 data = data_loader.load()

File /opt/conda/lib/python3.10/site-packages/meridian/data/load.py:759, in DataFrameDataLoader.load(self)
754 # So now we can use one of the channel mapper dicts to get the canonical
755 # channel names for each column.
756 media_channel_names = [
757 self.media_to_channel[c] for c in self.coord_to_columns.media
758 ]
--> 759 builder.with_media(
760 self.df,
761 media_execution_columns,
762 media_spend_columns,
763 media_channel_names,
764 self.coord_to_columns.time,
765 self.coord_to_columns.geo,
766 )
768 if (
769 self.reach_to_channel is not None
770 and self.frequency_to_channel is not None
(...)
774 # listed in `reach`, `frequency`, and `rf_spend` are already validated
775 # to correspond to the same channels, in user-given order.
776 reach_columns = list(self.coord_to_columns.reach)

File /opt/conda/lib/python3.10/site-packages/meridian/data/data_frame_input_data_builder.py:381, in DataFrameInputDataBuilder.with_media(self, df, media_cols, media_spend_cols, media_channels, time_col, geo_col)
379 media_spend_data = media_df.set_index([geo_col, time_col])[media_spend_cols]
380 media_spend_data.columns = media_channels
--> 381 self.media_spend = (
382 media_spend_data.stack()
383 .rename(constants.MEDIA_SPEND)
384 .rename_axis([
385 constants.GEO,
386 constants.TIME,
387 constants.MEDIA_CHANNEL,
388 ])
389 .to_xarray()
390 )
391 return self

File /opt/conda/lib/python3.10/site-packages/meridian/data/input_data_builder.py:338, in InputDataBuilder.media_spend(self, media_spend)
336 self._media_spend = self._normalize_coords(media_spend, constants.TIME)
337 self.geos = self.media_spend.coords[constants.GEO].values.tolist()
--> 338 self.time_coords = self.media_spend.coords[constants.TIME].values.tolist()

File /opt/conda/lib/python3.10/site-packages/meridian/data/input_data_builder.py:85, in InputDataBuilder.time_coords(self, value)
83 raise ValueError('`times` coords must be unique.')
84 if self.time_coords is not None and set(self.time_coords) != set(value):
---> 85 raise ValueError(f'`times` coords already set to {self.time_coords}.')
86 if self.media_time_coords is not None and not set(value).issubset(
87 self.media_time_coords
88 ):
89 raise ValueError(
90 '`times` coords must be subset of previously set `media_times`'
91 ' coords.'
92 )

ValueError: `times` coords already set to ['2022-07-21', '2022-07-22', ...

It seems that there is a mismatch in the time value, which stems from the fact that my media data has more rows than my remaining data.

Contributor guide

Open the contributing guide

Research direction

Start at DataFrameDataLoader.load in meridian/data/load.py, then inspect with_media in meridian/data/data_frame_input_data_builder.py and the time_coords setter in meridian/data/input_data_builder.py. Compare the media and KPI time-coordinate handling with with_organic_media. Done means media data can include the documented additional burn-in times without triggering the existing coordinate mismatch error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.