FAIR is a forecasting tool developed to support decision making in a retail environment. It provides multi-step-ahead sales forecasts at the category-store level, which are based on an interpretable and transparent model. These aspects make it an objective tool with which different promotional strategies (scenarios) can be compared and insights can be generated. Additionally, the FAIRforecast package provides plotting functions that generate figures showing the relative strength of the interactions between categories and important promotional variables.
The main function of the package, FAIR_train, fits a FAIR model to the data as detailed by Gür Ali and Gürlek (2020). The model is fitted in three stages. First, FAIR orthogonalizes the sales and marketing data with respect to seasonality by fitting category-store-specific linear models. This stage helps FAIR to guard against regularization induced bias (Hahn et al., 2018). Next, the deseasonalized sales is explained by the deseasonalized marketing with Elastic net models (Zou & Hastie, 2005). This stage employs regularization and pools data to the category level1 to be able to deal with high dimensionality. It can account for the cross-category effects within a store. The last stage extrapolates the residuals to incorporate the random disturbances that cannot be explained with the seasonality or marketing activity.
The FAIRforecast package is accompanied by a simulated sample dataset to showcase its abilities. The data contains the weekly category-store-level sales of a retail chain. It provides the promotional activity variables as well as the seasonality and calendar variables. The following is the list of variables.
week Identifier for the week.store_n Identifier for the store.category Product category.dollar_sales Total sales in dollars.price Average prices of SKUs within the category.display Average of a binary variable that indicates whether the SKU was displayed.discount Average percentage discount in the category.ad Average of a binary variable that indicates whether the SKU was advertised.thanksgiving, christmas and other_holidays Whether week covers the holiday.season Season of the year.first and fifteenth Whether the week covers the first or fifteenth day of the month, respectively.month Month of the year.The FAIRforecast package trains the forecasting model with its core function, FAIR_train. It requires the training data, the names of the different components of the data as strings, and the options. Type ?FAIR_train to see the details of the arguments. Note that marketing argument contains all the marketing columns in the data whereas cc_marketing contains only the ones that are assumed to have cross-category effects. The example below estimates within-category effects for “price”, “display”, “discount”, and “ad”; and also cross-category effects for “price”, “display”, and “discount”.
library(FAIRforecast)
data_ret <- sample_data()
train_data <- data_ret[data_ret$week < 184, ]
my_model <- FAIR_train(train_data = train_data,
store = "store_n",
category = "category",
time_id = "week",
seasonality = c("thanksgiving", "christmas",
"other_holidays", "season", "first",
"fifteenth", "month"),
marketing = c("price", "display", "discount", "ad"),
cc_marketing = c("price", "display", "discount"),
sales = "dollar_sales",
lag = 2,
alpha = c(0.1, 0.5, 0.9),
horizon = 13)
names(my_model)
#> [1] "fit" "models" "pars" "lag_data"It returns an R list with four elements:
fit A data.frame of the sales estimate and its components for each store-category-time_id combination in train_data.models The model for each stage and a data.frame for in-sample bias.pars Parameters used for the training.lag_data The data.frame of the last periods, used by FAIR_predict.Below is the head of fit data.frame. Base_Sales column shows the forecasted sales as a result of the “typical” marketing activity for the given seasonality. Marketing_deviation_multiplier is the multiplier that modifies the forecast by accounting the marketing scenario deviation from the established pattern. Similarly, Disturbance_multiplier is the multiplier that incorporates the random disturbances which cannot be captured by the seasonality or marketing variables. Sales_estimate column shows the final prediction calculated by multiplying the all2. The first lag estimates -in this case, 2- do not have the multipliers since Stage 2 uses the lagged marketing information.
head(my_model$fit)
#> store_n category week Base_Sales Marketing_deviation_multiplier
#> 1 1 beer 1 10202.560 NA
#> 2 1 beer 2 8212.279 NA
#> 3 1 beer 3 8008.027 1.003037
#> 4 1 beer 4 8241.773 1.007518
#> 5 1 beer 5 8111.515 1.005672
#> 6 1 beer 6 10011.854 1.001056
#> Disturbance_multiplier Sales_estimate
#> 1 NA 10201.560
#> 2 NA 8211.279
#> 3 0.9917328 7964.943
#> 4 0.9794253 8131.885
#> 5 1.1251614 9177.527
#> 6 1.0754403 10777.526The second element in the output of FAIR_train is a list of the models. The first in the list is store-category-specific Stage 1 models for each marketing variable and sales. Note that these are multiplicative OLS models and the dependent variable is the logarithmic transformation of the respective sales or marketing variable. Here, for example, is the OLS model that explains the seasonality in the discounts for the beer category in Store 1 on the logarithmic scale.
my_model$models$Stage1$`1.beer`$discount
#>
#> Call:
#> stats::lm(formula = myformula, data = train, model = F)
#>
#> Coefficients:
#> (Intercept) thanksgiving christmas other_holidays seasonspring
#> 3.851e-02 1.838e-04 8.570e-05 6.866e-05 1.756e-04
#> seasonsummer seasonwinter first fifteenth month2
#> 3.919e-04 5.747e-05 1.031e-04 6.162e-05 -2.100e-05
#> month3 month4 month5 month6 month7
#> -2.751e-04 -3.665e-04 -5.448e-04 -3.271e-04 -6.835e-04
#> month8 month9 month10 month11 month12
#> -4.856e-04 -1.269e-04 -2.302e-04 -1.435e-04 -3.448e-04
#> week
#> -1.167e-07The next in the list is category-specific Stage 2 Elastic net models. One can obtain the Elastic net coefficients for the beer category with the following code. Note that we have lags of the marketing variables and price, display, and discount of the other categories.
coef(my_model$models$Stage2$beer)
#> 40 x 1 sparse Matrix of class "dgCMatrix"
#> s0
#> (Intercept) -0.001015938
#> price .
#> display -0.122178079
#> discount .
#> ad .
#> price_lag1 .
#> price_lag2 .
#> display_lag1 .
#> display_lag2 .
#> discount_lag1 .
#> discount_lag2 .
#> ad_lag1 -0.032824503
#> ad_lag2 0.014401934
#> blades_price .
#> carbbev_price 0.043591816
#> cigets_price .
#> coffee_price -0.093470777
#> coldcer_price -0.111860909
#> deod_price 0.050689467
#> diapers_price .
#> factiss_price -0.046079257
#> fzdinent_price .
#> blades_display .
#> carbbev_display -0.413518772
#> cigets_display .
#> coffee_display -0.278604229
#> coldcer_display -1.194173778
#> deod_display -5.633566483
#> diapers_display .
#> factiss_display .
#> fzdinent_display .
#> blades_discount 3.159688406
#> carbbev_discount .
#> cigets_discount .
#> coffee_discount 1.152174725
#> coldcer_discount .
#> deod_discount .
#> diapers_discount .
#> factiss_discount .
#> fzdinent_discount 0.553481583Similarly, the Stage3 object in the models list is the output obtained from forecast::stlm() function in the 3rd Stage of FAIR.
Having fitted our FAIR model, we can predict the sales for the following weeks. To do that, we use FAIR_predict function. It requires the object we obtained from FAIR_train and the new data. The new data should have the store and category identifiers, time id, seasonality and calendar variables, and marketing activity columns. The names should match the ones in train_data argument from the training. time_id column should contain weeks after the last week of train_data and cannot be out of the horizon specified for FAIR_train.
test_data <- data_ret[data_ret$week >= 184,
c("store_n", "category", "week",
c("thanksgiving", "christmas", "other_holidays",
"season", "first", "fifteenth", "month"),
c("price", "display", "discount", "ad"))]
predictions <- FAIR_predict(my_model, test_data)
head(predictions)
#> store_n category week Base_Sales Marketing_deviation_multiplier
#> 1 1 beer 184 12892.75 0.5670866
#> 2 1 beer 185 12572.09 0.5674181
#> 3 1 beer 186 12939.05 0.5670711
#> 4 1 beer 187 12962.27 0.5670819
#> 5 1 beer 188 12757.40 0.5674915
#> 6 1 beer 189 11862.13 0.5700347
#> Disturbance_multiplier Forecast
#> 1 1.2172934 8899.003
#> 2 0.9292128 6627.660
#> 3 0.9464542 6943.478
#> 4 1.0344155 7602.643
#> 5 1.0012123 7247.494
#> 6 1.1072632 7486.122Output of FAIR_predict function is a data.frame with forecasts which has the same structure as fit object in the output list of FAIR_train.
Beyond being a forecasting tool, FAIR provides tools to understand the mechanism that governs the sales. First, the user can input a set of scenarios to FAIR_predict function to guide a decision. FAIR uses modern machine learning techniques to make sure coefficient estimates are not biased. Therefore, different marketing strategies can be compared on the basis of the sales predicted by FAIR. Additionally, the user can gain insights with the plotting functions variable_importance and cross_category.
variable_importance plots the average absolute coefficients of the marketing variables scaled by the maximum effect. The most important variable has the value of 100 and the others have a value proportional to their average absolute coefficient. The default plot shows the top 10 variables and includes only the variables which belong to the focal category, i.e., the cross-category variables are excluded. The user can choose the categories over which the averages are calculated. All the categories are included by default.
variable_importance(object = my_model, n_vars = 10, categories = NULL,
only_own = T)cross_category plots the matrix of the interaction strength. It averages the absolute value of the cross-category marketing variables of the x-axis category in the model for the y-axis category.
cross_category(my_model)Gür Ali, Ö. and Gürlek, R. (2020) Automatic Interpretable Retail Forecasting (FAIR) with Promotional Scenarios. International Journal of Forecasting, forthcoming.
Hahn, P. R., Carvalho, C. M., Puelz, D., and He, J. (2018). Regularization and confounding in linear regression for treatment effect estimation. Bayesian Analysis, 13(1):163– 182.
Zou, H. and Hastie, T. (2005), Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67: 301-320.