Read on Twitter
Excited about some new @ForecastForge features. Need to smooth out the rough edges but should be ready to launch next week.
Look at this chart of daily transactions. The last 30 days is from after they made a small site change.
The spikes are on the days emails are sent
Look at this chart of daily transactions. The last 30 days is from after they made a small site change.
The spikes are on the days emails are sent
I can use CausalImpact in R to see if the site change made a difference to the total number of transactions.
CausalImpact doesn't handle this too well. I've added a 7 day seasonality here but the email spikes are causing problems; look how wide the interval is in the top plot
CausalImpact doesn't handle this too well. I've added a 7 day seasonality here but the email spikes are causing problems; look how wide the interval is in the top plot
Adding the dates on which emails were sent helps narrow the predictive interval a lot; to the extent that it reports a 98% chance of a causal effect.
You will be able to do a similar analysis in @ForecastForge
You will be able to do a similar analysis in @ForecastForge
Start by preparing the data in Google Sheets.
1. The Date
2. Number of transactions
3. Whether or not an email was sent that day
4. Cumulative sum of transactions during the test period
The number in red is the total transactions during the final month
1. The Date
2. Number of transactions
3. Whether or not an email was sent that day
4. Cumulative sum of transactions during the test period
The number in red is the total transactions during the final month
Configure your forecast to also include a forecast for the cumulative running total
If you make a forecast without using the email dates as a regressor then you'll see the actual value is between the lower and upper bound here; in this case you'd normally say it was an inconclusive or negative test
But if you include the email dates as a helper column then look how much narrower the predictive interval is!
In this case you can conclude the intervention had an impact (95% probability in this case)
In this case you can conclude the intervention had an impact (95% probability in this case)
