Folks who work on predictive models as part of your product/team roadmap: How do choose your (quantitative) goals for model improvement?

- Do you goal on absolute or relative performance?
- Relative to your baseline or to previous model?
- How do you choose the goal threshold?

If you& #39;re kind enough to answer, I& #39;d also be interested to know if the models you& #39;re setting goals for perform a classification, regression, forecasting, or some other learning task.

And to put this in somewhat plainer language, it often comes up during planning that people will say:

We need to set a specific goal for this model for Q3, and then ask, "How much of a better model should be the goal, and in what particular way do we define & #39;better& #39;?"

Getting the feeling this is going to be one of those problems where none of us really knows what we& #39;re doing here and we& #39;re all just making it up as we go along.