Exciting day today, since our MSc student Bart is defending his thesis "The Error that is the Error Message"! Follow along here for the highlights of his work.

One of the big contributions of Bart's thesis is the EMCF: Error Message Component Framework. If we want to analyze how students understand error messages, we need to have good names for the things that error messages consist of.

Error messages, according to the framework, consist of Context and Feedback:

Both of those main components can be split up further:

The framework also supports weights for the different categories, to gain an understanding of how much feedback is given in different components. Weights are now defined as the number of words per component.

Using the EMCF, we can, for example, compare compiler error messages with runtime error messages.

To gain a deeper understanding of what novices expect from an error message, Bart asked students to create their own error messages, and compared them using his model

Many terms used in error messages are very unclear to novice programmers, 10 out of 11 students reported "EOL" and "literal" as unknown words.

This one is the absolute worst, says Bart. Just missing a quotation mark leads to this incomprehensible gibberish (in the eyes of a novice)

In case you were wondering... Bart passed his defense with a grade of 7.5/10!