You are correct. In the context of personalization, the challenge is an old 5-star model world, was to predict how much you would like a movie. If our prediction for two different movies was 4 and 5 stars, and you rated them 4 and 5 stars, RMSE was zero. If we were off in either direction, RMSE was a positive number.

The Netflix Prize metric for deciding whether teams improved our collaborative filtering was RMSE and you can read more about it by clicking here — make sure you click on the graphic of the Netflix Prize website where you can see the published RMSE for each team, and the leader has the smallest RMSE.

Gib