The metrics module in recpack contains a large amount of metrics commonly used to evaluate recommendation algorithms.

All metrics assume that we have access to a set y_true of true user interactions that we are trying to predict and a set of recommendation scores y_pred. We can then evaluate how well our algorithm was able to predict these interactions in y_true.

Most metrics are “Top-K Metrics”: they consider only the Top-K best scoring item predictions, as the number of recommendations that can be shown in a realistic setting is limited.

Global Metrics

A global metric reports only a single, global metric value.


Fraction of all items that are ranked among the Top-K recommendations for any user.


Expected Percentile Ranking.

Listwise Metrics

A listwise metric reports one metric value for every user. To obtain a global metric value, these per-user scores are averaged.


Computes the sum of gains of all items in a recommendation list.


Computes the normalized sum of gains of all items in a recommendation list.


Computes the fraction of true interactions that made it into the Top-K recommendations.


Computes number of Top-K recommendations that were hits divided by the minimum of K and number of true interactions of the user.


Computes the fraction of top-K recommendations that correspond to true interactions.


Computes the inverse of the rank of the first hit in the recommendation list.

Elementwise Metric

An elementwise metric reports a score for every user-item pair in the Top-K. To obtain a global metric value, these scores are summed per user, then averaged.


Computes the number of hits in a list of Top-K recommendations.


Computes the discounted gain of every item in the Top-K recommendations of a user.