Baycomp

Janez Demsar has reimplemented our library about Bayesian hypothesis testing for comparing competing algorithms in ML. It can now be installed directly with pip. Hereafter, a brief description. Baycomp is a library for Bayesian comparison of classifiers. Functions compare two classifiers on one or on multiple data sets. They compute …

Continue reading

Bayes+Hilbert=QM

QM is based on four main axioms, which were derived after a long process of trial and error. The motivations for the axioms are not always clear and even to experts the basic axioms of QM often appear counter-intuitive. In a recent paper [1], we have shown that: It is …

Continue reading

The importance of the region of practical equivalence (ROPE)

The difference between two classifiers (algorithms) can be very small; however there are no two classifiers whose accuracies are perfectly equivalent. By using an null hypothesis significance test (NHST), the null hypothesis is that the classifiers are equal. However, the null hypothesis is practically always false! By rejecting the null …

Continue reading

Bayesian Signed-Rank Test for comparing algorithms in machine learning

This post is about Bayesian nonparametric tests for comparing algorithms in ML. This time we will discuss about Python module signrank in bayesiantests (see our GitHub repository). It computes the Bayesian equivalent of the Wilcoxon signed-rank test. It return probabilities that, based on the measured performance, one model is better …

Continue reading