2021 |
Benavoli, Alessio; Wyse, Jason; White, Arthur Gaussian Processes to speed up MCMC with automatic exploratory-exploitation effect Inproceedings ProbProg21, 2021. Abstract | Links | BibTeX | Tags: @inproceedings{Benavoli2021b, title = {Gaussian Processes to speed up MCMC with automatic exploratory-exploitation effect}, author = {Alessio Benavoli and Jason Wyse and Arthur White}, url = {https://arxiv.org/abs/2109.13891}, year = {2021}, date = {2021-10-20}, booktitle = {ProbProg21}, crossref = {ProbProg21}, abstract = {We present a two-stage Metropolis-Hastings algorithm for sampling probabilistic models, whose log-likelihood is computationally expensive to evaluate, by using a surrogate Gaussian Process (GP) model. The key feature of the approach, and the difference w.r.t. previous works, is the ability to learn the target distribution from scratch (while sampling), and so without the need of pre-training the GP. This is fundamental for automatic and inference in Probabilistic Programming Languages In particular, we present an alternative first stage acceptance scheme by marginalising out the GP distributed function, which makes the acceptance ratio explicitly dependent on the variance of the GP. This approach is extended to Metropolis-Adjusted Langevin algorithm (MALA).}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } We present a two-stage Metropolis-Hastings algorithm for sampling probabilistic models, whose log-likelihood is computationally expensive to evaluate, by using a surrogate Gaussian Process (GP) model. The key feature of the approach, and the difference w.r.t. previous works, is the ability to learn the target distribution from scratch (while sampling), and so without the need of pre-training the GP. This is fundamental for automatic and inference in Probabilistic Programming Languages In particular, we present an alternative first stage acceptance scheme by marginalising out the GP distributed function, which makes the acceptance ratio explicitly dependent on the variance of the GP. This approach is extended to Metropolis-Adjusted Langevin algorithm (MALA). |
Benavoli, Alessio ; Azzimonti, Dario ; Piga, Dario Choice functions based multi-objective Bayesian optimisation Technical Report 2021. Abstract | Links | BibTeX | Tags: bayesian nonparametric, bayesian optimisation, Gaussian Process @techreport{Benavoli2021bb, title = {Choice functions based multi-objective Bayesian optimisation}, author = {Benavoli , Alessio and Azzimonti, Dario and Piga, Dario }, url = {https://arxiv.org/pdf/2110.08217.pdf}, year = {2021}, date = {2021-10-18}, abstract = {In this work we introduce a new framework for multi-objective Bayesian optimisation where the multi-objective functions can only be accessed via choice judgements, such as “I pick options x1, x2, x3 among this set of five options x1, x2, . . . , x5”. The fact that the option x4 is rejected means that there is at least one option among the selected ones x1, x2, x3 that I strictly prefer over x4 (but I do not have to specify which one). We assume that there is a latent vector function f for some dimension ne which embeds the options into the real vector space of dimension ne, so that the choice set can be represented through a Pareto set of non-dominated options. By placing a Gaussian process prior on f and deriving a novel likelihood model for choice data, we propose a Bayesian framework for choice functions learning. We then apply this surrogate model to solve a novel multi-objective Bayesian optimisation from choice data problem.}, keywords = {bayesian nonparametric, bayesian optimisation, Gaussian Process}, pubstate = {published}, tppubtype = {techreport} } In this work we introduce a new framework for multi-objective Bayesian optimisation where the multi-objective functions can only be accessed via choice judgements, such as “I pick options x1, x2, x3 among this set of five options x1, x2, . . . , x5”. The fact that the option x4 is rejected means that there is at least one option among the selected ones x1, x2, x3 that I strictly prefer over x4 (but I do not have to specify which one). We assume that there is a latent vector function f for some dimension ne which embeds the options into the real vector space of dimension ne, so that the choice set can be represented through a Pareto set of non-dominated options. By placing a Gaussian process prior on f and deriving a novel likelihood model for choice data, we propose a Bayesian framework for choice functions learning. We then apply this surrogate model to solve a novel multi-objective Bayesian optimisation from choice data problem. |
Benavoli, Alessio; de Campos, Cassio Bayesian Kernelised Test of (In)dependence with Mixed-type Variables Conference The 8th IEEE International Conference on Data Science and Advanced Analytics (DSAA). Porto, Portugal, 2021. Abstract | Links | BibTeX | Tags: bayesian nonparametric @conference{benavoli2021bayesian, title = {Bayesian Kernelised Test of (In)dependence with Mixed-type Variables}, author = {Alessio Benavoli and Cassio de Campos}, url = {https://arxiv.org/abs/2105.04001}, year = {2021}, date = {2021-10-04}, booktitle = {The 8th IEEE International Conference on Data Science and Advanced Analytics (DSAA). Porto, Portugal}, abstract = {A fundamental task in AI is to assess (in)dependence between mixed-type variables (text, image, sound). We propose a Bayesian kernelised correlation test of (in)dependence using a Dirichlet process model. The new measure of (in)dependence allows us to answer some fundamental questions: Based on data, are (mixed-type) variables independent? How likely is dependence/independence to hold? How high is the probability that two mixed-type variables are more than just weakly dependent? We theoretically show the properties of the approach, as well as algorithms for fast computation with it. We empirically demonstrate the effectiveness of the proposed method by analysing its performance and by comparing it with other frequentist and Bayesian approaches on a range of datasets and tasks with mixed-type variables. }, keywords = {bayesian nonparametric}, pubstate = {published}, tppubtype = {conference} } A fundamental task in AI is to assess (in)dependence between mixed-type variables (text, image, sound). We propose a Bayesian kernelised correlation test of (in)dependence using a Dirichlet process model. The new measure of (in)dependence allows us to answer some fundamental questions: Based on data, are (mixed-type) variables independent? How likely is dependence/independence to hold? How high is the probability that two mixed-type variables are more than just weakly dependent? We theoretically show the properties of the approach, as well as algorithms for fast computation with it. We empirically demonstrate the effectiveness of the proposed method by analysing its performance and by comparing it with other frequentist and Bayesian approaches on a range of datasets and tasks with mixed-type variables. |
Benavoli, Alessio ; Facchini, Alessandro ; Zaffalon, Marco The Weirdness Theorem and the Origin of Quantum Paradoxes Journal Article Foundations of Physics, 51 (95), 2021. Abstract | Links | BibTeX | Tags: Quantum mechanics @article{Benavoli2021f, title = {The Weirdness Theorem and the Origin of Quantum Paradoxes}, author = {Benavoli , Alessio and Facchini, Alessandro and Zaffalon, Marco}, url = {https://link.springer.com/article/10.1007/s10701-021-00499-w}, doi = {10.1007/s10701-021-00499-w}, year = {2021}, date = {2021-09-28}, journal = {Foundations of Physics}, volume = {51}, number = {95}, abstract = {We argue that there is a simple, unique, reason for all quantum paradoxes, and that such a reason is not uniquely related to quantum theory. It is rather a mathematical question that arises at the intersection of logic, probability, and computation. We give our ‘weirdness theorem’ that characterises the conditions under which the weirdness will show up. It shows that whenever logic has bounds due to the algorithmic nature of its tasks, then weirdness arises in the special form of negative probabilities or non-classical evaluation functionals. Weirdness is not logical inconsistency, however. It is only the expression of the clash between an unbounded and a bounded view of computation in logic. We discuss the implication of these results for quantum mechanics, arguing in particular that its interpretation should ultimately be computational rather than exclusively physical. We develop in addition a probabilistic theory in the real numbers that exhibits the phenomenon of entanglement, thus concretely showing that the latter is not specific to quantum mechanics.}, keywords = {Quantum mechanics}, pubstate = {published}, tppubtype = {article} } We argue that there is a simple, unique, reason for all quantum paradoxes, and that such a reason is not uniquely related to quantum theory. It is rather a mathematical question that arises at the intersection of logic, probability, and computation. We give our ‘weirdness theorem’ that characterises the conditions under which the weirdness will show up. It shows that whenever logic has bounds due to the algorithmic nature of its tasks, then weirdness arises in the special form of negative probabilities or non-classical evaluation functionals. Weirdness is not logical inconsistency, however. It is only the expression of the clash between an unbounded and a bounded view of computation in logic. We discuss the implication of these results for quantum mechanics, arguing in particular that its interpretation should ultimately be computational rather than exclusively physical. We develop in addition a probabilistic theory in the real numbers that exhibits the phenomenon of entanglement, thus concretely showing that the latter is not specific to quantum mechanics. |
Benavoli, Alessio; Azzimonti, Dario; Piga, Dario A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes Journal Article Machine Learning, pp. 1-39, 2021. Abstract | Links | BibTeX | Tags: bayesian nonparametric, Gaussian Process @article{benavoli2021, title = {A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes}, author = {Alessio Benavoli and Dario Azzimonti and Dario Piga}, url = {https://link.springer.com/article/10.1007/s10994-021-06039-x}, doi = {10.1007/s10994-021-06039-x}, year = {2021}, date = {2021-09-13}, journal = {Machine Learning}, pages = {1-39}, abstract = {Skew-Gaussian processes (SkewGPs) extend the multivariate Unified Skew-Normal distributions over finite dimensional vectors to distribution over functions. SkewGPs are more general and flexible than Gaussian processes, as SkewGPs may also represent asymmetric distributions. In a recent contribution we showed that SkewGP and probit likelihood are conjugate, which allows us to compute the exact posterior for non-parametric binary classification and preference learning. In this paper, we generalize previous results and we prove that SkewGP is conjugate with both the normal and affine probit likelihood, and more in general, with their product. This allows us to (i) handle classification, preference, numeric and ordinal regression, and mixed problems in a unified framework; (ii) derive closed-form expression for the corresponding posterior distributions. We show empirically that the proposed framework based on SkewGP provides better performance than Gaussian processes in active learning and Bayesian (constrained) optimization}, keywords = {bayesian nonparametric, Gaussian Process}, pubstate = {published}, tppubtype = {article} } Skew-Gaussian processes (SkewGPs) extend the multivariate Unified Skew-Normal distributions over finite dimensional vectors to distribution over functions. SkewGPs are more general and flexible than Gaussian processes, as SkewGPs may also represent asymmetric distributions. In a recent contribution we showed that SkewGP and probit likelihood are conjugate, which allows us to compute the exact posterior for non-parametric binary classification and preference learning. In this paper, we generalize previous results and we prove that SkewGP is conjugate with both the normal and affine probit likelihood, and more in general, with their product. This allows us to (i) handle classification, preference, numeric and ordinal regression, and mixed problems in a unified framework; (ii) derive closed-form expression for the corresponding posterior distributions. We show empirically that the proposed framework based on SkewGP provides better performance than Gaussian processes in active learning and Bayesian (constrained) optimization |
Kania, Lucas ; Schürch, Manuel ; Azzimonti, Dario ; Benavoli, Alessio Sparse Information Filter for Fast Gaussian Process Regression Conference European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases ECML PKDD, 2021. Abstract | Links | BibTeX | Tags: bayesian nonparametric, Gaussian Process @conference{Kania2021, title = { Sparse Information Filter for Fast Gaussian Process Regression}, author = {Kania, Lucas and Schürch, Manuel and Azzimonti, Dario and Benavoli, Alessio}, url = {https://2021.ecmlpkdd.org/wp-content/uploads/2021/07/sub_854.pdf http://alessiobenavoli.com/wp-content/uploads/2021/07/sub_854.pdf }, year = {2021}, date = {2021-09-01}, booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases ECML PKDD}, abstract = {Gaussian processes (GPs) are an important tool in machine learning and applied mathematics with applications ranging from Bayesian optimization to calibration of computer experiments. They constitute a powerful kernelized non-parametric method with well-calibrated un- certainty estimates, however, off-the-shelf GP inference procedures are limited to datasets with a few thousand data points because of their cubic computational complexity. For this reason, many sparse GPs techniques were developed over the past years. In this paper, we focus on GP regression tasks and propose a new algorithm to train variational sparse GP models. An analytical posterior update expression based on the Information Filter is derived for the variational sparse GP model. We benchmark our method on several real datasets with millions of data points against the state-of-the-art Stochastic Variational GP (SVGP) and sparse orthogonal variational inference for Gaussian Processes (SOLVEGP). Our method achieves comparable performances to SVGP and SOLVEGP while providing considerable speed-ups. Specifically, it is consistently four times faster than SVGP and on average 2.5 times faster than SOLVEGP.}, keywords = {bayesian nonparametric, Gaussian Process}, pubstate = {published}, tppubtype = {conference} } Gaussian processes (GPs) are an important tool in machine learning and applied mathematics with applications ranging from Bayesian optimization to calibration of computer experiments. They constitute a powerful kernelized non-parametric method with well-calibrated un- certainty estimates, however, off-the-shelf GP inference procedures are limited to datasets with a few thousand data points because of their cubic computational complexity. For this reason, many sparse GPs techniques were developed over the past years. In this paper, we focus on GP regression tasks and propose a new algorithm to train variational sparse GP models. An analytical posterior update expression based on the Information Filter is derived for the variational sparse GP model. We benchmark our method on several real datasets with millions of data points against the state-of-the-art Stochastic Variational GP (SVGP) and sparse orthogonal variational inference for Gaussian Processes (SOLVEGP). Our method achieves comparable performances to SVGP and SOLVEGP while providing considerable speed-ups. Specifically, it is consistently four times faster than SVGP and on average 2.5 times faster than SOLVEGP. |
Benavoli, Alessio ; Corani, Giorgio State Space approximation of Gaussian Processes for time series forecasting Conference Workshop on Advanced Analytics and Learning on Temporal Data, ECML/PKDD 21 , 2021. Abstract | Links | BibTeX | Tags: bayesian nonparametric, Gaussian Process @conference{Benavoli2021e, title = {State Space approximation of Gaussian Processes for time series forecasting}, author = {Benavoli, Alessio and Corani, Giorgio}, url = {http://alessiobenavoli.com/wp-content/uploads/2021/07/TS_final.pdf https://project.inria.fr/aaltd21/}, year = {2021}, date = {2021-07-27}, booktitle = {Workshop on Advanced Analytics and Learning on Temporal Data, ECML/PKDD 21 }, abstract = {Gaussian Processes (GPs), with a complex enough additive kernel, provide competitive results in time series forecasting compared to state-of-the-art approaches (arima, ETS) provided that: (i) during training the unnecessary components of the kernel are made irrelevant by automatic relevance determination; (ii) priors are assigned to each hyperparameter. However, GPs computational complexity grows cubically in time and quadratically in memory with the number of observations. The state space (SS) approximation of GPs allows to compute GPs based inferences with linear complexity. In this paper, we apply the SS representation to time series forecasting showing that SS models provide a performance comparable with that of full GP and better than state-of-the-art models (arima, ETS). Moreover, the SS representation allows us to derive new models by, for instance, combining ETS with kernels.}, keywords = {bayesian nonparametric, Gaussian Process}, pubstate = {published}, tppubtype = {conference} } Gaussian Processes (GPs), with a complex enough additive kernel, provide competitive results in time series forecasting compared to state-of-the-art approaches (arima, ETS) provided that: (i) during training the unnecessary components of the kernel are made irrelevant by automatic relevance determination; (ii) priors are assigned to each hyperparameter. However, GPs computational complexity grows cubically in time and quadratically in memory with the number of observations. The state space (SS) approximation of GPs allows to compute GPs based inferences with linear complexity. In this paper, we apply the SS representation to time series forecasting showing that SS models provide a performance comparable with that of full GP and better than state-of-the-art models (arima, ETS). Moreover, the SS representation allows us to derive new models by, for instance, combining ETS with kernels. |
Casanova, Arianna ; Benavoli, Alessio ; Zaffalon, Marco Nonlinear Desirability as a Linear Classification Problem Inproceedings ISIPTA'21 Int. Symposium on Imprecise Probability: Theories and Applications, PJMLR, 2021. Abstract | Links | BibTeX | Tags: desirability, imprecise probability @inproceedings{Casanova2021, title = {Nonlinear Desirability as a Linear Classification Problem}, author = {Casanova, Arianna and Benavoli, Alessio and Zaffalon, Marco}, url = {http://alessiobenavoli.com/wp-content/uploads/2021/07/casanova21-1.pdf}, year = {2021}, date = {2021-07-07}, booktitle = {ISIPTA'21 Int. Symposium on Imprecise Probability: Theories and Applications, PJMLR}, abstract = {The present paper proposes a generalization of linearity axioms of coherence through a geometrical approach, which leads to an alternative interpretation of desirability as a classification problem. In particular,we analyze different sets of rationality axioms and,for each one of them, we show that proving that a subject, who provides finite accept and reject statements, respects these axioms, corresponds to solving a binary classification task using, each time, a different (usually nonlinear) family of classifiers. Moreover,by borrowing ideas from machine learning, we show the possibility to define a feature mapping allowing us to reformulate the above nonlinear classification problems as linear ones in a higher-dimensional space.This allows us to interpret gambles directly as payoffs vectors of monetary lotteries, as well as to reduce the task of proving the rationality of a subject to a linear classification task}, keywords = {desirability, imprecise probability}, pubstate = {published}, tppubtype = {inproceedings} } The present paper proposes a generalization of linearity axioms of coherence through a geometrical approach, which leads to an alternative interpretation of desirability as a classification problem. In particular,we analyze different sets of rationality axioms and,for each one of them, we show that proving that a subject, who provides finite accept and reject statements, respects these axioms, corresponds to solving a binary classification task using, each time, a different (usually nonlinear) family of classifiers. Moreover,by borrowing ideas from machine learning, we show the possibility to define a feature mapping allowing us to reformulate the above nonlinear classification problems as linear ones in a higher-dimensional space.This allows us to interpret gambles directly as payoffs vectors of monetary lotteries, as well as to reduce the task of proving the rationality of a subject to a linear classification task |
Corani, Giorgio; Benavoli, Alessio; Augusto, Joao; Zaffalon, Marco Time series forecasting with Gaussian Processes needs priors Inproceedings European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases ECML PKDD , 2021. Links | BibTeX | Tags: bayesian nonparametric, Gaussian Process, Gaussian processes @inproceedings{corani2020automatic, title = {Time series forecasting with Gaussian Processes needs priors}, author = {Giorgio Corani and Alessio Benavoli and Joao Augusto and Marco Zaffalon}, url = {https://arxiv.org/abs/2009.08102}, year = {2021}, date = {2021-06-01}, booktitle = { European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases ECML PKDD }, keywords = {bayesian nonparametric, Gaussian Process, Gaussian processes}, pubstate = {published}, tppubtype = {inproceedings} } |
Benavoli, Alessio; Facchini, Alessandro; Zaffalon, Marco Quantum indistinguishability through exchangeable desirable gambles Inproceedings ISIPTA'21 Int. Symposium on Imprecise Probability: Theories and Applications, PJMLR, 2021. Abstract | Links | BibTeX | Tags: desirability, Quantum mechanics @inproceedings{benavoli2021quantum, title = {Quantum indistinguishability through exchangeable desirable gambles}, author = {Alessio Benavoli and Alessandro Facchini and Marco Zaffalon}, url = {https://arxiv.org/abs/2105.04336}, year = {2021}, date = {2021-06-01}, booktitle = {ISIPTA'21 Int. Symposium on Imprecise Probability: Theories and Applications, PJMLR}, abstract = {Two particles are identical if all their intrinsic properties, such as spin and charge, are the same, meaning that no quantum experiment can distinguish them. In addition to the well known principles of quantum mechanics, understanding systems of identical particles requires a new postulate, the so called symmetrization postulate. In this work, we show that the postulate corresponds to exchangeability assessments for sets of observables (gambles) in a quantum experiment, when quantum mechanics is seen as a normative and algorithmic theory guiding an agent to assess her subjective beliefs represented as (coherent) sets of gambles. Finally, we show how sets of exchangeable observables (gambles) may be updated after a measurement and discuss the issue of defining entanglement for indistinguishable particle systems. }, keywords = {desirability, Quantum mechanics}, pubstate = {published}, tppubtype = {inproceedings} } Two particles are identical if all their intrinsic properties, such as spin and charge, are the same, meaning that no quantum experiment can distinguish them. In addition to the well known principles of quantum mechanics, understanding systems of identical particles requires a new postulate, the so called symmetrization postulate. In this work, we show that the postulate corresponds to exchangeability assessments for sets of observables (gambles) in a quantum experiment, when quantum mechanics is seen as a normative and algorithmic theory guiding an agent to assess her subjective beliefs represented as (coherent) sets of gambles. Finally, we show how sets of exchangeable observables (gambles) may be updated after a measurement and discuss the issue of defining entanglement for indistinguishable particle systems. |
Benavoli, Alessio; Balleri, Alessio; Farina, Alfonso Joint Waveform and Guidance Control Optimization by Statistical Linearisation for Target Rendezvous Conference IEEE Radar Conference 2021, May 8-14, Atlanta (USA), 2021. Abstract | Links | BibTeX | Tags: Cognitive radar, radar tracking @conference{Benavoli2021d, title = {Joint Waveform and Guidance Control Optimization by Statistical Linearisation for Target Rendezvous}, author = {Alessio Benavoli and Alessio Balleri and Alfonso Farina}, url = {http://alessiobenavoli.com/wp-content/uploads/2021/05/ieee_radarconf_21.pdf}, year = {2021}, date = {2021-05-18}, booktitle = { IEEE Radar Conference 2021, May 8-14, Atlanta (USA)}, abstract = {The algorithm proposed in this paper jointly selects the transmitted waveform and the control input so that a radar sensor on a moving platform can prosecute a target by minimising a predefined cost that accounts for the energy of the transmitted radar signal, the energy of the platform control input and the relative position error between the platform and the target. The cost is a function of the waveform design and control input. The algorithm extends the existing Joint Waveform Guidance and Control Optimisation (JWGCO) solution to non-linear equations to account for the dependency of the radar measurement accuracies on Signal to Noise Ratio (SNR) ratio and, as a consequence, the target position. The performance of the proposed solution based on statistical linearisation is assessed with a set of simulations for a pulsed Doppler radar transmitting linearly frequency modulated chirps.}, keywords = {Cognitive radar, radar tracking}, pubstate = {published}, tppubtype = {conference} } The algorithm proposed in this paper jointly selects the transmitted waveform and the control input so that a radar sensor on a moving platform can prosecute a target by minimising a predefined cost that accounts for the energy of the transmitted radar signal, the energy of the platform control input and the relative position error between the platform and the target. The cost is a function of the waveform design and control input. The algorithm extends the existing Joint Waveform Guidance and Control Optimisation (JWGCO) solution to non-linear equations to account for the dependency of the radar measurement accuracies on Signal to Noise Ratio (SNR) ratio and, as a consequence, the target position. The performance of the proposed solution based on statistical linearisation is assessed with a set of simulations for a pulsed Doppler radar transmitting linearly frequency modulated chirps. |
Benavoli, Alessio; Azzimonti, Dario; Piga, Dario Preferential Bayesian optimisation with Skew Gaussian Processes Inproceedings 2021 Genetic and Evolutionary Computation Conference Companion (GECCO '21 Companion), July 10--14, 2021, Lille, France , ACM, New York, NY, USA, 2021, ISBN: 978-1-4503-8351-6/21/07. Abstract | Links | BibTeX | Tags: Skew Gaussian Process; Bayesian Optimisation @inproceedings{benavoli2020preferential, title = {Preferential Bayesian optimisation with Skew Gaussian Processes}, author = {Alessio Benavoli and Dario Azzimonti and Dario Piga}, url = {https://arxiv.org/abs/2008.06677}, doi = {10.1145/3449726.3463128}, isbn = {978-1-4503-8351-6/21/07}, year = {2021}, date = {2021-05-01}, booktitle = {2021 Genetic and Evolutionary Computation Conference Companion (GECCO '21 Companion), July 10--14, 2021, Lille, France }, journal = {arXiv preprint arXiv:2008.06677}, publisher = {ACM}, address = {New York, NY, USA}, abstract = {Bayesian optimisation (BO) is a very effective approach for sequential black-box optimization where direct queries of the objective function are expensive. However, there are cases where the objective function can only be accessed via preference judgments, such as "this is better than that" between two candidate solutions (like in A/B tests or recommender systems). The state-of-the-art approach to Preferential Bayesian Optimization (PBO) uses a Gaussian process to model the preference function and a Bernoulli likelihood to model the observed pairwise comparisons. Laplace's method is then employed to compute posterior inferences and, in particular, to build an appropriate acquisition function. In this paper, we prove that the true posterior distribution of the preference function is a Skew Gaussian Process (SkewGP), with highly skewed pairwise marginals and, thus, show that Laplace's method usually provides a very poor approximation. We then derive an efficient method to compute the exact SkewGP posterior and use it as surrogate model for PBO employing standard acquisition functions (Upper Credible Bound, etc.). We illustrate the benefits of our exact PBO-SkewGP in a variety of experiments, by showing that it consistently outperforms PBO based on Laplace's approximation both in terms of convergence speed and computational time. We also show that our framework can be extended to deal with mixed preferential-categorical BO, typical for instance in smart manufacturing, where binary judgments (valid or non-valid) together with preference judgments are available. }, keywords = {Skew Gaussian Process; Bayesian Optimisation}, pubstate = {published}, tppubtype = {inproceedings} } Bayesian optimisation (BO) is a very effective approach for sequential black-box optimization where direct queries of the objective function are expensive. However, there are cases where the objective function can only be accessed via preference judgments, such as "this is better than that" between two candidate solutions (like in A/B tests or recommender systems). The state-of-the-art approach to Preferential Bayesian Optimization (PBO) uses a Gaussian process to model the preference function and a Bernoulli likelihood to model the observed pairwise comparisons. Laplace's method is then employed to compute posterior inferences and, in particular, to build an appropriate acquisition function. In this paper, we prove that the true posterior distribution of the preference function is a Skew Gaussian Process (SkewGP), with highly skewed pairwise marginals and, thus, show that Laplace's method usually provides a very poor approximation. We then derive an efficient method to compute the exact SkewGP posterior and use it as surrogate model for PBO employing standard acquisition functions (Upper Credible Bound, etc.). We illustrate the benefits of our exact PBO-SkewGP in a variety of experiments, by showing that it consistently outperforms PBO based on Laplace's approximation both in terms of convergence speed and computational time. We also show that our framework can be extended to deal with mixed preferential-categorical BO, typical for instance in smart manufacturing, where binary judgments (valid or non-valid) together with preference judgments are available. |
2020 |
Benavoli, Alessio; Azzimonti, Dario; Piga, Dario Skew Gaussian Processes for Classification Journal Article Machine Learning, 109 , pp. 1877–1902, 2020. Abstract | Links | BibTeX | Tags: bayesian nonparametric, Gaussian Process @article{benavoli2020skew, title = {Skew Gaussian Processes for Classification}, author = {Alessio Benavoli and Dario Azzimonti and Dario Piga}, url = {https://arxiv.org/abs/2005.12987}, doi = {10.1007/s10994-020-05906-3}, year = {2020}, date = {2020-09-04}, journal = {Machine Learning}, volume = {109}, pages = {1877–1902}, abstract = {Gaussian processes (GPs) are distributions over functions, which provide a Bayesian nonparametric approach to regression and classification. In spite of their success, GPs have limited use in some applications, for example, in some cases a symmetric distribution with respect to its mean is an unreasonable model. This implies, for instance, that the mean and the median coincide, while the mean and median in an asymmetric (skewed) distribution can be different numbers. In this paper, we propose Skew-Gaussian processes (SkewGPs) as a non-parametric prior over functions. A SkewGP extends the multivariate Unified Skew-Normal distribution over finite dimensional vectors to a stochastic processes. The SkewGP class of distributions includes GPs and, therefore, SkewGPs inherit all good properties of GPs and increase their flexibility by allowing asymmetry in the probabilistic model. By exploiting the fact that SkewGP and probit likelihood are conjugate model, we derive closed form expressions for the marginal likelihood and predictive distribution of this new nonparametric classifier. We verify empirically that the proposed SkewGP classifier provides a better performance than a GP classifier based on either Laplace's method or Expectation Propagation. }, keywords = {bayesian nonparametric, Gaussian Process}, pubstate = {published}, tppubtype = {article} } Gaussian processes (GPs) are distributions over functions, which provide a Bayesian nonparametric approach to regression and classification. In spite of their success, GPs have limited use in some applications, for example, in some cases a symmetric distribution with respect to its mean is an unreasonable model. This implies, for instance, that the mean and the median coincide, while the mean and median in an asymmetric (skewed) distribution can be different numbers. In this paper, we propose Skew-Gaussian processes (SkewGPs) as a non-parametric prior over functions. A SkewGP extends the multivariate Unified Skew-Normal distribution over finite dimensional vectors to a stochastic processes. The SkewGP class of distributions includes GPs and, therefore, SkewGPs inherit all good properties of GPs and increase their flexibility by allowing asymmetry in the probabilistic model. By exploiting the fact that SkewGP and probit likelihood are conjugate model, we derive closed form expressions for the marginal likelihood and predictive distribution of this new nonparametric classifier. We verify empirically that the proposed SkewGP classifier provides a better performance than a GP classifier based on either Laplace's method or Expectation Propagation. |
Ristic, Branko; Gilliam, Christopher; Byrne, Marion; Benavoli, Alessio A tutorial on uncertainty modeling for machine reasoning Journal Article Information Fusion, 55 , pp. 30 - 44, 2020, ISSN: 1566-2535. Abstract | Links | BibTeX | Tags: imprecise probability @article{RISTIC2019, title = {A tutorial on uncertainty modeling for machine reasoning}, author = {Branko Ristic and Christopher Gilliam and Marion Byrne and Alessio Benavoli}, url = {http://alessiobenavoli.com/wp-content/uploads/2019/08/Uncertainty_tutorial.pdf}, doi = {https://doi.org/10.1016/j.inffus.2019.08.001}, issn = {1566-2535}, year = {2020}, date = {2020-01-01}, journal = {Information Fusion}, volume = {55}, pages = {30 - 44}, abstract = {Increasingly we rely on machine intelligence for reasoning and decision making under uncertainty. This tutorial reviews the prevalent methods for model-based autonomous decision making based on observations and prior knowledge, primarily in the context of classification. Both observations and the knowledge-base available for reasoning are treated as being uncertain. Accordingly, the central themes of this tutorial are quantitative modeling of uncertainty, the rules required to combine such uncertain information, and the task of decision making under uncertainty. The paper covers the main approaches to uncertain knowledge representation and reasoning, in particular, Bayesian probability theory, possibility theory, reasoning based on belief functions and finally imprecise probability theory. The main feature of the tutorial is that it illustrates various approaches with several testing scenarios, and provides MATLAB solutions for them as a supplementary material for an interested reader.}, keywords = {imprecise probability}, pubstate = {published}, tppubtype = {article} } Increasingly we rely on machine intelligence for reasoning and decision making under uncertainty. This tutorial reviews the prevalent methods for model-based autonomous decision making based on observations and prior knowledge, primarily in the context of classification. Both observations and the knowledge-base available for reasoning are treated as being uncertain. Accordingly, the central themes of this tutorial are quantitative modeling of uncertainty, the rules required to combine such uncertain information, and the task of decision making under uncertainty. The paper covers the main approaches to uncertain knowledge representation and reasoning, in particular, Bayesian probability theory, possibility theory, reasoning based on belief functions and finally imprecise probability theory. The main feature of the tutorial is that it illustrates various approaches with several testing scenarios, and provides MATLAB solutions for them as a supplementary material for an interested reader. |
Piga, Dario; Bemporad, Alberto; Benavoli, Alessio Rao-Blackwellized sampling for batch and recursive Bayesian inference of Piecewise Affine models Journal Article Automatica, 117 , 2020, ISSN: 0005-1098. Abstract | Links | BibTeX | Tags: Bayesian inference, Markov Chain Monte Carlo, Particle filters, Piecewise-affine regression, Recursive identification @article{PIGA2020109002, title = {Rao-Blackwellized sampling for batch and recursive Bayesian inference of Piecewise Affine models}, author = {Dario Piga and Alberto Bemporad and Alessio Benavoli}, url = {http://alessiobenavoli.com/wp-content/uploads/2020/05/piga2020a-2.pdf}, doi = {https://doi.org/10.1016/j.automatica.2020.109002}, issn = {0005-1098}, year = {2020}, date = {2020-01-01}, journal = {Automatica}, volume = {117}, abstract = {This paper addresses batch (offline) and recursive (online) Bayesian inference of Piecewise Affine (PWA) regression models. By exploiting the particular structure of PWA models, efficient Rao-Blackwellized Monte Carlo sampling algorithms are developed to approximate the joint posterior distribution of the model parameters. Only the marginal posterior of the parameters used to describe the regressor-space partition is approximated, either in a batch mode using a Metropolis–Hastings Markov-Chain Monte Carlo (MCMC) sampler, or sequentially using particle filters, while the conditional distribution of the other model parameters is computed analytically. Probability distributions for the predicted outputs given new test inputs are derived and modifications of the proposed approaches to address maximum-a-posteriori estimate are discussed. The performance of the proposed algorithms is shown via a numerical example and through a benchmark case study on data-driven modelling of the electronic component placement process in a pick-and-place machine.}, keywords = {Bayesian inference, Markov Chain Monte Carlo, Particle filters, Piecewise-affine regression, Recursive identification}, pubstate = {published}, tppubtype = {article} } This paper addresses batch (offline) and recursive (online) Bayesian inference of Piecewise Affine (PWA) regression models. By exploiting the particular structure of PWA models, efficient Rao-Blackwellized Monte Carlo sampling algorithms are developed to approximate the joint posterior distribution of the model parameters. Only the marginal posterior of the parameters used to describe the regressor-space partition is approximated, either in a batch mode using a Metropolis–Hastings Markov-Chain Monte Carlo (MCMC) sampler, or sequentially using particle filters, while the conditional distribution of the other model parameters is computed analytically. Probability distributions for the predicted outputs given new test inputs are derived and modifications of the proposed approaches to address maximum-a-posteriori estimate are discussed. The performance of the proposed algorithms is shown via a numerical example and through a benchmark case study on data-driven modelling of the electronic component placement process in a pick-and-place machine. |
Azzimonti, Dario; Schürch, Manuel; Benavoli, Alessio; Zaffalon, Marco Orthogonally Decoupled Variational Fourier Features Journal Article arXiv preprint arXiv:2007.06363, 2020. @article{azzimonti2020orthogonally, title = {Orthogonally Decoupled Variational Fourier Features}, author = {Dario Azzimonti and Manuel Schürch and Alessio Benavoli and Marco Zaffalon}, url = {https://arxiv.org/pdf/2007.06363.pdf}, year = {2020}, date = {2020-01-01}, journal = {arXiv preprint arXiv:2007.06363}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
Schürch, Manuel; Azzimonti, Dario; Benavoli, Alessio; Zaffalon, Marco Recursive estimation for sparse Gaussian process regression Journal Article Automatica, 120 , pp. 109-127, 2020, ISSN: 0005-1098. Abstract | Links | BibTeX | Tags: Gaussian processes, Kalman filter, Non-parametric regression, Parameter estimation, Recursive estimation @article{SCHURCH2020109127, title = {Recursive estimation for sparse Gaussian process regression}, author = {Manuel Schürch and Dario Azzimonti and Alessio Benavoli and Marco Zaffalon}, url = {http://alessiobenavoli.com/wp-content/uploads/2020/07/2020automatica-gps-2.pdf}, doi = {https://doi.org/10.1016/j.automatica.2020.109127}, issn = {0005-1098}, year = {2020}, date = {2020-01-01}, journal = {Automatica}, volume = {120}, pages = {109-127}, abstract = {Gaussian Processes (GPs) are powerful kernelized methods for non-parametric regression used in many applications. However, their use is limited to a few thousand of training samples due to their cubic time complexity. In order to scale GPs to larger datasets, several sparse approximations based on so-called inducing points have been proposed in the literature. In this work we investigate the connection between a general class of sparse inducing point GP regression methods and Bayesian recursive estimation which enables Kalman Filter like updating for online learning. The majority of previous work has focused on the batch setting, in particular for learning the model parameters and the position of the inducing points, here instead we focus on training with mini-batches. By exploiting the Kalman filter formulation, we propose a novel approach that estimates such parameters by recursively propagating the analytical gradients of the posterior over mini-batches of the data. Compared to state of the art methods, our method keeps analytic updates for the mean and covariance of the posterior, thus reducing drastically the size of the optimization problem. We show that our method achieves faster convergence and superior performance compared to state of the art sequential Gaussian Process regression on synthetic GP as well as real-world data with up to a million of data samples.}, keywords = {Gaussian processes, Kalman filter, Non-parametric regression, Parameter estimation, Recursive estimation}, pubstate = {published}, tppubtype = {article} } Gaussian Processes (GPs) are powerful kernelized methods for non-parametric regression used in many applications. However, their use is limited to a few thousand of training samples due to their cubic time complexity. In order to scale GPs to larger datasets, several sparse approximations based on so-called inducing points have been proposed in the literature. In this work we investigate the connection between a general class of sparse inducing point GP regression methods and Bayesian recursive estimation which enables Kalman Filter like updating for online learning. The majority of previous work has focused on the batch setting, in particular for learning the model parameters and the position of the inducing points, here instead we focus on training with mini-batches. By exploiting the Kalman filter formulation, we propose a novel approach that estimates such parameters by recursively propagating the analytical gradients of the posterior over mini-batches of the data. Compared to state of the art methods, our method keeps analytic updates for the mean and covariance of the posterior, thus reducing drastically the size of the optimization problem. We show that our method achieves faster convergence and superior performance compared to state of the art sequential Gaussian Process regression on synthetic GP as well as real-world data with up to a million of data samples. |
2019 |
Benavoli, A; Balleri, A; Farina, A Joint Waveform and Guidance Control Optimization for Target Rendezvous Journal Article IEEE Transactions on Signal Processing, 67 (16), pp. 4357-4369, 2019, ISSN: 1053-587X. Abstract | Links | BibTeX | Tags: Cognitive radar, Kalman filter, linear quadratic Gaussian control @article{Benavoli2019d, title = {Joint Waveform and Guidance Control Optimization for Target Rendezvous}, author = {A Benavoli and A Balleri and A Farina}, url = {http://alessiobenavoli.com/wp-content/uploads/2019/08/Joint_waveform_and_guidance_control.pdf}, doi = {10.1109/TSP.2019.2929951}, issn = {1053-587X}, year = {2019}, date = {2019-08-01}, journal = {IEEE Transactions on Signal Processing}, volume = {67}, number = {16}, pages = {4357-4369}, abstract = {The algorithm developed in this paper jointly selects the optimal transmitted waveform and the control input so that a radar sensor on a moving platform with linear dynamics can reach a target by minimizing a predefined cost. The cost proposed in this paper accounts for the energy of the transmitted radar signal, the energy of the platform control input, and the relative position error between the platform and the target, which is a function of the waveform design and control input. Similarly to the linear quadratic Gaussian control problem, we demonstrate that the optimal solution satisfies the separation principle between filtering and optimization and, therefore, the optimum can be found analytically. The performance of the proposed solution is assessed with a set of simulations for a pulsed Doppler radar transmitting linearly frequency modulated chirps. Results show the effectiveness of the proposed approach for optimal waveform design and optimal guidance control.}, keywords = {Cognitive radar, Kalman filter, linear quadratic Gaussian control}, pubstate = {published}, tppubtype = {article} } The algorithm developed in this paper jointly selects the optimal transmitted waveform and the control input so that a radar sensor on a moving platform with linear dynamics can reach a target by minimizing a predefined cost. The cost proposed in this paper accounts for the energy of the transmitted radar signal, the energy of the platform control input, and the relative position error between the platform and the target, which is a function of the waveform design and control input. Similarly to the linear quadratic Gaussian control problem, we demonstrate that the optimal solution satisfies the separation principle between filtering and optimization and, therefore, the optimum can be found analytically. The performance of the proposed solution is assessed with a set of simulations for a pulsed Doppler radar transmitting linearly frequency modulated chirps. Results show the effectiveness of the proposed approach for optimal waveform design and optimal guidance control. |
Benavoli, Alessio ; Facchini, Alessandro ; Zaffalon, Marco Bernstein's socks, polynomial-time provable coherence and entanglement Inproceedings Bock, De J; de Campos, C; de Cooman, G; Quaeghebeur, E; Wheeler, G (Ed.): ISIPTA ;'19: Proceedings of the Eleventh International Symposium on Imprecise Probability: Theories and Applications, JMLR, 2019. Abstract | Links | BibTeX | Tags: bounded rationality, Sum-of-squares polynomials @inproceedings{zaffalon2019b, title = {Bernstein's socks, polynomial-time provable coherence and entanglement}, author = {Benavoli, Alessio and Facchini, Alessandro and Zaffalon, Marco}, editor = {J De Bock and C de Campos and G de Cooman and E Quaeghebeur and G Wheeler}, url = {http://alessiobenavoli.com/wp-content/uploads/2019/05/benavoli19-1.pdf}, year = {2019}, date = {2019-07-07}, booktitle = {ISIPTA ;'19: Proceedings of the Eleventh International Symposium on Imprecise Probability: Theories and Applications}, publisher = {JMLR}, series = {PJMLR}, abstract = {We recently introduced a bounded rationality ap- proach for the theory of desirable gambles. It is based on the unique requirement that being non- negative for a gamble has to be defined so that it can be provable in polynomial time. In this paper we continue to investigate properties of this class of models. In particular we verify that the space of Bernstein polynomials in which non- negativity is specified by the Krivine-Vasilescu certificate is yet another instance of this theory. As a consequence, we show how it is possible to construct in it a thought experiment uncovering entanglement with classical (hence non quantum) coins.}, keywords = {bounded rationality, Sum-of-squares polynomials}, pubstate = {published}, tppubtype = {inproceedings} } We recently introduced a bounded rationality ap- proach for the theory of desirable gambles. It is based on the unique requirement that being non- negative for a gamble has to be defined so that it can be provable in polynomial time. In this paper we continue to investigate properties of this class of models. In particular we verify that the space of Bernstein polynomials in which non- negativity is specified by the Krivine-Vasilescu certificate is yet another instance of this theory. As a consequence, we show how it is possible to construct in it a thought experiment uncovering entanglement with classical (hence non quantum) coins. |
Piga, Dario ; Benavoli, Alessio Semialgebraic Outer Approximations for Set-Valued Nonlinear Filtering Inproceedings Proc. on European Control Conference (ECC), 2019. Links | BibTeX | Tags: filtering, set of probabilities, SOS @inproceedings{Piga2019, title = {Semialgebraic Outer Approximations for Set-Valued Nonlinear Filtering}, author = {Piga, Dario and Benavoli, Alessio}, url = {http://alessiobenavoli.com/wp-content/uploads/2019/03/main_v5.pdf}, year = {2019}, date = {2019-03-24}, booktitle = {Proc. on European Control Conference (ECC)}, keywords = {filtering, set of probabilities, SOS}, pubstate = {published}, tppubtype = {inproceedings} } |
Benavoli, Alessio ; Facchini, Alessandro ; Zaffalon, Marco Computational Complexity and the Nature of Quantum Mechanics Technical Report 2019. Abstract | Links | BibTeX | Tags: Quantum mechanics @techreport{Benavoli2019bb, title = {Computational Complexity and the Nature of Quantum Mechanics}, author = {Benavoli, Alessio and Facchini, Alessandro and Zaffalon, Marco }, url = {https://arxiv.org/abs/1902.04569}, year = {2019}, date = {2019-02-14}, abstract = {Quantum theory (QT) has been confirmed by numerous experiments, yet we still cannot fully grasp the meaning of the theory. As a consequence, the quantum world appears to us paradoxical. Here we shed new light on QT by having it follow from two main postulates (i) the theory should be logically consistent; (ii) inferences in the theory should be computable in polynomial time. The first postulate is what we require to each well-founded mathematical theory. The computation postulate defines the physical component of the theory. We show that the computation postulate is the only true divide between QT, seen as a generalised theory of probability, and classical probability. All quantum paradoxes, and entanglement in particular, arise from the clash of trying to reconcile a computationally intractable, somewhat idealised, theory (classical physics) with a computationally tractable theory (QT) or, in other words, from regarding physics as fundamental rather than computation. }, keywords = {Quantum mechanics}, pubstate = {published}, tppubtype = {techreport} } Quantum theory (QT) has been confirmed by numerous experiments, yet we still cannot fully grasp the meaning of the theory. As a consequence, the quantum world appears to us paradoxical. Here we shed new light on QT by having it follow from two main postulates (i) the theory should be logically consistent; (ii) inferences in the theory should be computable in polynomial time. The first postulate is what we require to each well-founded mathematical theory. The computation postulate defines the physical component of the theory. We show that the computation postulate is the only true divide between QT, seen as a generalised theory of probability, and classical probability. All quantum paradoxes, and entanglement in particular, arise from the clash of trying to reconcile a computationally intractable, somewhat idealised, theory (classical physics) with a computationally tractable theory (QT) or, in other words, from regarding physics as fundamental rather than computation. |
Benavoli, Alessio; Facchini, Alessandro; Piga, Dario; Zaffalon, Marco Sum-of-squares for bounded rationality Journal Article International Journal of Approximate Reasoning, 105 , pp. 130 - 152, 2019, ISSN: 0888-613X. Abstract | Links | BibTeX | Tags: Sum-of-squares polynomials @article{Benavoli2019b, title = {Sum-of-squares for bounded rationality}, author = {Alessio Benavoli and Alessandro Facchini and Dario Piga and Marco Zaffalon}, url = {https://arxiv.org/abs/1705.02663}, doi = {10.1016/j.ijar.2018.11.012}, issn = {0888-613X}, year = {2019}, date = {2019-01-01}, journal = {International Journal of Approximate Reasoning}, volume = {105}, pages = {130 - 152}, abstract = {In the gambling foundation of probability theory, rationality requires that a subject should always (never) find desirable all nonnegative (negative) gambles, because no matter the result of the experiment the subject never (always) decreases her money. Evaluating the nonnegativity of a gamble in infinite spaces is a difficult task. In fact, even if we restrict the gambles to be polynomials in Rn, the problem of determining nonnegativity is NP-hard. The aim of this paper is to develop a computable theory of desirable gambles. Instead of requiring the subject to desire all nonnegative gambles, we only require her to desire gambles for which she can efficiently determine the nonnegativity (in particular sum-of-squares polynomials). We refer to this new criterion as bounded rationality.}, keywords = {Sum-of-squares polynomials}, pubstate = {published}, tppubtype = {article} } In the gambling foundation of probability theory, rationality requires that a subject should always (never) find desirable all nonnegative (negative) gambles, because no matter the result of the experiment the subject never (always) decreases her money. Evaluating the nonnegativity of a gamble in infinite spaces is a difficult task. In fact, even if we restrict the gambles to be polynomials in Rn, the problem of determining nonnegativity is NP-hard. The aim of this paper is to develop a computable theory of desirable gambles. Instead of requiring the subject to desire all nonnegative gambles, we only require her to desire gambles for which she can efficiently determine the nonnegativity (in particular sum-of-squares polynomials). We refer to this new criterion as bounded rationality. |
2018 |
Benavoli, Alessio Dual Probabilistic Programming Proceeding PROBPROG 2018 The International Conference on Probabilistic Programming, Wiesner Building E15 MIT, Cambridge, MA, USA 2018. Links | BibTeX | Tags: dual probabilistic programming, rationality @proceedings{Benavoli2018, title = {Dual Probabilistic Programming}, author = {Benavoli, Alessio }, url = {http://alessiobenavoli.com/wp-content/uploads/2018/09/dual_PP.pdf}, year = {2018}, date = {2018-10-04}, organization = {PROBPROG 2018 The International Conference on Probabilistic Programming, Wiesner Building E15 MIT, Cambridge, MA, USA}, keywords = {dual probabilistic programming, rationality}, pubstate = {published}, tppubtype = {proceedings} } |
2017 |
Benavoli, Alessio ; Corani, Giorgio ; Demsar, Janez ; Zaffalon, Marco Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysis Journal Article Journal of Machine Learning Research, 18 (77), pp. 1-36, 2017. Abstract | Links | BibTeX | Tags: bayesian statistics, machine learning @article{benavoli2016e, title = {Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysis}, author = {Benavoli, Alessio and Corani, Giorgio and Demsar, Janez and Zaffalon, Marco}, url = {http://jmlr.org/papers/v18/16-305.html}, year = {2017}, date = {2017-10-02}, journal = {Journal of Machine Learning Research}, volume = {18}, number = {77}, pages = {1-36}, abstract = {The machine learning community adopted the use of null hypothesis significance testing (NHST) in order to ensure the statistical validity of results. Many scientific fields however realized the shortcomings of frequentist reasoning and in the most radical cases even banned its use in publications. We should do the same: just as we have embraced the Bayesian paradigm in the development of new machine learning methods, so we should also use it in the analysis of our own results. We argue for abandonment of NHST by exposing its fallacies and, more importantly, offer better - more sound and useful - alternatives for it. }, keywords = {bayesian statistics, machine learning}, pubstate = {published}, tppubtype = {article} } The machine learning community adopted the use of null hypothesis significance testing (NHST) in order to ensure the statistical validity of results. Many scientific fields however realized the shortcomings of frequentist reasoning and in the most radical cases even banned its use in publications. We should do the same: just as we have embraced the Bayesian paradigm in the development of new machine learning methods, so we should also use it in the analysis of our own results. We argue for abandonment of NHST by exposing its fallacies and, more importantly, offer better - more sound and useful - alternatives for it. |
Piga, Dario ; Benavoli, Alessio A unified framework for deterministic and probabilistic D-stability analysis of uncertain polynomial matrices Journal Article Automatic Control, IEEE Transactions on, 62 (10), pp. 5437-5444, 2017. Abstract | Links | BibTeX | Tags: Control @article{benavoli2016c, title = {A unified framework for deterministic and probabilistic D-stability analysis of uncertain polynomial matrices}, author = {Piga, Dario and Benavoli, Alessio}, url = {http://arxiv.org/abs/1604.02031}, doi = {10.1109/TAC.2017.2699281}, year = {2017}, date = {2017-10-01}, journal = {Automatic Control, IEEE Transactions on}, volume = {62}, number = {10}, pages = {5437-5444}, abstract = { Many problems in systems and control theory can be formulated in terms of robust D-stability analysis, which aims at verifying if all the eigenvalues of an uncertain matrix lie in a given region D of the complex plane. Robust D-stability analysis is an NP-hard problem and many polynomial-time algorithms providing either sufficient or necessary conditions for an uncertain matrix to be robustly D-stable have been developed in the past decades. Despite the vast literature on the subject, most of the contributions consider specific families of uncertain matrices, mainly with interval or polytopic uncertainty. In this work, we present a novel approach providing sufficient conditions to verify if a family of matrices, whose entries depend polynomially on some uncertain parameters, is robustly D-stable. The only assumption on the stability region D is that its complement is a semialgebraic set described by polynomial constraints, which comprises the main important cases in stability analysis. Furthermore, the D-stability analysis problem is formulated in a probabilistic framework. In this context, the uncertain parameters characterizing the considered family of matrices are described by a set of non a priori specified probability measures. Only the support and some of the moments (e.g., expected values) are assumed to be known and, among all possible probability measures, we seek the one which provides the minimum probability of D-stability. The robust and the probabilistic D-stability analysis problems are formulated in a unified framework, and relaxations based on the theory of moments are used to solve the D-stability analysis problem through convex optimization. Application to robustness and probabilistic analysis of dynamical systems is discussed. }, keywords = {Control}, pubstate = {published}, tppubtype = {article} } Many problems in systems and control theory can be formulated in terms of robust D-stability analysis, which aims at verifying if all the eigenvalues of an uncertain matrix lie in a given region D of the complex plane. Robust D-stability analysis is an NP-hard problem and many polynomial-time algorithms providing either sufficient or necessary conditions for an uncertain matrix to be robustly D-stable have been developed in the past decades. Despite the vast literature on the subject, most of the contributions consider specific families of uncertain matrices, mainly with interval or polytopic uncertainty. In this work, we present a novel approach providing sufficient conditions to verify if a family of matrices, whose entries depend polynomially on some uncertain parameters, is robustly D-stable. The only assumption on the stability region D is that its complement is a semialgebraic set described by polynomial constraints, which comprises the main important cases in stability analysis. Furthermore, the D-stability analysis problem is formulated in a probabilistic framework. In this context, the uncertain parameters characterizing the considered family of matrices are described by a set of non a priori specified probability measures. Only the support and some of the moments (e.g., expected values) are assumed to be known and, among all possible probability measures, we seek the one which provides the minimum probability of D-stability. The robust and the probabilistic D-stability analysis problems are formulated in a unified framework, and relaxations based on the theory of moments are used to solve the D-stability analysis problem through convex optimization. Application to robustness and probabilistic analysis of dynamical systems is discussed. |
Benavoli, Alessio ; Facchini, Alessandro ; Zaffalon, Marco Bayes + Hilbert = Quantum Mechanics Conference Proceedings of the 14th International Conference on Quantum Physics and Logic (QPL 2017), Nijmegen, The Netherlands, 3-7 July, 2017. Links | BibTeX | Tags: Quantum mechanics @conference{Benavoli2017m, title = {Bayes + Hilbert = Quantum Mechanics}, author = {Benavoli, Alessio and Facchini, Alessandro and Zaffalon, Marco }, url = {http://qpl.science.ru.nl/papers/QPL_2017_paper_4.pdf}, year = {2017}, date = {2017-07-05}, booktitle = {Proceedings of the 14th International Conference on Quantum Physics and Logic (QPL 2017), Nijmegen, The Netherlands, 3-7 July}, keywords = {Quantum mechanics}, pubstate = {published}, tppubtype = {conference} } |
Balleri, Alessio ; Farina, Alfonso ; Benavoli, Alessio Coordination of optimal guidance law and adaptive radiated waveform for interception and rendezvous problems Journal Article IET Radar, Sonar & Navigation, 11 (7), pp. 1132 - 1139, 2017, ISSN: 1751-8784. Abstract | Links | BibTeX | Tags: Control @article{benavoli2017a, title = {Coordination of optimal guidance law and adaptive radiated waveform for interception and rendezvous problems}, author = { Balleri, Alessio and Farina, Alfonso and Benavoli, Alessio}, url = {http://digital-library.theiet.org/content/journals/10.1049/iet-rsn.2016.0547}, doi = {10.1049/iet-rsn.2016.0547}, issn = {1751-8784}, year = {2017}, date = {2017-07-01}, journal = {IET Radar, Sonar & Navigation}, volume = {11}, number = {7}, pages = {1132 - 1139}, publisher = {Institution of Engineering and Technology}, abstract = {We present an algorithm that allows an interceptor aircraft equipped with an airborne radar to meet another air target (the intercepted) by developing a guidance law and automatically adapting and optimising the transmitted waveform on a pulse to pulse basis. The algorithm uses a Kalman filter to predict the relative position and speed of the interceptor with respect to the target. The transmitted waveform is automatically selected based on its ambiguity function and accuracy properties along the approaching path. For each pulse, the interceptor predicts its position and velocity with respect to the target, takes a measurement of range and radial velocity and, with the Kalman filter, refines the relative range and range rate estimates. These are fed into a Linear Quadratic Gaussian (LQG) controller that ensures the interceptor reaches the target automatically and successfully with minimum error and with the minimum guidance energy consumption. }, keywords = {Control}, pubstate = {published}, tppubtype = {article} } We present an algorithm that allows an interceptor aircraft equipped with an airborne radar to meet another air target (the intercepted) by developing a guidance law and automatically adapting and optimising the transmitted waveform on a pulse to pulse basis. The algorithm uses a Kalman filter to predict the relative position and speed of the interceptor with respect to the target. The transmitted waveform is automatically selected based on its ambiguity function and accuracy properties along the approaching path. For each pulse, the interceptor predicts its position and velocity with respect to the target, takes a measurement of range and radial velocity and, with the Kalman filter, refines the relative range and range rate estimates. These are fed into a Linear Quadratic Gaussian (LQG) controller that ensures the interceptor reaches the target automatically and successfully with minimum error and with the minimum guidance energy consumption. |
Maradia, Umang; Benavoli, Alessio; Boccadoro, Marco; Klyuev, Mikhail; Gambardella, Luca; Wegener, Konrad; Bonesana, Claudio; Zaffalon, Marco EDM Drilling optimisation using stochastic optimisation Inproceedings Forthcoming CIRP Conference on Intelligent Computation in Manufacturing Engineering, ICME ‘17, pp. 1–8, Ischia, (IT), Forthcoming. BibTeX | Tags: smart manufacturing @inproceedings{benavoli2017o, title = {EDM Drilling optimisation using stochastic optimisation}, author = {Umang Maradia and Alessio Benavoli and Marco Boccadoro and Mikhail Klyuev and Luca Gambardella and Konrad Wegener and Claudio Bonesana and Marco Zaffalon}, year = {2017}, date = {2017-07-01}, booktitle = {CIRP Conference on Intelligent Computation in Manufacturing Engineering, ICME ‘17}, pages = {1--8}, address = {Ischia, (IT)}, keywords = {smart manufacturing}, pubstate = {forthcoming}, tppubtype = {inproceedings} } |
Baller, Alessio ; Benavoli, Alessio ; Farina, Alfonso Coordination of Guidance and Adaptive Radiated Waveform for Interception and Rendezvous Problems Conference 18th International Radar Symposium (IRS 2017), Prague, Czech Republic, 28-30 June, 2017. BibTeX | Tags: radar tracking @conference{Baller2017, title = {Coordination of Guidance and Adaptive Radiated Waveform for Interception and Rendezvous Problems}, author = {Baller, Alessio and Benavoli, Alessio and Farina, Alfonso}, year = {2017}, date = {2017-06-29}, booktitle = {18th International Radar Symposium (IRS 2017), Prague, Czech Republic, 28-30 June}, keywords = {radar tracking}, pubstate = {published}, tppubtype = {conference} } |
Benavoli, Alessio ; Facchini, Alessandro ; Piga, Dario ; Zaffalon, Marco SOS for bounded rationality Conference Proc. ISIPTA'17 Int. Symposium on Imprecise Probability: Theories and Applications, , PJMLR, 2017. Abstract | Links | BibTeX | Tags: bounded rationality, SOS @conference{Benavoli2017b, title = {SOS for bounded rationality}, author = { Benavoli, Alessio and Facchini, Alessandro and Piga, Dario and Zaffalon, Marco}, url = {https://arxiv.org/abs/1705.02663}, year = {2017}, date = {2017-05-07}, booktitle = {Proc. ISIPTA'17 Int. Symposium on Imprecise Probability: Theories and Applications, }, pages = {1--12}, publisher = {PJMLR}, abstract = {In the gambling foundation of probability theory, rationality requires that a subject should always (never) find desirable all nonnegative (negative) gambles, because no matter the result of the exper- iment the subject never (always) decreases her money. Evaluating the nonnegativity of a gamble in infinite spaces is a difficult task. In fact, even if we restrict the gambles to be polynomials in R n , the problem of determining nonnegativity is NP-hard. The aim of this paper is to develop a computable theory of desirable gambles. Instead of requiring the subject to accept all nonnegative gambles, we only require her to accept gambles for which she can efficiently determine the nonnegativity (in particular SOS polynomials). We call this new criterion bounded rationality.}, keywords = {bounded rationality, SOS}, pubstate = {published}, tppubtype = {conference} } In the gambling foundation of probability theory, rationality requires that a subject should always (never) find desirable all nonnegative (negative) gambles, because no matter the result of the exper- iment the subject never (always) decreases her money. Evaluating the nonnegativity of a gamble in infinite spaces is a difficult task. In fact, even if we restrict the gambles to be polynomials in R n , the problem of determining nonnegativity is NP-hard. The aim of this paper is to develop a computable theory of desirable gambles. Instead of requiring the subject to accept all nonnegative gambles, we only require her to accept gambles for which she can efficiently determine the nonnegativity (in particular SOS polynomials). We call this new criterion bounded rationality. |
Benavoli, Alessio ; Facchini, Alessandro ; Vicente-Perez, Jose ; Zaffalon, Marco A polarity theory for sets of desirable gambles Conference Proc. ISIPTA'17 Int. Symposium on Imprecise Probability: Theories and Applications, , PJMLR, 2017. Abstract | Links | BibTeX | Tags: desirability, lexicographic @conference{Benavoli2017c, title = {A polarity theory for sets of desirable gambles}, author = {Benavoli, Alessio and Facchini, Alessandro and Vicente-Perez, Jose and Zaffalon, Marco}, url = {https://arxiv.org/abs/1705.09574}, year = {2017}, date = {2017-05-07}, booktitle = {Proc. ISIPTA'17 Int. Symposium on Imprecise Probability: Theories and Applications, }, pages = {1-12}, publisher = {PJMLR}, abstract = {Coherent sets of almost desirable gambles and credal sets are known to be equivalent models. That is, there exists a bijection between the two collections of sets preserving the usual operations, e.g. conditioning. Such a correspondence is based on the polarity theory for closed convex cones. Learning from this simple observation, in this paper we introduce a new (lexicographic) polarity theory for general convex cones and then we apply it in order to establish an analogous correspondence between coherent sets of desirable gambles and convex sets of lexicographic probabilities.}, keywords = {desirability, lexicographic}, pubstate = {published}, tppubtype = {conference} } Coherent sets of almost desirable gambles and credal sets are known to be equivalent models. That is, there exists a bijection between the two collections of sets preserving the usual operations, e.g. conditioning. Such a correspondence is based on the polarity theory for closed convex cones. Learning from this simple observation, in this paper we introduce a new (lexicographic) polarity theory for general convex cones and then we apply it in order to establish an analogous correspondence between coherent sets of desirable gambles and convex sets of lexicographic probabilities. |
Benavoli, Alessio ; Colacito, Almudean ; Facchini, Alessandro ; Zaffalon, Marco Accepting and Rejecting Gambles: A Logical Point of View Conference Progic 2017: The 8th Workshop on Combining Probability and Logic, Munich 2017, 2017. Abstract | BibTeX | Tags: desirability @conference{Benavoli2017, title = {Accepting and Rejecting Gambles: A Logical Point of View}, author = { Benavoli, Alessio and Colacito, Almudean and Facchini, Alessandro and Zaffalon, Marco }, year = {2017}, date = {2017-03-30}, booktitle = {Progic 2017: The 8th Workshop on Combining Probability and Logic, Munich 2017}, abstract = {A powerful theory of uncertainty is that of coherent sets of desirable gambles (or desirability). It encompasses, in a uniform way, the Bayesian theory of probability as well as Bayesian robustness and many other theories of uncertainty. In recent years, several attempts have been carried out to explicitly formulate desirability as a logical system; for our purposes, the one by Gillett, Scherl and Shafer is particularly relevant. The goal of this paper is first to provide an appropriate semantics, with the aim to obtain a full completeness result for the logical system of Gillett, Scherl and Shafer. The second goal is to study, from a logical point of view, a generalisation of desirability that allows rejecting gambles to be possible too. Thus we enrich the system by adding a rejection operator, and we formulate some additional deductive rules concerning this operation. The obtained systems are investigated, both from a syntactical and a semantical point of view.}, keywords = {desirability}, pubstate = {published}, tppubtype = {conference} } A powerful theory of uncertainty is that of coherent sets of desirable gambles (or desirability). It encompasses, in a uniform way, the Bayesian theory of probability as well as Bayesian robustness and many other theories of uncertainty. In recent years, several attempts have been carried out to explicitly formulate desirability as a logical system; for our purposes, the one by Gillett, Scherl and Shafer is particularly relevant. The goal of this paper is first to provide an appropriate semantics, with the aim to obtain a full completeness result for the logical system of Gillett, Scherl and Shafer. The second goal is to study, from a logical point of view, a generalisation of desirability that allows rejecting gambles to be possible too. Thus we enrich the system by adding a rejection operator, and we formulate some additional deductive rules concerning this operation. The obtained systems are investigated, both from a syntactical and a semantical point of view. |
Corani, Giorgio ; Benavoli, Alessio ; Demšar, Janez ; Mangili, Francesca ; Zaffalon, Marco Statistical comparison of classifiers through Bayesian hierarchical modelling Journal Article Machine Learning, pp. 1–21, 2017, ISSN: 1573-0565. Links | BibTeX | Tags: bayesian statistics @article{Corani2017, title = {Statistical comparison of classifiers through Bayesian hierarchical modelling}, author = {Corani, Giorgio and Benavoli, Alessio and Demšar, Janez and Mangili, Francesca and Zaffalon, Marco}, url = {http://arxiv.org/abs/1609.08905}, doi = {10.1007/s10994-017-5641-9}, issn = {1573-0565}, year = {2017}, date = {2017-01-01}, journal = {Machine Learning}, pages = {1--21}, keywords = {bayesian statistics}, pubstate = {published}, tppubtype = {article} } |
Benavoli, Alessio ; Facchini, Alessandro ; Zaffalon, Marco A Gleason-Type Theorem for Any Dimension Based on a Gambling Formulation of Quantum Mechanics Journal Article Foundations of Physics, 47 (7), pp. 991–1002, 2017, ISSN: 1572-9516. Abstract | Links | BibTeX | Tags: Quantum mechanics @article{benavoli2016f, title = {A Gleason-Type Theorem for Any Dimension Based on a Gambling Formulation of Quantum Mechanics}, author = {Benavoli, Alessio and Facchini, Alessandro and Zaffalon, Marco}, url = {http://arxiv.org/abs/1606.03615}, doi = {10.1007/s10701-017-0097-0}, issn = {1572-9516}, year = {2017}, date = {2017-01-01}, journal = {Foundations of Physics}, volume = {47}, number = {7}, pages = {991--1002}, abstract = {Based on a gambling formulation of quantum mechanics, we derive a Gleason-type theorem that holds for any dimension n of a quantum system, and in particular for n = 2 . The theorem states that the only logically consistent probability assignments are exactly the ones that are definable as the trace of the product of a projector and a density matrix operator. In addition, we detail the reason why dispersion-free probabilities are actually not valid, or rational, probabilities for quantum mechanics, and hence should be excluded from consideration.}, keywords = {Quantum mechanics}, pubstate = {published}, tppubtype = {article} } Based on a gambling formulation of quantum mechanics, we derive a Gleason-type theorem that holds for any dimension n of a quantum system, and in particular for n = 2 . The theorem states that the only logically consistent probability assignments are exactly the ones that are definable as the trace of the product of a projector and a density matrix operator. In addition, we detail the reason why dispersion-free probabilities are actually not valid, or rational, probabilities for quantum mechanics, and hence should be excluded from consideration. |
Balleri, A; Farina, A; Benavoli, A Balleri A., Griffiths Baker H C (Ed.): pp. 137-154, in 'Biologically-Inspired Radar and Sonar: Lessons from nature.' Institution of Engineering and Technology, 2017. Abstract | Links | BibTeX | Tags: biologically inspired, radar tracking @inbook{benavoli2017n, title = {Biologically-inspired coordination of guidance and adaptive radiated waveform for interception and rendezvous problems}, author = {Balleri, A. and Farina, A. and Benavoli , A.}, editor = {Balleri, A., Griffiths, H., Baker, C.}, url = {http://digital-library.theiet.org/content/books/10.1049/sbra514e_ch7}, doi = {10.1049/SBRA514E_ch7}, year = {2017}, date = {2017-01-01}, pages = {137-154}, publisher = {in 'Biologically-Inspired Radar and Sonar: Lessons from nature.' Institution of Engineering and Technology}, series = {Radar, Sonar and Navigation}, abstract = {In this chapter, we take inspiration from the bat and develop an algorithm that guides an airborne radar interceptor towards a target by jointly developing an optimal guidance and automatically adapting and optimising the transmitted waveform on a pulse-to-pulse basis. The algorithm uses a Kalman filter to predict the relative position and speed of the interceptor with respect to the target.}, keywords = {biologically inspired, radar tracking}, pubstate = {published}, tppubtype = {inbook} } In this chapter, we take inspiration from the bat and develop an algorithm that guides an airborne radar interceptor towards a target by jointly developing an optimal guidance and automatically adapting and optimising the transmitted waveform on a pulse-to-pulse basis. The algorithm uses a Kalman filter to predict the relative position and speed of the interceptor with respect to the target. |
2016 |
Benavoli, Alessio ; Facchini, Alessandro ; Zaffalon, Marco Quantum mechanics: The Bayesian theory generalized to the space of Hermitian matrices Journal Article Physics Review A, 94 , pp. 042106, 2016. Abstract | Links | BibTeX | Tags: Quantum mechanics @article{benavoli2016d, title = {Quantum mechanics: The Bayesian theory generalized to the space of Hermitian matrices}, author = {Benavoli, Alessio and Facchini, Alessandro and Zaffalon, Marco}, url = {http://arxiv.org/abs/1605.08177 }, doi = {10.1103/PhysRevA.94.042106}, year = {2016}, date = {2016-10-01}, journal = {Physics Review A}, volume = {94}, pages = {042106}, publisher = {American Physical Society}, abstract = {We consider the problem of gambling on a quantum experiment and enforce rational behavior by a few rules. These rules yield, in the classical case, the Bayesian theory of probability via duality theorems. In our quantum setting, they yield the Bayesian theory generalized to the space of Hermitian matrices. This very theory is quantum mechanics: in fact, we derive all its four postulates from the generalized Bayesian theory. This implies that quantum mechanics is self-consistent. It also leads us to reinterpret the main operations in quantum mechanics as probability rules: Bayes' rule (measurement), marginalization (partial tracing), independence (tensor product). To say it with a slogan, we obtain that quantum mechanics is the Bayesian theory in the complex numbers.}, keywords = {Quantum mechanics}, pubstate = {published}, tppubtype = {article} } We consider the problem of gambling on a quantum experiment and enforce rational behavior by a few rules. These rules yield, in the classical case, the Bayesian theory of probability via duality theorems. In our quantum setting, they yield the Bayesian theory generalized to the space of Hermitian matrices. This very theory is quantum mechanics: in fact, we derive all its four postulates from the generalized Bayesian theory. This implies that quantum mechanics is self-consistent. It also leads us to reinterpret the main operations in quantum mechanics as probability rules: Bayes' rule (measurement), marginalization (partial tracing), independence (tensor product). To say it with a slogan, we obtain that quantum mechanics is the Bayesian theory in the complex numbers. |
Benavoli, Alessio ; Facchini, Alessandro ; Zaffalon, Marco Quantum rational preferences and desirability Inproceedings Proceedings of The 1st International Workshop on “Imperfect Decision Makers: Admitting Real-World Rationality”, NIPS 2016, 2016. Abstract | Links | BibTeX | Tags: Quantum mechanics @inproceedings{benavoli2016h, title = {Quantum rational preferences and desirability}, author = {Benavoli, Alessio and Facchini, Alessandro and Zaffalon, Marco}, url = {http://arxiv.org/abs/1610.06764}, year = {2016}, date = {2016-01-01}, booktitle = {Proceedings of The 1st International Workshop on “Imperfect Decision Makers: Admitting Real-World Rationality”, NIPS 2016}, journal = {ArXiv e-prints 1610.06764}, abstract = {We develop a theory of quantum rational decision making in the tradition of Anscombe and Aumann's axiomatisation of preferences on horse lotteries. It is essentially the Bayesian decision theory generalised to the space of Hermitian matrices. Among other things, this leads us to give a representation theorem showing that quantum complete rational preferences are obtained by means of expected utility considerations. }, keywords = {Quantum mechanics}, pubstate = {published}, tppubtype = {inproceedings} } We develop a theory of quantum rational decision making in the tradition of Anscombe and Aumann's axiomatisation of preferences on horse lotteries. It is essentially the Bayesian decision theory generalised to the space of Hermitian matrices. Among other things, this leads us to give a representation theorem showing that quantum complete rational preferences are obtained by means of expected utility considerations. |
Benavoli, Alessio ; de Campos, Cassio P Bayesian Dependence Tests for Continuous, Binary and Mixed Continuous-Binary Variables Journal Article Entropy, 18 (9), pp. 1-24, 2016. Abstract | Links | BibTeX | Tags: @article{benavoli2016g, title = {Bayesian Dependence Tests for Continuous, Binary and Mixed Continuous-Binary Variables}, author = {Benavoli, Alessio and de Campos, Cassio P.}, url = {http://www.idsia.ch/~alessio/benavoli2016g.pdf}, doi = {10.3390/e18090326}, year = {2016}, date = {2016-01-01}, journal = {Entropy}, volume = {18}, number = {9}, pages = {1-24}, publisher = {Multidisciplinary Digital Publishing Institute}, abstract = {Tests for dependence of continuous, discrete and mixed continuous-discrete variables are ubiquitous in science. The goal of this paper is to derive Bayesian alternatives to frequentist null hypothesis significance tests for dependence. In particular, we will present three Bayesian tests for dependence of binary, continuous and mixed variables. These tests are nonparametric and based on the Dirichlet Process, which allows us to use the same prior model for all of them. Therefore, the tests are “consistent” among each other, in the sense that the probabilities that variables are dependent computed with these tests are commensurable across the different types of variables being tested. By means of simulations with artificial data, we show the effectiveness of the new tests.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Tests for dependence of continuous, discrete and mixed continuous-discrete variables are ubiquitous in science. The goal of this paper is to derive Bayesian alternatives to frequentist null hypothesis significance tests for dependence. In particular, we will present three Bayesian tests for dependence of binary, continuous and mixed variables. These tests are nonparametric and based on the Dirichlet Process, which allows us to use the same prior model for all of them. Therefore, the tests are “consistent” among each other, in the sense that the probabilities that variables are dependent computed with these tests are commensurable across the different types of variables being tested. By means of simulations with artificial data, we show the effectiveness of the new tests. |
Benavoli, Alessio ; Zaffalon, Marco State Space representation of non-stationary Gaussian Processes Unpublished 2016. Links | BibTeX | Tags: Gaussian processes @unpublished{benavoli2016b, title = {State Space representation of non-stationary Gaussian Processes}, author = {Benavoli, Alessio and Zaffalon, Marco}, url = {http://arxiv.org/abs/1601.01544}, year = {2016}, date = {2016-01-01}, volume = {ArXiv e-prints 1601.01544}, keywords = {Gaussian processes}, pubstate = {published}, tppubtype = {unpublished} } |
Benavoli, Alessio ; Piga, Dario A probabilistic interpretation of set-membership filtering: Application to polynomial systems through polytopic bounding Journal Article Automatica, 70 , pp. 158 - 172, 2016. Abstract | Links | BibTeX | Tags: @article{benavoli2016a, title = {A probabilistic interpretation of set-membership filtering: Application to polynomial systems through polytopic bounding}, author = {Benavoli, Alessio and Piga, Dario}, url = {http://www.idsia.ch/~alessio/benavoli2016a.pdf}, doi = {http://dx.doi.org/10.1016/j.automatica.2016.03.021}, year = {2016}, date = {2016-01-01}, journal = {Automatica}, volume = {70}, pages = {158 - 172}, abstract = {Set-membership estimation is usually formulated in the context of set-valued calculus and no probabilistic calculations are necessary. In this paper, we show that set-membership estimation can be equivalently formulated in the probabilistic setting by employing sets of probability measures. Inference in set-membership estimation is thus carried out by computing expectations with respect to the updated set of probability measures P as in the probabilistic case. In particular, it is shown that inference can be performed by solving a particular semi-infinite linear programming problem, which is a special case of the truncated moment problem in which only the zero-th order moment is known (i.e., the support). By writing the dual of the above semi-infinite linear programming problem, it is shown that, if the nonlinearities in the measurement and process equations are polynomial and if the bounding sets for initial state, process and measurement noises are described by polynomial inequalities, then an approximation of this semi-infinite linear programming problem can efficiently be obtained by using the theory of sum-of-squares polynomial optimization. We then derive a smart greedy procedure to compute a polytopic outer-approximation of the true membership-set, by computing the minimum-volume polytope that outer-bounds the set that includes all the means computed with respect to P. }, keywords = {}, pubstate = {published}, tppubtype = {article} } Set-membership estimation is usually formulated in the context of set-valued calculus and no probabilistic calculations are necessary. In this paper, we show that set-membership estimation can be equivalently formulated in the probabilistic setting by employing sets of probability measures. Inference in set-membership estimation is thus carried out by computing expectations with respect to the updated set of probability measures P as in the probabilistic case. In particular, it is shown that inference can be performed by solving a particular semi-infinite linear programming problem, which is a special case of the truncated moment problem in which only the zero-th order moment is known (i.e., the support). By writing the dual of the above semi-infinite linear programming problem, it is shown that, if the nonlinearities in the measurement and process equations are polynomial and if the bounding sets for initial state, process and measurement noises are described by polynomial inequalities, then an approximation of this semi-infinite linear programming problem can efficiently be obtained by using the theory of sum-of-squares polynomial optimization. We then derive a smart greedy procedure to compute a polytopic outer-approximation of the true membership-set, by computing the minimum-volume polytope that outer-bounds the set that includes all the means computed with respect to P. |
Benavoli, Alessio ; Corani, Giorgio ; Mangili, Francesca Should We Really Use Post-Hoc Tests Based on Mean-Ranks? Journal Article Journal of Machine Learning Research, 17 (5), pp. 152-161, 2016. @article{benavoli2015c, title = {Should We Really Use Post-Hoc Tests Based on Mean-Ranks?}, author = {Benavoli, Alessio and Corani, Giorgio and Mangili, Francesca}, url = {http://www.idsia.ch/~alessio/benavoli2015c.pdf}, year = {2016}, date = {2016-01-01}, journal = {Journal of Machine Learning Research}, volume = {17}, number = {5}, pages = {152-161}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
de Campos, Cassio P; Benavoli, Alessio Joint Analysis of Multiple Algorithms and Performance Measures Journal Article New Generation Computing, 35 (1), pp. 69–86, 2016, ISSN: 1882-7055. Abstract | Links | BibTeX | Tags: @article{deCampos2016, title = {Joint Analysis of Multiple Algorithms and Performance Measures}, author = {de Campos, Cassio P. and Benavoli, Alessio}, url = {http://people.idsia.ch/~alessio/decampos-benavoli-ngc2016.pdf}, doi = {10.1007/s00354-016-0005-8}, issn = {1882-7055}, year = {2016}, date = {2016-01-01}, journal = {New Generation Computing}, volume = {35}, number = {1}, pages = {69--86}, abstract = {There has been an increasing interest in the development of new methods using Pareto optimality to deal with multi-objective criteria (for example, accuracy and time complexity). Once one has developed an approach to a problem of interest, the problem is then how to compare it with the state of art. In machine learning, algorithms are typically evaluated by comparing their performance on different data sets by means of statistical tests. Standard tests used for this purpose are able to consider jointly neither performance measures nor multiple competitors at once. The aim of this paper is to resolve these issues by developing statistical procedures that are able to account for multiple competing measures at the same time and to compare multiple algorithms altogether. In particular, we develop two tests: a frequentist procedure based on the generalized likelihood ratio test and a Bayesian procedure based on a multinomial-Dirichlet conjugate model. We further extend them by discovering conditional independences among measures to reduce the number of parameters of such models, as usually the number of studied cases is very reduced in such comparisons. Data from a comparison among general purpose classifiers are used to show a practical application of our tests.}, keywords = {}, pubstate = {published}, tppubtype = {article} } There has been an increasing interest in the development of new methods using Pareto optimality to deal with multi-objective criteria (for example, accuracy and time complexity). Once one has developed an approach to a problem of interest, the problem is then how to compare it with the state of art. In machine learning, algorithms are typically evaluated by comparing their performance on different data sets by means of statistical tests. Standard tests used for this purpose are able to consider jointly neither performance measures nor multiple competitors at once. The aim of this paper is to resolve these issues by developing statistical procedures that are able to account for multiple competing measures at the same time and to compare multiple algorithms altogether. In particular, we develop two tests: a frequentist procedure based on the generalized likelihood ratio test and a Bayesian procedure based on a multinomial-Dirichlet conjugate model. We further extend them by discovering conditional independences among measures to reduce the number of parameters of such models, as usually the number of studied cases is very reduced in such comparisons. Data from a comparison among general purpose classifiers are used to show a practical application of our tests. |
2015 |
Corani, Giorgio ; Benavoli, Alessio A Bayesian approach for comparing cross-validated algorithms on multiple data sets Inproceedings Accepted to the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD 2015), pp. 1–20, 2015. @inproceedings{corani2015b, title = {A Bayesian approach for comparing cross-validated algorithms on multiple data sets}, author = {Corani, Giorgio and Benavoli, Alessio}, url = {http://www.idsia.ch/~alessio/corani2015a.pdf}, year = {2015}, date = {2015-01-01}, booktitle = {Accepted to the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD 2015)}, pages = {1--20}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
Benavoli, Alessio ; Corani, Giorgio ; Mangili, Francesca ; Zaffalon, Marco A Bayesian nonparametric procedure for comparing algorithms Inproceedings Proceedings of the 31th International Conference on Machine Learning (ICML 2015), pp. 1–9, 2015. @inproceedings{benavoli2015b, title = {A Bayesian nonparametric procedure for comparing algorithms}, author = {Benavoli, Alessio and Corani, Giorgio and Mangili, Francesca and Zaffalon, Marco}, url = {http://www.idsia.ch/~alessio/benavoli2015b.pdf}, year = {2015}, date = {2015-01-01}, booktitle = {Proceedings of the 31th International Conference on Machine Learning (ICML 2015)}, pages = {1--9}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
Benavoli, Alessio ; Mangili, Francesca Gaussian Processes for Bayesian hypothesis tests on regression functions Inproceedings Proceedings of the 18th International Conference on Artificial Intelligence and Statistics (AISTAT 2015), pp. 1–9, 2015. @inproceedings{benavoli2015a, title = {Gaussian Processes for Bayesian hypothesis tests on regression functions}, author = {Benavoli, Alessio and Mangili, Francesca}, url = {http://www.idsia.ch/~alessio/benavoli2015a.pdf}, year = {2015}, date = {2015-01-01}, booktitle = {Proceedings of the 18th International Conference on Artificial Intelligence and Statistics (AISTAT 2015)}, pages = {1--9}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
Mangili, Francesca ; Benavoli, Alessio ; de Campos, Cassio P; Zaffalon, Marco Reliable survival analysis based on the Đirichlet Process Journal Article Biometrical Journal, 57 , pp. 1002–1019, 2015. @article{mangili2015a, title = {Reliable survival analysis based on the Đirichlet Process}, author = {Mangili, Francesca and Benavoli, Alessio and de Campos, Cassio P. and Zaffalon, Marco}, url = {http://www.idsia.ch/~alessio/mangili2015a.pdf}, doi = {10.1002/bimj.201500062}, year = {2015}, date = {2015-01-01}, journal = {Biometrical Journal}, volume = {57}, pages = {1002–1019}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
Corani, Giorgio ; Benavoli, Alessio A Bayesian approach for comparing cross-validated algorithms on multiple data sets Journal Article Machine Learning, 100 (2), pp. 285–304, 2015. @article{corani2015a, title = {A Bayesian approach for comparing cross-validated algorithms on multiple data sets}, author = {Corani, Giorgio and Benavoli, Alessio}, url = {http://www.idsia.ch/~alessio/corani2015a.pdf}, doi = {10.1007/s10994-015-5486-z}, year = {2015}, date = {2015-01-01}, journal = {Machine Learning}, volume = {100}, number = {2}, pages = {285--304}, publisher = {Springer US}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
Benavoli, Alessio ; Mangili, Francesca ; Ruggeri, Fabrizio ; Zaffalon, Marco Imprecise Dirichlet Process With Application to the Hypothesis Test on the Probability That X ≤ Y Journal Article Journal of Statistical Theory and Practice, 9 (3), pp. 658-684, 2015. @article{benavoli2014c, title = {Imprecise Dirichlet Process With Application to the Hypothesis Test on the Probability That X ≤ Y}, author = {Benavoli, Alessio and Mangili, Francesca and Ruggeri, Fabrizio and Zaffalon, Marco}, url = {http://www.idsia.ch/~alessio/benavoli2014c.pdf}, doi = {http://dx.doi.org/10.1080/15598608.2014.985997}, year = {2015}, date = {2015-01-01}, journal = {Journal of Statistical Theory and Practice}, volume = {9}, number = {3}, pages = {658-684}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
Mangili, Francesca ; Benavoli, Alessio New prior near-ignorance models on the simplex Journal Article International Journal of Approximate Reasoning, 56, Part B (0), pp. 278 - 306, 2015. @article{Mangili2014, title = {New prior near-ignorance models on the simplex}, author = {Mangili, Francesca and Benavoli, Alessio}, url = {http://www.idsia.ch/~alessio/mangili2014.pdf}, doi = {http://dx.doi.org/10.1016/j.ijar.2014.08.005}, year = {2015}, date = {2015-01-01}, journal = {International Journal of Approximate Reasoning}, volume = {56, Part B}, number = {0}, pages = {278 - 306}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
Benavoli, Alessio ; de Campos, Cassio P Statistical Tests for Joint Analysis of Performance Measures Inproceedings Suzuki, Joe ; Ueno, Maomi (Ed.): Advanced Methodologies for Bayesian Networks: Second International Workshop, AMBN 2015, Yokohama, Japan, November 16-18, 2015. Proceedings, pp. 76–92, Springer International Publishing, Cham, 2015, ISBN: 978-3-319-28379-1. Links | BibTeX | Tags: bayesian nonparametric, hypothesis testing @inproceedings{Benavoli2015i, title = {Statistical Tests for Joint Analysis of Performance Measures}, author = {Benavoli, Alessio and de Campos, Cassio P.}, editor = {Suzuki, Joe and Ueno, Maomi}, url = {http://dx.doi.org/10.1007/978-3-319-28379-1_6}, doi = {10.1007/978-3-319-28379-1_6}, isbn = {978-3-319-28379-1}, year = {2015}, date = {2015-01-01}, booktitle = {Advanced Methodologies for Bayesian Networks: Second International Workshop, AMBN 2015, Yokohama, Japan, November 16-18, 2015. Proceedings}, pages = {76--92}, publisher = {Springer International Publishing}, address = {Cham}, keywords = {bayesian nonparametric, hypothesis testing}, pubstate = {published}, tppubtype = {inproceedings} } |
Publications
2021 |
Gaussian Processes to speed up MCMC with automatic exploratory-exploitation effect Inproceedings ProbProg21, 2021. |
Choice functions based multi-objective Bayesian optimisation Technical Report 2021. |
Bayesian Kernelised Test of (In)dependence with Mixed-type Variables Conference The 8th IEEE International Conference on Data Science and Advanced Analytics (DSAA). Porto, Portugal, 2021. |
The Weirdness Theorem and the Origin of Quantum Paradoxes Journal Article Foundations of Physics, 51 (95), 2021. |
A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes Journal Article Machine Learning, pp. 1-39, 2021. |
Sparse Information Filter for Fast Gaussian Process Regression Conference European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases ECML PKDD, 2021. |
State Space approximation of Gaussian Processes for time series forecasting Conference Workshop on Advanced Analytics and Learning on Temporal Data, ECML/PKDD 21 , 2021. |
Nonlinear Desirability as a Linear Classification Problem Inproceedings ISIPTA'21 Int. Symposium on Imprecise Probability: Theories and Applications, PJMLR, 2021. |
Time series forecasting with Gaussian Processes needs priors Inproceedings European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases ECML PKDD , 2021. |
Quantum indistinguishability through exchangeable desirable gambles Inproceedings ISIPTA'21 Int. Symposium on Imprecise Probability: Theories and Applications, PJMLR, 2021. |
Joint Waveform and Guidance Control Optimization by Statistical Linearisation for Target Rendezvous Conference IEEE Radar Conference 2021, May 8-14, Atlanta (USA), 2021. |
Preferential Bayesian optimisation with Skew Gaussian Processes Inproceedings 2021 Genetic and Evolutionary Computation Conference Companion (GECCO '21 Companion), July 10--14, 2021, Lille, France , ACM, New York, NY, USA, 2021, ISBN: 978-1-4503-8351-6/21/07. |
2020 |
Skew Gaussian Processes for Classification Journal Article Machine Learning, 109 , pp. 1877–1902, 2020. |
A tutorial on uncertainty modeling for machine reasoning Journal Article Information Fusion, 55 , pp. 30 - 44, 2020, ISSN: 1566-2535. |
Rao-Blackwellized sampling for batch and recursive Bayesian inference of Piecewise Affine models Journal Article Automatica, 117 , 2020, ISSN: 0005-1098. |
Orthogonally Decoupled Variational Fourier Features Journal Article arXiv preprint arXiv:2007.06363, 2020. |
Recursive estimation for sparse Gaussian process regression Journal Article Automatica, 120 , pp. 109-127, 2020, ISSN: 0005-1098. |
2019 |
Joint Waveform and Guidance Control Optimization for Target Rendezvous Journal Article IEEE Transactions on Signal Processing, 67 (16), pp. 4357-4369, 2019, ISSN: 1053-587X. |
Bernstein's socks, polynomial-time provable coherence and entanglement Inproceedings Bock, De J; de Campos, C; de Cooman, G; Quaeghebeur, E; Wheeler, G (Ed.): ISIPTA ;'19: Proceedings of the Eleventh International Symposium on Imprecise Probability: Theories and Applications, JMLR, 2019. |
Semialgebraic Outer Approximations for Set-Valued Nonlinear Filtering Inproceedings Proc. on European Control Conference (ECC), 2019. |
Computational Complexity and the Nature of Quantum Mechanics Technical Report 2019. |
Sum-of-squares for bounded rationality Journal Article International Journal of Approximate Reasoning, 105 , pp. 130 - 152, 2019, ISSN: 0888-613X. |
2018 |
Dual Probabilistic Programming Proceeding PROBPROG 2018 The International Conference on Probabilistic Programming, Wiesner Building E15 MIT, Cambridge, MA, USA 2018. |
2017 |
Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysis Journal Article Journal of Machine Learning Research, 18 (77), pp. 1-36, 2017. |
A unified framework for deterministic and probabilistic D-stability analysis of uncertain polynomial matrices Journal Article Automatic Control, IEEE Transactions on, 62 (10), pp. 5437-5444, 2017. |
Bayes + Hilbert = Quantum Mechanics Conference Proceedings of the 14th International Conference on Quantum Physics and Logic (QPL 2017), Nijmegen, The Netherlands, 3-7 July, 2017. |
Coordination of optimal guidance law and adaptive radiated waveform for interception and rendezvous problems Journal Article IET Radar, Sonar & Navigation, 11 (7), pp. 1132 - 1139, 2017, ISSN: 1751-8784. |
EDM Drilling optimisation using stochastic optimisation Inproceedings Forthcoming CIRP Conference on Intelligent Computation in Manufacturing Engineering, ICME ‘17, pp. 1–8, Ischia, (IT), Forthcoming. |
Coordination of Guidance and Adaptive Radiated Waveform for Interception and Rendezvous Problems Conference 18th International Radar Symposium (IRS 2017), Prague, Czech Republic, 28-30 June, 2017. |
SOS for bounded rationality Conference Proc. ISIPTA'17 Int. Symposium on Imprecise Probability: Theories and Applications, , PJMLR, 2017. |
A polarity theory for sets of desirable gambles Conference Proc. ISIPTA'17 Int. Symposium on Imprecise Probability: Theories and Applications, , PJMLR, 2017. |
Accepting and Rejecting Gambles: A Logical Point of View Conference Progic 2017: The 8th Workshop on Combining Probability and Logic, Munich 2017, 2017. |
Statistical comparison of classifiers through Bayesian hierarchical modelling Journal Article Machine Learning, pp. 1–21, 2017, ISSN: 1573-0565. |
A Gleason-Type Theorem for Any Dimension Based on a Gambling Formulation of Quantum Mechanics Journal Article Foundations of Physics, 47 (7), pp. 991–1002, 2017, ISSN: 1572-9516. |
Balleri A., Griffiths Baker H C (Ed.): pp. 137-154, in 'Biologically-Inspired Radar and Sonar: Lessons from nature.' Institution of Engineering and Technology, 2017. |
2016 |
Quantum mechanics: The Bayesian theory generalized to the space of Hermitian matrices Journal Article Physics Review A, 94 , pp. 042106, 2016. |
Quantum rational preferences and desirability Inproceedings Proceedings of The 1st International Workshop on “Imperfect Decision Makers: Admitting Real-World Rationality”, NIPS 2016, 2016. |
Bayesian Dependence Tests for Continuous, Binary and Mixed Continuous-Binary Variables Journal Article Entropy, 18 (9), pp. 1-24, 2016. |
State Space representation of non-stationary Gaussian Processes Unpublished 2016. |
A probabilistic interpretation of set-membership filtering: Application to polynomial systems through polytopic bounding Journal Article Automatica, 70 , pp. 158 - 172, 2016. |
Should We Really Use Post-Hoc Tests Based on Mean-Ranks? Journal Article Journal of Machine Learning Research, 17 (5), pp. 152-161, 2016. |
Joint Analysis of Multiple Algorithms and Performance Measures Journal Article New Generation Computing, 35 (1), pp. 69–86, 2016, ISSN: 1882-7055. |
2015 |
A Bayesian approach for comparing cross-validated algorithms on multiple data sets Inproceedings Accepted to the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD 2015), pp. 1–20, 2015. |
A Bayesian nonparametric procedure for comparing algorithms Inproceedings Proceedings of the 31th International Conference on Machine Learning (ICML 2015), pp. 1–9, 2015. |
Gaussian Processes for Bayesian hypothesis tests on regression functions Inproceedings Proceedings of the 18th International Conference on Artificial Intelligence and Statistics (AISTAT 2015), pp. 1–9, 2015. |
Reliable survival analysis based on the Đirichlet Process Journal Article Biometrical Journal, 57 , pp. 1002–1019, 2015. |
A Bayesian approach for comparing cross-validated algorithms on multiple data sets Journal Article Machine Learning, 100 (2), pp. 285–304, 2015. |
Imprecise Dirichlet Process With Application to the Hypothesis Test on the Probability That X ≤ Y Journal Article Journal of Statistical Theory and Practice, 9 (3), pp. 658-684, 2015. |
New prior near-ignorance models on the simplex Journal Article International Journal of Approximate Reasoning, 56, Part B (0), pp. 278 - 306, 2015. |
Statistical Tests for Joint Analysis of Performance Measures Inproceedings Suzuki, Joe ; Ueno, Maomi (Ed.): Advanced Methodologies for Bayesian Networks: Second International Workshop, AMBN 2015, Yokohama, Japan, November 16-18, 2015. Proceedings, pp. 76–92, Springer International Publishing, Cham, 2015, ISBN: 978-3-319-28379-1. |