The study, titled “A Formal Framework for Knowledge Acquisition: Going Beyond Machine Learning,” was published in Entropy.

J. Sunil Rao, Ph.D., and Daniel A. Díaz-Pachón, Ph.D., both faculty in the Division of Biostatistics, have developed a mathematical framework through research that enables precise definitions of learning and knowledge in collaboration with Stockholm University.

In this framework, learning is defined as an increase in true belief, which is measured by a concept known as active information. This measure contrasts the agent's level of belief with that of a person who is “completely ignorant” and can be used to assess if learning took place when the agent's level of belief in a true statement grew or declined. Knowledge acquisition, on the other hand, requires that the belief be justified, resulting in a higher level of belief.

The framework combines elements of both frequentism and Bayesianism – two interpretations of probability with the former being per frequency of occurrence and the latter being per reasonable expectation – to probability and statistical inference.

“This formulation has important implications for supervised machine learning,” said Dr. Díaz-Pachón, Research Assistant Professor at DPHS.

“The performance of a machine learning algorithm is typically assessed in terms of prediction accuracy, with less regard for how closely the input represents reality. That is, the purpose of machine learning is learning rather than knowledge acquisition,” he added.

Thus, even when the agent interprets the data objectively and correctly, full knowledge acquisition may nonetheless fail asymptotically. “This is a disadvantage since knowledge acquisition often provides deeper insights than mere learning,” said Dr. Díaz-Pachón.

It is important to consider the distinction between learning and knowledge acquisition, and how this framework can be used to identify the limitations of machine learning, which often focuses on learning rather than knowledge acquisition.

This framework can be applied in a variety of contexts, including coin tossing, historical and future events, and study replication. It can also be extended to a sequential setting, where information and data are updated over time.

Written by Deycha Torres Hernández

Published on December 20, 2022