Educating LLMs to cause like Bayesians

Evaluating LLMs’ Bayesian capabilities

As with people, to be efficient, an LLM’s person interactions require continuous updates to its probabilistic estimates of the person’s preferences primarily based on every new interplay with them. Right here we ask: do LLMs act as if they’ve probabilistic estimates which are up to date as anticipated from optimum Bayesian inference? To the extent that the LLM’s conduct deviates from the optimum Bayesian technique, how can we decrease these deviations?

To check this, we used a simplified flight advice activity, wherein the LLMs work together as assistants with a simulated person for 5 rounds. In every spherical, three flight choices have been introduced to each the person and the assistant. Every flight was outlined by a departure time, a period, quite a few stops, and a value. Every simulated person was characterised by a set of preferences: for every characteristic, they might have a robust or weak choice for prime or low values of the characteristic (e.g., they could choose longer or shorter flights), or no choice concerning this characteristic.

We in contrast the LLMs’ conduct to that of a mannequin, a Bayesian assistant, that follows the optimum Bayesian technique. This mannequin maintains a chance distribution that displays its estimates of the person’s preferences, and makes use of Bayes’ rule to replace this distribution as new details about the person’s selections turns into out there. Not like many real-life eventualities, the place it’s troublesome to specify and implement the Bayesian technique computationally, on this managed setting it’s straightforward to implement and permits us to exactly estimate the extent to which LLMs deviate from it.

The objective of the assistant was to suggest the flight that matches the person’s alternative. On the finish of every spherical, the person indicated to the assistant whether or not or not it selected appropriately, and offered it with the proper reply.