Huang Track Records Model

A bandit model created by Alice C.W. Huang, which depicts how scientific networks whose agents depend heavily upon track records to make judgments about the credence of hypotheses may be less accurate than networks whose agents rely more on evidential reasoning to judge hypotheses. View the full paper here.

Abstract

In the literature on expert trust, it is often assumed that track records are the gold standard for evaluating expertise, and the difficulty of expert identification arises from either the lack of access to track records, or the inability to assess them (Goldman [2001]; Schurz [2012]; Nguyen [2020]). I show, using a computational model, that even in an idealized environment where agents have a God’s eye view on track records, they may fail to identify experts. Under plausible conditions, selecting testimony based on track records ends up reducing overall accuracy, and preventing the community from identifying the real experts.

Model Details

Parameters include

• Size of network
• Objective probability of B
• Objective probability of A
• Network structure: complete, cycle, wheel