Numerous new regulatory frameworks and policy initiatives demand checks of machine learning algorithms. We argue that algorithm checking procedures have to be understood as information production processes surrounded by economic incentive schemes and that the modern market design approach provides a helpful framework and toolbox to develop appropriate checking procedures.

A good reference point is the Turing Box-model suggested by Epstein et al. (2018), in which algorithms are put into a clearing house to be checked by examiners. We argue that additionally having appropriate economic incentive schemes and market mechanisms in place is crucial for effective information production in such settings. We term this combination Turing Markets.

Within the frame of the proposed project, we will run a competition for designs of Turing Markets, which shall produce information on the trustworthiness of a financial market trading algorithm. The most promising Turing Market designs will then be put to the test in a lab in the field-experiment. Each design constitutes an experimental arm to which groups of machine learning experts will be assigned.

The approach of carefully designing Turing Markets for particular classes of machine learning algorithms could well generalize beyond this proof-of-concept setting. Through community building and outreach efforts, we aim at kickstarting a community of academic researchers, practitioners and regulators interested in further cultivating this idea.


Baden-Württemberg Stiftung gGmbH , Stuttgart , DE

Project duration

01.12.2020 - 30.11.2023


Digital Economy