Potemkin Understanding in Large Language Models

I was invited to give a talk to OpenAI’s Economic Research team. The talk involved how to evaluate and benchmark large language models for depth of conceptual understanding. Here are the slides.