Inflection classes introduce "gratuitous complexity" to languages (Baerman et al. 2010). They represent a layer of structure that mediates between form and meaning, preventing a direct mapping from one to the other. This can make it difficult for child learners and adult speakers to predict the forms of words and may increase the need for speakers to lexically store inflected words. If languages were designed to have only as much structure as is needed for efficient communication, no languages would have inflection classes. Yet they are a core organizing principle in many languages. In this talk I ask: Why do (some) languages tolerate inflection classes? An answer dating back to the Neogrammarians is that sound change creates morphological complexity as a byproduct. However, this does not explain why languages may change in ways that make inflection class structure more pervasive, or why there seem to be cross-linguistic differences in the likelihood of inflection class structure emerging and persisting. In this talk I explore how a systems-oriented approach, focusing on the ways in which elements in an inflectional system interact, can help to solve this mystery.