Language models as function approximators of text data: disrupting comprehension through and adversarial attack

Abstract

Large Language Models such as ChatGPT are examples of Machine Learning models (ML), and as such, they are best regarded as “function approximators,” or vast correlation machines. That is, given a dataset, however big, of pairs of inputs and outputs, a ML network will construct a function that best describes (approximates) the distribution in the data, and from whence it can generate similar input-output pairs if prompted. In the case of ChatGPT, the dataset is composed of text—the products of the language faculty. In this sense, ChatGPT does not model a natural language per se, but the statistical distribution of text, the latter treated as recorded, static data. A fortiori, the claim that some of the results emanating from the development of Large Language Models bear any consequence for the study of language, especially regarding linguistic competence and acquisitional studies, appear to be the case of committing a category mistake.