Reading list

Foundation models must handle multiple generative processes, yet mechanistic interpretability largely studies capabilities in isolation; it remains unclear how a single transformer organizes multiple, potentially conflicting 'world models'. Previous experiments on Othello playing neural-networks test world-model learning but focus on a single game with a single set of rules. We introduce MetaOthello, a controlled suite of Othello variants with shared syntax but different rules or tokenizations, and train small GPTs on mixed-variant data to study how multiple world models are organized in a shared representation space. We find that transformers trained on mixed-game data do not partition their capacity into isolated sub-models; instead, they converge on a mostly shared board-state representation that transfers causally across variants. Linear probes trained on one variant can intervene on another's internal state with effectiveness approaching that of matched probes. For isomorphic games with token remapping, representations are equivalent up to a single orthogonal rotation that generalizes across layers. When rules partially overlap, early layers maintain game-agnostic representations while a middle layer identifies game identity, and later layers specialize. MetaOthello offers a path toward understanding not just whether transformers learn world models, but how they organize many at once.

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.

Social media platforms often assume that users can self-correct against misinformation. However, social media users are not equally susceptible to all misinformation as their biases influence what types of misinformation might thrive and who might be at risk. We call “diverse misinformation” the complex relationships between human biases and demographics represented in misinformation. To investigate how users’ biases impact their susceptibility and their ability to correct each other, we analyze classification of deepfakes as a type of diverse misinformation. We chose deepfakes as a case study for three reasons: (1) their classification as misinformation is more objective; (2) we can control the demographics of the personas presented; (3) deepfakes are a real-world concern with associated harms that must be better understood. Our paper presents an observational survey (N = 2016) where participants are exposed to videos and asked questions about their attributes, not knowing some might be deepfakes. Our analysis investigates the extent to which different users are duped and which perceived demographics of deepfake personas tend to mislead. We find that accuracy varies by demographics, and participants are generally better at classifying videos that match them. We extrapolate from these results to understand the potential population-level impacts of these biases using a mathematical model of the interplay between diverse misinformation and crowd correction. Our model suggests that diverse contacts might provide “herd correction” where friends can protect each other. Altogether, human biases and the attributes of misinformation matter greatly, but having a diverse social group may help reduce susceptibility to misinformation.

Opinion dynamics, the study of how individual beliefs and collective public opinion evolve, is a fertile domain for applying the framework of statistical physics to complex social phenomena. Like physical systems, societies exhibit macroscopic regularities arising from numerous localized interactions, leading to outcomes such as consensus or fragmentation. Opinion dynamics has grown from a fringe interest to an active field of physics, attracting interdisciplinary methods from computer science, economics, and sociology. This review covers the ongoing rapid progress of the field, driven by an unprecedented surge in available large-scale behavioral data, bridging the inherently interdisciplinary literature. We begin with essential concepts and definitions, encompassing the nature of opinions and their microscopic and macroscopic dynamics. This foundation leads to an overview of empirical research, from lab experiments to large-scale data analysis, which informs and validates models of opinion dynamics. We then categorize models by the macroscopic phenomena they describe (e.g., consensus, polarization, echo chambers) and the microscopic mechanisms of opinion change they encode (e.g., homophily, assimilation). The review covers common analytical and computational tools to study these models, such as stochastic processes, exact and approximate treatments, simulation techniques, and optimization methods. Finally, we explore emerging frontiers, including the ongoing effort to establish a closer connection from empirical data to models, and the recent developments in building artificial intelligence agents as testbeds for novel social phenomena. By systematizing terminology across disciplines and emphasizing analogies with traditional physics, we highlight connections between theoretical models, concepts in social psychology, and observable societal phenomena. This review aims to consolidate current knowledge, provide a robust theoretical foundation, and shape future research directions in opinion dynamics.

Hammond, Kristina Lerman, Juniper Lovato, John J. Openshaw, S. Redner, Samuel V. Scarpino, Guillaume St-Onge, Timothy R. Tangherlini, Jean-Gabriel Young From pathogens and computer viruses to genes and memes, contagion models have found widespread utility across the natural and social sciences. Despite their success and breadth of adoption, the approach and structure of these models remain surprisingly siloed by field. Given the siloed nature of their development and widespread use, one persistent assumption is that a given contagion can be studied in isolation, independently from what else might be spreading in the population. In reality, countless contagions of biological and social nature interact within hosts (interacting with existing beliefs, or the immune system) and across hosts (interacting in the environment, or affecting transmission mechanisms). Additionally, from a modeling perspective, we know that relaxing these assumptions has profound effects on the physics and translational implications of the models. Here, we review mechanisms for interactions in social and biological contagions, as well as the models and frameworks developed to include these interactions in the study of the contagions. We highlight existing problems related to the inference of interactions and to the scalability of mathematical models and identify promising avenues of future inquiries. In doing so, we highlight the need for interdisciplinary efforts under a unified science of contagions and for removing a common dichotomy between social and biological contagions.

Science is essential to innovation and economic prosperity. Although studies have shown that national scientific development is affected by geographic, historic, and economic factors, it remains unclear whether there are universal structures and trajectories of national scientific development that can inform forecasting and policymaking. Here, by examining countries' scientific 'exports'-publications that are indexed in international databases-we reveal a three-cluster structure in the relatedness network of disciplines that underpin national scientific development and the organization of global science. Tracing the evolution of national research portfolios reveals that while nations are proceeding to more diverse research profiles individually, scientific production is increasingly specialized in global science over the past decades. By uncovering the underlying structure of scientific development and connecting it with economic development, our results may offer a new perspective on the evolution of global science.

Large language models are increasingly used to adapt math word problems for personalized learning at scale, but it remains an open question whether those adaptations are consistent across models, preserve cultural diversity at scale, and reveal which cultural entities models treat as most salient. Weanalyze how Claude Opus 4, GPT 4.1, and Gemini 2.5 Pro adapt 60 English math word problems into Bengali, Hindi,Punjabi (India), Urdu, Sindhi (Pakistan), Italian, and Sicilian (Italy), a language set spanning the full resource spectrum, from high-resource Italian and Hindi to under-studied Sindhi, Sicilian, and Punjabi. We annotate 6,489 entity transformations, coding whether models preserve, localize, generalize, omit, or change entities such as names, foods, and places. Models agree on transformation type in 62.5% of cases and on specific substitutions in only 33.5%, meaning model choice directly shapes which cultural world students encounter. All 21 language-model combinations show entropy collapse, with adaptation compressing rather than expanding cultural diversity. Models prioritize surface markers such as names, foods, and currencies while preserving deeper structural features such as grade-level systems that embed culturally specific assumptions. Despite prompts specifying target countries, models misattribute regional context by using Bangladeshi taka for Indian Bengali students and produce cross-cultural contamination, such as adapting egg hunts as Eid activities. Some failures are visible in individual translations. Others, including diversity collapse, systematic preference for surface markers, and consistent regional misattribution, emerge only through corpus-level analysis. The surface plausibility that makes adapted problems look correct is precisely what makes deeper failures easy to overlook.

Because of the “all-or-none” character of nervous activity, neural events and the relations among them can be treated by means of propositional logic. It is found that the behavior of every net can be described in these terms, with the addition of more complicated logical means for nets containing circles; and that for any logical expression satisfying certain conditions, one can find a net behaving in the fashion it describes. It is shown that many particular choices among possible neurophysiological assumptions are equivalent, in the sense that for every net behaving under one assumption, there exists another net which behaves under the other and gives the same results, although perhaps not in the same time. Various applications of the calculus are discussed.

Word embeddings are increasingly being used as a tool to study word associations in specific corpora. However, it is unclear whether such embeddings reflect enduring properties of language or if they are sensitive to inconsequential variations in the source documents. We find that nearest-neighbor distances are highly sensitive to small changes in the training corpus for a variety of algorithms. For all methods, including specific documents in the training set can result in substantial variations. We show that these effects are more prominent for smaller training corpora. We recommend that users never rely on single embedding models for distance calculations, but rather average over multiple bootstrap samples, especially for small corpora.

Models of how things spread often assume that transmission mechanisms are fixed over time. However, social contagions—the spread of ideas, beliefs, innovations—can lose or gain in momentum as they spread: ideas can get reinforced, beliefs strengthened, products refined. We study the impacts of such self-reinforcement mechanisms in cascade dynamics. We use different mathematical modeling techniques to capture the recursive, yet changing nature of the process. We find a critical regime with a range of power-law cascade size distributions with nonuniversal scaling exponents. This regime clashes with classic models, where criticality requires fine-tuning at a precise critical point. Self-reinforced cascades produce critical-like behavior over a wide range of parameters, which may help explain the ubiquity of power-law distributions in empirical social data.

Our belief systems are shaped by social processes, such as observations and influence, and by cognitive processes, such as the drive for internal coherence. These processes steer how individual beliefs evolve and become connected. The resulting belief networks contain both causal and associative links, including spurious ones, such as stereotypes. Here, we develop an agent-based model of belief networks that demonstrates how two basic mechanisms -- social interaction and a drive for internal coherence -- can give rise to such stereotypes without any underlying reality. We further demonstrate how stereotypes, when coupled with shared group identity, can give rise to affective polarization, even in the absence of ideological conflicts.