Aluminum (Al) toxicity is a major stress factor limiting crop productivity in acid soil. Although there is great genotypic variation in tolerance to Al toxicity, the underlying molecular mechanisms are poorly understood. Here, we report that, in barley (Hordeum vulgare), the fourth largest cereal crop produced in the world, both retrotransposon insertion and DNA methylation are involved in regulating differential Al tolerance. HvAACT1 is a major gene responsible for citrate secretion from the roots for external detoxification of Al. A multiretrotransposon-like (MRL) sequence insertion at least 15.3 kb in length was detected in the upstream genomic region of HvAACT1 that displayed promoter activity and significantly enhanced HvAACT1 expression, especially in the root tips of Al-tolerant accessions. Furthermore, in a number of accessions with low levels of HvAACT1 expression, this MRL insertion was present but highly methylated. Geographical analysis showed that accessions with this MRL insertion are distributed mainly in European areas with acid soils. Two wild barley accessions were found to possess this MRL insertion, but with a high degree of methylation. These results indicate that the MRL insertion and its degree of DNA methylation influence HvAACT1 expression and that demethylation of this MRL insertion, which facilitates adaptation to acid soils, occurred following barley domestication. Moreover, our results indicate that barley accessions in East Asia and Europe have developed independent but equivalent strategies to withstand Al toxicity in acid soils.