Telomeres protect the ends of eukaryotic chromosomes and are essential for cell viability. In mammals, telomere dynamics vary with life history traits (e.g. body mass and longevity), suggesting differential selection depending on physiological characteristics. Telomeres, in analogy to centromeric regions, also represent candidate meiotic drivers and subtelomeric DNA evolves rapidly. We analyzed the evolutionary history of mammalian genes implicated in telomere homeostasis (TEL genes). We detected widespread positive selection and we tested two alternative hypotheses: (i) fast evolution is driven by changes in life history traits; (ii) a conflict with selfish DNA elements at the female meiosis represents the underlying selective pressure. By accounting for the phylogenetic relationships among mammalian species, we show that life history traits do not contribute to shape diversity of TEL genes. Conversely, the evolutionary rate of TEL genes correlates with expression levels during meiosis and episodes of positive selection across mammalian species are associated with karyotype features (number of chromosome arms). We thus propose a telomere drive hypothesis, whereby (sub)telomeres and telomere-binding proteins are engaged in an intra-genomic conflict similar to the one described for centromeres.