Human betaherpesviruses 6A and 6B (HHV-6A and HHV-6B) are highly prevalent in human populations. The genomes of these viruses can be stably integrated at the telomeres of human chromosomes and be vertically transmitted (inherited chromosomally integrated HHV-6A/HHV-6B, iciHHV-6A/iciHHV-6B). We reconstructed the population structures of HHV-6A and HHV-6B, showing that HHV-6A diverged less than HHV-6B genomes from the projected common ancestral population. Thus, HHV-6B genomes experienced stronger drift, as also supported by calculation of nucleotide diversity and Tajima's D. Analysis of ancestry proportions indicated that HHV-6A exogenous viruses and iciHHV-6A derived most of their genomes from distinct ancestral sources. Conversely, ancestry proportions were similar in exogenous HHV-6B viruses and iciHHV-6B. In line with previous indications, this suggests the distinct exogenous viral populations that originated iciHHV-6B in subjects with European and Asian ancestry are still causing infections in the corresponding geographic areas. Notably, for both iciHHV-6A and iciHHV-6B, we found that European and American sequences tend to have high proportions of ancestry from viral populations that experienced considerable drift, suggesting that they underwent one or more bottlenecks followed by population expansion. Finally, analysis of HHV-6B exogenous viruses sampled in Japan indicated that proportions of ancestry components of most of these viruses are different from the majority of those sampled in the USA. More generally, we show that, in both viral species, both integrated and exogenous viral genomes have different ancestry components, partially depending on geographic location. It would be extremely important to determine whether such differences account for the diversity of HHV-6A/HHV-6B-associated clinical symptoms and epidemiology. Also, the sequencing of additional exogenous and integrated viral genomes will be instrumental to confirm and expand our conclusions, which are based on a relatively small number of genomes, sequenced with variable quality, and with unequal sampling in terms of geographic origin.