Cognitive impairment affects about 50% of multiple sclerosis (MS) patients, but the mechanisms underlying this remain unclear. The default mode network (DMN) has been linked with cognition, but in MS its role is still poorly understood. Moreover, within an extended DMN network including the cerebellum (CBL-DMN), the contribution of cortico-cerebellar connectivity to MS cognitive performance remains unexplored. The present study investigated associations of DMN and CBL-DMN structural connectivity with cognitive processing speed in MS, in both cognitively impaired (CIMS) and cognitively preserved (CPMS) MS patients. 68 MS patients and 22 healthy controls (HCs) completed a symbol digit modalities test (SDMT) and had 3T brain magnetic resonance imaging (MRI) scans that included a diffusion weighted imaging protocol. DMN and CBL-DMN tracts were reconstructed with probabilistic tractography. These networks (DMN and CBL-DMN) and the cortico-cerebellar tracts alone were modeled using a graph theoretical approach with fractional anisotropy (FA) as the weighting factor. Brain parenchymal fraction (BPF) was also calculated. In CIMS SDMT scores strongly correlated with the FA-weighted global efficiency (GE) of the network [GE(CBL-DMN): ρ = 0.87, R2 = 0.76, p < 0.001; GE(DMN): ρ = 0.82, R2 = 0.67, p < 0.001; GE(CBL): ρ = 0.80, R2 = 0.64, p < 0.001]. In CPMS the correlation between these measures was significantly lower [GE(CBL-DMN): ρ = 0.51, R2 = 0.26, p < 0.001; GE(DMN): ρ = 0.48, R2 = 0.23, p = 0.001; GE(CBL): ρ = 0.52, R2 = 0.27, p < 0.001] and SDMT scores correlated most with BPF (ρ = 0.57, R2 = 0.33, p < 0.001). In a multivariable regression model where SDMT was the independent variable, FA-weighted GE was the only significant explanatory variable in CIMS, while in CPMS BPF and expanded disability status scale were significant. No significant correlation was found in HC between SDMT scores, MRI or network measures. DMN structural GE is related to cognitive performance in MS, and results of CBL-DMN suggest that the cerebellum structural connectivity to the DMN plays an important role in information processing speed decline.