Two oligosaccharide fractions (MLO 2-1 and 2-2) were purified from enzymatic hydrolysate of mulberry leaf polysaccharide. The results of simulated digestion showed that MLO 2-2 was a digestible oligosaccharide, which could be degraded by human digestive juice; while MLO 2-1 possessed the non-digestible property in the upper gastrointestinal tract and performed the function by regulating the gut microbiota. Hence, MLO 2-1 was selected for the further analysis. The structure of MLO 2-1 was elucidated as follow: α-T-Glcp-(1→3)-β-Glcp-(1→5)-α-Araf-(1→5)-α-Araf-1→5)-α-Araf-(1→3)-α-(2-OAc)-Glcp-1. The in vitro fecal fermentation results showed that MLO 2-1 could modulate the composition of gut microbiota. Meanwhile, MLO 2-1 was effectively metabolized by fecal bacteria to produce lactate and short chain fatty acids, especially acetate and butyrate. The specific metabolic pathways of MLO 2-1 by gut microbiota were further illuminated. Gut microbiota analysis revealed that MLO 2-1 selectively promoted the growth of Ligilactobacillus murinus, a commensal bacterium presented a reduced level in T2DM mice. Animal experiments indicated that MLO 2-1 and L. murinus exhibited hypoglycemic activities. These results demonstrated that MLO 2-1 might alleviate T2DM by selectively accelerating the proliferation of L. murinus.