Milk, as the highest quality food resource, plays an important role in improving human nutrition and health. With the improvement of residents’ living standards, consumers have put forward higher requirements for the quality of dairy products. Fermented dairy products have become an important part of the daily diet for all age groups around the world, as they are rich in nutrients such as protein, fat, sugar, and minerals, and have health benefits such as regulating intestinal balance, improving lactose intolerance, reducing inflammation in the body, protecting heart health, and unique flavors that are loved by consumers. Flavor not only affects the quality of fermented milk, but is also a key indicator for consumers in product selection. This article reviews the composition and formation pathways of flavor compounds in fermented milks and the effects of exogenous fermentation conditions on the production of aromatic compounds. In addition, it describes the research progress and future development directions on flavor improvement of fermented milk, aiming to provide theoretical basis and new ideas for the flavor, quality, and nutritional functions of fermented milk.