Casein and α-lactalbumin (α-LA) are the main allergens in cow’s milk, which can affect the skin, respiratory system, and gastrointestinal tract, even cause anaphylactic shock. Developing a rapid and sensitive detection method of casein and α-LA is still pursued. Herein, a surface-enhanced Raman scattering based lateral flow assay (SERS-LFA) method for rapid and highly sensitive detection of milk allergens in food was established for effectively preventing allergic symptoms. Gold@silver nanoparticles (Au@Ag NPs) were synthesized as SERS active substrate to prepare the antibody-modified SERS Probe and SERS-LFA strips toward casein and α-LA were assembled according to the sandwich mode. The detection results were calculated according to colorimetric and Raman signal. The introduction of SRES signal significantly increased the sensitivity of detection with the limit of detection (LOD) of 0.19 ng/mL and 1.74 pg/mL, and exhibited an excellent linear relationship within the range of 0.55-791.50 ng/mL and 0.1 pg/mL-100.0 ng/mL for casein and α-LA, respectively. Furthermore, SERS-LFA strips was highly specific with the recovery rates for corresponding 80.36%-105.12% and 85.73%-118.22% for casein and α-LA, respectively. Therefore, the SERS-LFA could be in great potential to develop a unique allergen detection method.