To study the relationships between the elastic modulus of Al-Si alloys and the content and morphology of the Si phase, hypereutectic Al-3Si, Al-5Si, Al-7Si, Al-9Si, near-eutectic Al-12.5Si and hypoeutectic Al-18Si alloys were prepared via the melting method, and the Al-9Si, Al-12.5Si and Al-18Si alloys were modified with P and Sr, respectively. The elastic modulus of the alloys was detected via the sound velocity method. The area fraction of the Si phase in the microstructures was measured via ImageJ software, and its relationship with the volume fraction of the Si phase was revealed. The functional relationships between the elastic modulus of the Al-Si alloys and the Si content and the area fraction of the Si phase in the microstructures were obtained via data fitting. The results show that with increasing Si content, the elastic modulus of the alloys increases continually. After being modified by P and Sr, the modulus of the selected Al-9Si, Al-12.5Si and Al-18Si alloys decreases. The morphological change in the Si phase leads to a decrease in the volume fraction of the Si phase, which results in a change in the elastic modulus.