As men age, the penis may gradually shrink due to reduced blood flow, lower testosterone, and buildup of fatty deposits in arteries. Other factors like obesity, smoking, certain medications, and medical conditions can also contribute to this change, especially after age 40