Preheat the oven to 350°F. Grease a muffin tin, and use muffin papers if desired.
Using a mixer, cream the sugar and butter together.
When light and fluffy, add the lemon extract, vanilla extract, and eggs, and beat together until well blended.
Add the lemon zest, sour cream, and poppy seeds, and mix until all ingredients are just blended in.
In a separate bowl combine the flour, baking powder, and salt. Stir until the mixture is uniform.
To combine the dry and wet ingredients, add about a third of the flour mixture and a third of the milk to the creamed butter and stir gently. Then add the next third of the flour and milk, and stir gently. Scrape the sides of the bowl to make sure all ingredients are well incorporated. Add the remaining flour and milk. Stir until just blended.
Fill each muffin tin 2/3 full.
Bake for 24 minutes. You may want to start checking the muffins at about 20 minutes by inserting a toothpick to see if they are done. When the toothpick comes out clean, remove the muffins from the oven.
Combine the powdered sugar, milk, lemon extract, and lemon zest in a small bowl. Stir until well blended.
When the muffins have cooled to room temperature, drizzle the glaze over the muffins.