In a large mixing bowl beat butter with an electric mixer on medium to high speed for 30 seconds. Add granulated sugar, baking powder, and salt. Beat until combined, scraping sides of bowl occasionally. Beat in egg, milk, and vanilla until combined. Beat in as much of the flour as you can with the mixer. Using a wooden spoon, stir in any remaining flour. Divide dough in half. Cover and chill about 2 hours or until dough is easy to handle.
Preheat oven to 375°F. On a lightly floured surface, roll one dough portion at a time to 1/8 to 1/4 inch thick. Using 1-inch and 3- to 3-1/2-inch heart-shape cookie cutters, cut six small hearts and one large heart from dough for each reindeer. Place large hearts for faces 1 inch apart on an ungreased cookie sheet. Overlapping three small hearts for each antler, attach two antlers to top of each face.
Bake for 7 to 10 minutes or until edges are very light brown. Transfer cookies to a wire rack, taking care not to break antlers; cool.
Outline and flood reindeer faces with glaze-consistency Royal Icing tinted light brown. Outline each antler heart with glaze-consistency white Royal Icing. While icing is still wet, use a small round tip to pipe glaze-consistency light brown icing inside white outline. Cut small white gumdrops into slices. While icing on each face is still wet, press in two gumdrop slices for eyes and a red candy-coated chocolate piece for a nose. Use icing to attach miniature brown candy-coated chocolate pieces to gumdrop eyes for pupils.
Layer cookies between sheets of waxed paper in an airtight container; cover. Store at room temperature for up to 2 days or freeze for up to 3 months.
In a large mixing bowl stir together the powdered sugar, meringue powder, and cream of tartar. Add the water and vanilla. Beat with an electric mixer on low speed until combined; beat on high speed for 7 to 10 minutes or until icing is very stiff. If not using right away, cover bowl with a damp paper towel and cover paper towel with plastic wrap; chill for up to 48 hours.
Meringue powder is a mixture of pasteurized dried egg whites, sugar, and edible gums. Look for it in the baking aisle of your supermarket or at a specialty food store.