In a large bowl beat butter with an electric mixer on medium to high speed for 30 seconds. Add sugar, baking powder, and salt. Beat until well combined, scraping bowl occasionally. Beat in egg and vanilla until combined. Beat in as much of the flour as you can with mixer. Using a wooden spoon, stir in any remaining flour.
Divide dough into thirds. Into one-third stir in desired amount of red paste food coloring until combined; set aside. Into another third of the dough stir in desired amount of green paste food coloring. Leave the remaining third plain.
Shape each portion of dough into a 4 1/2 x 2 x 2-inch brick. Wrap bricks in plastic wrap or waxed paper. Chill at least 2 hours or until dough is firm enough to slice. With a long sharp knife, cut each brick lengthwise into four slices (You will have four pieces from each color of dough that are each 4 1/2 inches long and 2 inches wide.)
Stack six pieces of dough together, alternating colors. (You will have two bricks, each with six layers.) Gently press together dough to seal layers. If necessary, wrap bricks in plastic wrap or waxed paper and chill for 30 minutes or until dough is firm enough to slice.
Preheat oven to 350 degrees F.
Using a long sharp knife, cut each log lengthwise into eight slices (each piece will have six layers in alternating colors.) Lay slices tightly on a clean surface to make an 18x6-inch rectangle. Brush seams with water and dust dough lightly with flour. Using a rolling pin, gently roll over the dough to seal seams and make a large sheet of dough.
Use a 2 3/4-inch oval cookie cutter to cut out 16 to 20 cookies from dough. Transfer cutouts to an ungreased cookie sheet. Divide scraps of dough into 8 to 10 pieces (need half of the amount of cookies made) and roll each piece into a 6-inch long rope. Transfer ropes to another ungreased cookie sheet and form ropes into ovals about the same size as the cookies. Flatten slightly. Cut hollow ovals out of the ropes using the same oval cookie cutter.
Bake cutouts 7 minutes and ovals 6 minutes or until edges are just firm. Do not over bake. Cool on cookie sheets for 1 minute. Transfer to wire rack and cool completely.
To assemble cookies, attach a pop stick to the flat side of a cookie with royal icing. Using a pastry bag fitted with a small tip, pipe a border of icing around the edge of the flat side of the cookie and attach an oval ring creating a wall. Fill with 1/2 teaspoon candy. Pipe another border on top of the oval ring and attach another cookie, flat side down, encasing the candy inside. Repeat with remaining cookies, icing, and candy. Let stand until set.