Beat peanut butter and shortening together in a large bowl with an electric mixer on medium speed for 30 seconds. Add baking soda, beating until blended. Beat in eggs and vanilla. Beat in oats and flour. Divide dough in half, and save one half for another use.
Press rounded tablespoonfuls into bottom and up the sides of mini muffin cups (Use the end of a mortar to press dough evenly into muffin cups.)
Bake for 9 minutes, or until edges are golden. Reform, pressing tarts with the mortar. Cool in pans. Transfer to wire racks to cool completely. Fill each tart evenly with Caramel Filling.
In a large glass measuring cup, combine caramels and water. Microwave on 50 percent power 1 to 2 minutes, or until caramels are almost melted. Stir caramel mixture with a wooden spoon until completely melted.