This could be caused by lack of proper pollination or by fluctuating soil moisture levels, which leads to a condition called blossom end rot. Mulching around the plants may help the moisture problem. If poor pollination is the problem, you may need to hand pollinate the flowers.