Experts and BHG readers answer.
Young peppers falling off
"When small pepper fruits fall off soon after the flower falls off, it may be an indication that the fruit did not get pollinated. Peppers don't necessarily require insect pollinators. It may work simply to gently shake the plant to shed pollen within the flower. You could also use a small artist's brush to poke around in the flower.
Peppers can also abort their fruits if environmental conditions are not right, either if the temperature is too cold or too hot."