Plants in pots have needs that contradict each other. Specifically, they want 1) a steady, copious supply of water, and 2) soil that's light and airy. With garden dirt, you can't have both things at once (unless your garden grows on almost pure sand, which is more a difficulty than a boon since it obliges you to water every day). When you saturate most kinds of dirt, you drive out the air. Without air, roots suffocate. Then, deprived of roots, the top of the plant can't get enough water (making for a strange condition in which roots drown and leaves die of thirst). As the leaves turn yellow and then wither, stems die. In the worst case, roots and top both die (as they often do in the landscape when weeks of rain keep the ground sopping wet, or put it underwater).
In a pot, garden dirt would stay continually saturated in the bottom few inches, for the same reason that a wet sponge held on end is soaking wet at the bottom and only damp at the top. Your plant would root only above the saturated zone and its growth would be restricted. On the other hand, with potting mix, even the wettest portion at the bottom of the pot has enough air for roots to grow vigorously (as you'll discover at the end of the growing season when you unpot geraniums onto the compost pile and see how thickly the roots have grown at the bottom of the root ball).