You probably want low-growing plants and shrubs that will secure the soil to prevent erosion. Here are a few suggestions for your area and conditions:
Bush cinquefoil (Potentilla fruticosa)
Butterfly weed (Asclepias tuberosa)
Chinese juniper
Forsythia
Honeysuckle
Physostegia virginiana (Obedient plant)
Oxeye daisy (Leucanthemum vulgare)
Peony
Viburnum
Euonymus alatas (Winged eunonymous)
Sedum spp.
In addition, check with your local county extension service for clumping, ornamental grasses that thrive there.