No fruit from mulberry trees

I live in Tampa, FL, and planted two mulberry trees about 6-8 years ago. I have never gotten any berries at all. Last year, and this year (now) I see the berries starting-all fresh and green, and within one to two days, they look like dry fuzz. I have increased the watering, but that might be the reason I even see the green fruit! My two trees have lots and lots of small upright branches, don't seem to need pruning. The leaves also start out green, but get all spotty and discolored later.