Some sawdust is not harmful. A lot might be. There is no doubt that it will do nice things for the soil's structure. It should make it loose, well aerated, and drain well. Woody material saps nitrogen from the soil, however. This wouldn't be a big issue with a small amount of sawdust, but could be important if you mix a lot in. Also, some sawdust can be very hydrophobic (water repellent). Again, this could be an issue with a lot of sawdust, not so much with just moderate amounts. So, my suggestion, go ahead and add it, but no more than an inch or two, till it in thoroughly, and also add in some nitrogen fertilizer at the same time. All should be fine.