Hi,
Thanks for writing. While the stems, leaves, and seeds of wisteria are poisonous to people and pets, they're harmless to most birds and insects. So you should be able to enjoy wisteria without worry --- as long as you don't have children or pets.
---Justin, Senior Garden Editor, BHG.com