Probably not. Carpenter ants can destroy a tree by mining out the trunk, but you'd probably know it if that's what you have: big black ants that are creating large cavities in the trunk! But it's more like you've got some sort of aphid infestation somewhere in the tree and the ants are "farming" the aphids, something that is very common. If they're not really creating any serious problems for you, I wouldn't worry about it. Though it wouldn't hurt to just glance up into the tree to make sure there's not some really serious infestation of aphids or other pest. A small infestation is no big deal, not worth dealing with.