Not an insane idea at all. In fact, I have removed several chimneys for just the same reasons. From what you are telling me your have actually 5 stories of chimney at aprox. 9 feet each. The basement, 1st floor, 2nd floor, attic floor, and the story outside above the roof, So if you figure 9’x5 stories = 45 feet, times 6 x3 x2 or 810 square feet of brick surface to remove. You will need a permit, a dumpster, 2 men for a day, drop cloths, small sledge hammers, etc. You need to start at the roof and take it down from the top. Then you need to close up the roof hole and patch the roofing. When you get to the basement level there will be lots of soot in the old chimney which will have to be removed. To have this done professional would cost you between $2500 and $4000. depending on where you live.