Killing them isnt the best answer. Having a bee keeper remove them is a far better option as mentioned above. But here are a few reasons.
1) the Bees have surely built a honeycomb. You need to remove this. The bees have most assuredly gone in between some joists and built a nice little honey factory.
2) If you are noticing 10 or so bees there are probably over 1 Gallon worth of bees in there. I only saw 8-12 bees at a time, the collector gathered over 3 gallons worth of bees.
3) I still get an annual spring visit from Bees looking to make the old home a new home. Even after properly removed, cleaned, caulked, and expanding foam sprayed there is a scent that attracts more bees. You need to have them removed and then properly seal off their home or you will get bee's back. (Edit: they only visit, no new hive)
4) It cost me less to have them removed than killed. And was definetly more interesting.