My grandfather was a welder by trade. He told me after they had put heat to tempered steel, they cooled the part down in transmission fluid to help keep its temper.
Some of the machinist I've talked to/worked with use oil for some metals and water for others. I believe pre-heat treated steel would be fine in water but a standard mild steel would work better in oil to help keep the surface from getting brittle.
If you're serious about annealing and stress relieving it, you can get an old toaster over and heat to maximum heat, add the bolt until it gets to it's hottest temp, then lower the heat by 100 degrees every 45 minutes until it gets down to 200° or so, then take it out to cool down. Of course, this is after you've welded on it.