It really is a buyer's market, so anyone hiring a philosopher would do well to aim for the very best person who applies, and not worry much that this person will turn them down for something better. In our job search this year I decided to gamble and recommend to the search committee that we do just this (not quite in those terms though). They took my advice (perhaps not realizing what a gamble I thought it might be--I'm the only philosopher on the search committee), and it paid off. The people we brought to campus interviewed extremely well, did a great job in the classroom, impressed everyone they met, and were eminently hire-able. What's amazing is that I don't think any of them got better offers elsewhere. Other people in my top twenty are showing up as having accepted post-docs, so it's possible that deep into my pool of best applicants for a teaching job no one has been offered a tenure-track or quasi-tenure-track position (I'm guessing that a job like this would be preferred to a post-doc, and that guess might be wrong). So that's my first bit of advice: by all means try to weed out people who don't really seem to be looking for a teaching position (although I have doubts about this--wouldn't everyone rather do a little teaching really well for more money than more teaching probably less well for less money, i.e. take a job at a research school?), but from my (limited) experience, it's a big mistake to rule out anyone on the grounds that she is simply too good.
The other decision I made fairly late in the day, and it was more like a realization than a decision, was that it mattered to me that we should hire someone interesting and not someone who would encourage ways of thinking in our students that I would want to undo. I just couldn't bring myself to recommend candidates who were too extreme, either in the continental or the mainstream analytic direction. My top candidates had research programs that were either close to mine or else diverse in a way that I found exciting. Of my top four, two have written papers that I either think I might have written (on a good day) or else wish I could have written, and two work in quite different areas than I work in, but are so creative that I could easily see areas of overlap and possible mutual inspiration. I really wish we could have hired all four.
What I discovered above all was that the whole process was much more painful than I had anticipated. A couple of years ago I went to a conference at Åbo Akademi University in Finland, and several people there commented on what a great philosophy department Lars Hertzberg has put together. Of course Hertzberg is best known and most admired for his scholarly work, but I sensed that some people felt his work in building (or helping to build--I don't know the history) a group of such intelligent, pleasant, and interesting people was almost as significant an achievement. I had this in mind when I set about looking through our applications, feeling like a kid in a candy store. One applicant prompted me to write the words "seems almost too good to be true" in my notes. Another literally made my jaw drop. A couple did work that struck me as so cool that all I could really think about it was, "Wow!" And most of these people also provided excellent student evaluations, as well as at least one letter detailing just how good their teaching is. But we could only hire one person.
156 applicants means 155 rejections. Some of the 155 don't live all that far away, and several work in areas close enough to mine that it seems inevitable I will come into contact with them again in future. Their pain is greater than mine, of course, but it's horrible having to give genuinely excellent candidates bad news. I never anticipated losing as much sleep as I have. The consolation is that we have hired one of the people I described in the paragraph above, so I can't believe our luck. It comes at a price though.
I know that some people think it's ridiculous that people like me want to hire people better than us. But of course we do! We could easily have brought half a dozen people to campus for interviews who had tremendous publication records (in any terms you like: quantity, prestige, quality, fit, ...) as well as persuasive evidence of excellent teaching, not to mention collegiality, and all the rest of it. Of course you never know how good someone really is until you meet them, see them teach, and continue to do these things over a number of years. But why anyone would decide to shortlist people with less going for them on paper is beyond me. It isn't that a school like ours needs geniuses to teach the courses we offer. It's that there is no good reason to pass over an Einstein who can teach in favor of Bill Nye or Walter White, just because you are kind of a Bill Nye school. My reference to Einstein might make it sound as though I emphasized research above teaching, but I didn't. It's hard to judge teaching ability on paper, though, so research is a useful way to differentiate between candidates who provide equally strong evidence of teaching excellence. It's also part of our job description to engage in scholarly work. For the record, the person we are hiring was described by one of my colleagues who saw his teaching demonstration as having "hit it out of the park" and by another as having done as good a job as he had ever seen in such a demonstration. The wonderful and frightening thing is that these are the people getting jobs at schools like ours, and in some cases not getting the job but coming a very close second: jaw-droppingly good researchers who teach as well as anyone you have ever seen, and who come across as (and are said by people who know them to be) good people willing to help out wherever and whenever necessary.