I decided to ask if recruiters followed up on references you don't provide. Or more precisely, what I asked was: might you check references other than those provided by the candidate? One quarter of respondents said yes. One of those respondents explained that they do a full security check before hiring (and presumably you as a candidate would be aware of that) and another said that if they knew somebody who'd worked with the candidate before, they might call that person. Aside from the security check person, that means that nearly a quarter of these recruiters are willing to call people other than those you have provided and I start to wonder if they're necessarily going to let you know that ahead of time. I know our HR rules wouldn't allow me to do this formally and I would never take a chance of creating a grievable situation by straying from the formalities, but I still think that there's enough respondents saying this that you should be made aware it's a possibility.
So when do they check references anyhow? How soon do you need to provide a heads-up to the people on your list? Obviously, the sooner the better. You should really be asking the people you're using as references if you can do so. I've worked with one individual who has twice used me as a reference for positions and I've known nothing about it until the phone call came from the chair of the search committee. This is really NOT a good idea. I don't intentionally do it, but I do occasionally say really stupid things when caught off guard (anybody who's ever heard me do a Q&A at a conference session can attest to this) and that's probably not going to work in your favor.
On the flip side, 8% of respondents noted that they only check references when they have a concern about a candidate. I find that a little high, but maybe their other recruitment policies are so strict as to not make references matter much.
Do references matter much? Obviously for some they do, some they don't. Most of the people I've talked to regard them as being of limited utility. Depending on state labor law, the reference may not be able to provide you with anything beyond dates of employ. I think it's a fair assumption that candidates are probably only going to list references that they're sure will be positive about them so that can result in limited utility too. However, I've had references come clean about a candidate in ways I never expected and so I'm willing to give it a shot. I vividly recall speaking with a friend/colleague/candidate-approved reference quite a long time ago about a potential candidate of whom the reference said "not ready for the adult world yet." Not only "ouch," but very, very accurate as it transpired. We were pretty much aware of it already, but having a reference confirm it forced us not to ignore it as a possible misperception and also allowed us to include it in the justification to hire another candidate.
Alaska state law actually protects the supervisor as long as the candidate has signed the appropriate form and thus allows me to be completely forthright about employees who have worked for me. I'm told that's not typical and even when state law may allow, HR policies may not, so if you're concerned as to what a supervisor can or cannot say about you, a quick bit of research into state law and your institution's HR policies might be in order. That is, if you've got something to worry about whether it be something in your own record or a previous supervisor who is, shall we say, less than stable. At any rate, a bad reference can wipe you out of the pool but I don't hear many of those. Mostly I'm watching for the answers to a few specific factual questions and a different perspective on the candidate. Sometimes with otherwise equal candidates these might feed into the mix more strongly.
And do give your recruiters a little credit. They can often recognize a crazy person or somebody with vengeance issues when they talk to them. Let me tell you, over the past few years I've grown a lot less worried about the crazies I might use as references than I am about the sane-appearing ones. As both candidate and recruiter, I've found that the perfectly stable supervisors or colleagues are the ones who usually end up being loose cannons. Who--because of their good relationship with you--often relax a little too much into the reference call. I wouldn't call it common--most people understand what they should and shouldn't be saying here. As a recruiter, when I hear something off from an otherwise excellent reference, I'm going to pay far more attention to it than when I hear a bunch of things off from somebody who is clearly a loon. And even better, when the loon gives me a solid reference for the candidate, I'm likely to think "wow, this person even won over a crazy person. S/He'll do great on the ref desk!"
Back to when they call references.
20% call before they do any interviews, 25% between phone and in-person interviews, and over half will call just before they hire. Now the interesting part about that half who wait, that suggests to me that the reference check is really a pro forma thing more than something that could affect your decision-making between candidates. Should you bet on that? Probably not (that's just barely over half which does leave the remaining nearly half who don't wait.) But if you're keeping in touch with your references--like I argue you should--if you find out they've been called it can give you some idea of where you are in the process and how serious they are about you. That can be really reassuring at times. Given how long job recruitments can take, I've had a few that were suddenly resurrected for me when one of my references let me know that they'd been called: I'd pretty much written off the job since it had been so long since I'd heard anything.
On a related matter. I think it's safe to assume that unless they specifically ask for letters of recommendation, you probably shouldn't include them. I know this isn't necessarily fair: one of my good friends was jobhunting a few years back and her best reference/immediate past supervisor died shortly after she'd submitted her application for a position. If I remember right, she had a more general letter of recommendation from him and the recruiters weren't willing to accept that. But that's extreme.
The problem with letters of recommendation are all the same problems you have with any other application document: if it's not tailored to the job at hand, it's not going to do much good. I have a set of questions that I ask references and while some of them are pretty basic, at least one is not typical and given the response I get on the phone when I ask it--dead silence--I'm fairly certain that no letter of recommendation would typically include that piece of information, and I can promise you that I really do care about it. And since you're curious, I'll tell you. I ask if the candidate has ever had any disciplinary problems. This comes from an ancient place of work that had more than its share of workers who went postal. Trust me, it matters to me. Most people don't ask that one judging by the look on my HR consultant's face when I told her I wanted to ask about potential for violence. I suspect that question went all the way up to the University Counsel for approval and wordsmithing.
Last on this before we get to move on to the interviewing thing. Remember how I suggested you give your references a heads-up? I also asked to see how many recruiters cared/noticed/whatever if the reference knew about the job description when the reference call was placed. Since I do--well, not so much care as think it sends a good signal about how interested the candidate is--I was curious to see if anybody else had this preference as well. For the most part, not so much. 16% expect that your reference will be familiar with the job description, 28% say no, they don't expect that. 9% go so far as to send the reference the job description ahead of time (saving you the work). Here's the graphical interpretation for those of you who work better that way:
My conclusion? Well, since I've already revealed my own personal bias I may as well take the chance to advocate for it and suggest that you consider sharing the job description with your references before they get the phone call. Can't hurt. But if you've got limited time and ability to do all this, this is probably one of the steps you could skip and have it hurt you the least.