Your reference is unprepared
Some references may feel like they are the victim of a surprise attack because they werent expecting a call, and therefore didnt prepare for it. They may have trouble remembering you, the exact dates of your employment, or the specifics of your job. Worse yet, they may even delete the message if they dont know who is calling.
Its important to ask if people feel comfortable serving as a reference. This gives them an out if they dont feel like they can rave about your work performance.