What a great debate and enlightening responses from all and I will try and summarize:
(a) Several root causes as highlighted by Thomas, suggesting recruiters do post interview follow up to give a real answer.
(b) It helps if the recruiter is involved with interviews, identifying a tangible gap and Andrew agreed that information is the key.
(c) Providing feedback should not be difficult according to Peter, though at the final stage it may more be scope or fit.
(d) Feedback throughout the process by setting the tone and expectations from the start is endorsed by Greg.
(e) The irony of honest feedback is profiled with candor by Sandra highlighting the real risk of honest feedback being used against the recruiter.
I am not sure how many of you have been involved with any claims of discrimination during the recruitment process? All of the discrimination cases that I have been involved with has been during employment (bullying, harassment etc), as result of employee and managers lodging claims and counter claims against each other. Whilst there is always a potential risk, the way in which candidates ask for feedback is probably the best indicator of self-improving Sally, or litigious Larry wanting to know.
Many of the applicants of today could be potential hiring managers of tomorrow and are expecting to be treated with respect by giving them constructive feedback, even if the truth is coated with a healthy dose of protecting the guilty and the innocent.
Feel free to add any other responses or thoughts!