The recruitment industry has faced a variety of trends over the last decade. The rise of social media and the economy of ideas has led to a market that is far more candidate driven. Due to sites such as LinkedIn and others, it is now much easier for candidates to approach the decision makers in small to mid-level companies directly rather than going through a recruiter. Even larger companies have begun posting jobs directly onto third party websites in an effort to attract more diverse candidates.

Due to these trends, recruiters are finding it harder to identify positions they can recruit for before candidates have made the first steps. The same information on job roles is available for both candidates and recruiters on job aggregator sites like indeed.com, which means that the value to a candidate of a recruiter faces a diminishing return. As companies explore more and more avenues to gain direct solicitations from candidates and attract the long tail, the ability for a recruiter to get in there quickly to suggest quality people lowers. In order to remain competitive, recruiters need to stay one step ahead of candidates and become more data focused.

Why the existing system isn’t working

Web data in recruitment has been dominated over the past few years by the rise of job aggregators. Job aggregators (Indeed, SimplyHired, Glassdoor, reed.co.uk, Jobster, etc) scrape jobs from both careers pages as well as recruitment agency portals and job boards. These aggregators have become so successful because they have mastered the art of search engine optimisation (SEO). By containing many more links and keywords than other jobs sites, they make themselves an attractive golden source for candidates to use that appear highest on the common search engines like Google.

Unfortunately, as advertising revenue drives job aggregator top-lines, having a large volume of jobs and thus chances for PPC clicks is more beneficial to aggregators than providing clean, structured, organised information.

This has created a number of issues…

• Clutter - Job listings are often just copied from other sources so the aggregator sites are full of duplicate listings.

• Incompleteness - Most employers only post certain jobs to paid job boards, which means not all of their openings are listed on the aggregator sites.

• Expired listings - Most job aggregators don’t work directly with companies and so cannot know when a job has already been filled or should no longer be advertised.

• Lack of consistent structure - Job aggregators are often organized based purely on keywords, making searches difficult for recruiters/candidates who need to know all the different ways that employers class the same job title.

• Listings that link to other listings - Often job boards link to other job boards and listings creating a long chain of hyperlinks for a candidate to actually apply for a job.

• Front sites - Many aggregator sites have started parsing the jobs they copy into disciplines and then creating job boards for those. This then gives the illusion of scale and the linking to separate, specialised job boards when they were actually obtained from the same place.

In addition to having to sort through the deluge of jobs (which may or may not be legitimate) on aggregator sites, recruiters also need to:

1. Pay to post to a job board

2. Pay an aggregator via PPC bidding to post listings

3. Pay to access a resume database of candidates, only available in the first place because recruiters collectively provided them.

Bizarrely, if a recruitment company pays to list a position on a job board, and then bids on an aggregator site, it could be competing against itself directly.

For these reasons and many others, it is clear that the recruitment industry needs a radical shift in thinking. If recruiters had better, more structured access to jobs data than candidates, they would have a distinct advantage in finding desirable open positions. Better information in this space would mean more valuable insight, more unique output and increased speed of execution.

Data is the answer

Existing job listings on the web are all over the place - duplicated, unclean, expired and many more. There is no common language, no structure, and no way to systematically process and analyse job postings clearly to build unique insight.

Solutions need to focus on cleaning, de-duplicating and analysing the job postings available on the web. By treating these job postings as data, recruiters can start to build a better, more efficient system for generating leads.

Lean data in a regular, consistent format that includes job descriptions, employment types, job requirements, company descriptions, posted dates, min/max salaries is useful because it allows the development of semantic relational databases. Relational databases can be easily filtered, analysed and searched to identify trends and find jobs within given parameters - thus solving the inconsistent labeling issue.

There is a simple first step. Using the same basic web scraping technology of sites like indeed, recruiters can access high quality job postings direct from the source - the company careers pages themselves. Companies tend not to keep expired job postings on their website because it slows their HR departments down and ultimately creates real costs from having to screen candidates applying for an irrelevant role. Thus, when a job listing is on a careers page, you can generally assume it was intended to be there, that it is timely and that the position is still open. Secondly, due to the resurgence of applicant tracking systems which naturally structure job positions for companies on their careers pages, the likelihood of organised elements and nodes on careers pages is higher.

Data resources for recruiters

Recently, a few startups (such as data platform import.io) have begun providing recruiters with just such a solution. They create live APIs to company careers pages, which allows them to pull the job listings into a structured database in real time. These jobs are then indexed and placed into a comprehensive searchable dashboard, which recruiters can subscribe to. The corresponding user interface allows a recruiter to quickly track jobs from relevant companies. If jobs are listed on the website, a recruiter can be notified within the hour and make that all-important sales call to kick off discussions.

In bigger picture terms, having a real time web feed of job listings and historical dataset allows analysts and recruiters to understand how a company strategy is evolving, live, and indeed how the nature of business works. For example, if a recruiter knows that a company has been recently hiring a lot of personal assistants, it may be a sign that they are in a time of additional cash. If the company has recently been looking for more senior managers they may be able to guess that there have been some senior departures, or that it’s are also looking to expand.

In summary, for a recruiter, having a clean, live, searchable database of job listings offers a value add well above a normal job aggregator. A structured web database contains structured, clean, deduplicated, unique information that can be analysed to derive useful insights to make that recruiting conversation that much more informed, and data driven.