Frictional Unemployment and the Death of Boolean

In 2006, the company I was working for was bought by a group of private equity firms and they decided to outsource IT. My VP kept me in the business and trained me in process improvement. For the longest time, I wondered what I could do to protect my laid-off coworkers. Let's do what I was trained to do.

The Reduce Frictional Unemployment Project

Define Phase

Problem Statement - The United States has 8.3 million unemployed people and 5.4 million job openings. If we can improve how they find each other, we can reduce unemployment.

Goal Statement - My end goal is to reduce frictional unemployment.

Scope - Focus on frictional unemployment. I think I can have more of an effect on frictional unemployment than I could on structural or cyclical unemployment.

Business Case
Logical: It is unknown how many of the 5.4 million job openings go unfilled because of frictional unemployment. It could be 5,000. It could be more. Each one matters.
Emotional: Psychologists found that job loss leads to financial stress, which leads to depression and poor health. There are 8.3 million unemployed potentially exposed to these risks.

Measure/Analyze Phase

Frictional unemployment is caused by search frictions. In awarding the 2010 Nobel Prize in Economics to Peter Diamond, Dale Mortensen, and Christopher Pissarides, the Royal Swedish Academy of Sciences described search frictions as (I added brackets for clarification):

Most real-world transactions involve various forms of impediments to trade, or “frictions”. Buyers [Employers] may have trouble finding the goods [qualified candidates] they are looking for and sellers [job seekers] may not be able to find buyers [employers] for the goods they have to offer. These frictions can take many forms and may have many sources, including worker and firm heterogeneity, imperfect information, and costs of transportation.

To get a better understanding of the search friction known as imperfect information, we need to dive into the field of Information Retrieval. Imperfect information includes:

  • Misspellings
  • Synonyms - Different words and phrases that share the same meaning.
  • Homonyms - Matching words and phrases that have a different meaning.
  • Missing information

As far as I can tell, the leading job search engines, candidate search engines, applicant tracking systems, and the government are not doing a lot to improve how the last three issues are handled. Because of this, it is likely that some hard-to-fill jobs are really just hard-to-find.

What's in it for Recruiters? Recruiters may be more focused on getting the most-qualified candidate rather than reducing unemployment. In theory, the tools and processes that make it easier for recruiters to find the best candidate will reduce search frictions and unemployment.

Improve Phase

So, I want to improve how information flows between recruiters and job seekers by improving how recruiters and job seekers use synonyms and homonyms. I decided to focus on job titles. Many search engines allow for searches to be limited to job titles. If recruiters and job seekers knew the synonyms for their job title, they could get better search results. They could see more relevant results by including synonymous job titles and see less irrelevant results by avoiding the working relationships listed in online profiles and job descriptions.

I built a thesaurus of job titles. I needed three pieces of information to build it: a list of job titles, the volume of searches for each, and the volume of usage in job openings. For the list of job titles, I found O*NET's Lay Titles file. For the volume of searches, I used Google Adwords. For the usage in job openings, I used DirectEmployer's XML Feed. Then, I spent hours and hours going through the list and grouping similar job titles together. I have to say that it is difficult to build a thesaurus, and the results are not perfect. I linked the thesaurus to Indeed's Job Trends and Google Trends to make it easy to find where phrases are being used in job titles and searches.

The next step: How can an outsider get recruiters and job seekers to use synonyms more wisely?

Attempt #1

I thought I could educate recruiters about the problem by blogging. I wrote a blog post to show that many job postings do not show up in the relevant searches because the job postings do not include the keywords from synonymous job titles. People search for "Cyber Security Jobs", but many "Information Security Analyst" job descriptions do not include the word Cyber. People search for "Internship", but many "Intern" job descriptions do not include the word Internship.

The implied call to action was to include the right keywords. The results were abysmal. There is currently less than 5 Information Security Analyst jobs that include all 8 keywords out of 900+ jobs. It is safe to say that my advice had no effect on the intern and internship job postings. The "Intern without Internship" jobs in red are missing out on a lot of searches.

Attempt #2

I thought that, if I could get my thesaurus to show up in search results on Google and Bing, the recruiters who are searching for the synonyms would get a better answer and improve their job postings. I applied some search engine optimization tactics. I built a sitemap and submitted it to Google and Bing. I received links to my site from the previous blog post, so that helped. Unfortunately, the process of building links is slow. I received 800 hits to my site in the past month. It is good, but it is not enough to really drive change. I am guessing that most recruiters do not search for synonyms that often.

Attempt #3

It seems to me that synonym problem will only be fixed if a thesaurus is built into the tools and processes that recruiters and job seekers use. A thesaurus of job titles could improve how recruiters create job postings and search for candidates. When a recruiter creates a job posting, the search engine could show the recruiter which set of synonyms the job title matches (prototype). If it does not match the job title to a synonym set, it could warn the recruiter and suggest a job title. Until the job search engines implement a thesaurus, applicant tracking systems could help the job posting show up in the relevant searches by identifying keywords that should be in a job description but are missing. When a recruiter searches for a candidate, the search engine could give the recruiter the option to automatically include all related synonyms in the results.

I would like to see Indeed and LinkedIn build a thesaurus into their systems. How can an outsider force them to implement a thesaurus?

For attempt #3, I am making it an open source thesaurus of job titles.

Indeed may resist. First, the quality of my thesaurus may not be good enough. Second, any thesaurus does not scale well across countries, because countries have different languages. Third, implementing a thesaurus would increase processing cost, since the system is looking up multiple synonyms instead of just one.

Ultimately, Indeed will implement some type of thesaurus, because an open source thesaurus of job titles represents the most significant threat to their US business. Job seekers will use the search engine that provides them with the best results. Information Retrieval states that the best search results include all the relevant items and exlude all the irrelevant items. By including synonyms, additional relevant items are returned in the results. By focusing a search on a job title, more irrelevant items are excluded from the results. Indeed has many competitors who are fighting to provide the most value to job seekers. The last thing Indeed wants is to be replaced as the industry leader by one of its competitors. Therefore, Indeed will implement a thesaurus.

Without a strong competitive threat, it is hard to tell if LinkedIn will implement a thesaurus. We will see what happens.

We put a man on the moon. We decoded the human genome. Can we make it easy for job seekers and recruiters to provide and find the right information? Some believe that it will always be difficult to find the right candidates or job openings (a realistic view). I believe it should be easy (an optimistic view).

Thank you for all the shares of my previous posts. It tells me that I'm onto something. I think that, as more people share this post, it will be more likely that the search engines and applicant tracking systems build better ways of handling synonyms into their systems.

John Carty
Follow me on Twitter @JohnPCarty

Views: 424

Comment by Nicholas Meyler on July 31, 2015 at 12:10am

I would be inclined to think of the necessity to expand the search parameters to permutations and combinations of various key-words or 'buzzwords' (and. of course, their synonyms, which makes a 'Thesaurus Algorithm' approach possible).  Job titles, in and of themselves, are often trivial and nearly meaningless, I have found.

I think it would be more Utilitarian and Pragmatic to focus on identifying the key requirements of the position itself, and then using a 'Thesaurus Algorithm' to identify possible candidates.  This is what the intelligent recruiter needs to do, on a daily basis, and is what makes the job so much fun and so appealing -- it gives us a great opportunity to 'think outside the box'.

What scares me, though, is the idea of software replacing my creative or analytical thought processes.  This whole process might result in the elimination of the role of the recruiter altogether.  

Taking the example of chess, it is still a game played by humans, with a great deal of respect accorded to the World Champion and top players, although computer programs can beat any human...  Would the same hold true for recruiting and executive search, though?

Very well-written and thoughtful article.  Thank you for sharing.

Comment by John P Carty on July 31, 2015 at 11:22am

Nicholas - Thanks for the comment!

You mention that job titles are "nearly meaningless." I agree that some job titles are meaningless. It is certainly a messy process trying to group job titles together. Many job titles don't have clear operational definitions and there is overlap across the detailed occupations. In those situations, it is probably best to search for the key requirements.

Having said that, when a recruiter or job seeker is searching by using a job title, I think the search engines should give the recruiter or job seeker the option to automatically include common synonyms in the results.

Also, I doubt technology will replace recruiters. HR Bartender's article "Will Technology Replace Recruiters" goes more in depth into that issue. Glen Cathey's blog (which I allude to in this post's title) lays out the case against semantic search solutions.  If you read his "Search: Beyond Boolean" article, you will notice that my thesaurus is focused on the simplest type of search and it would be difficult to include the higher levels, 


You need to be a member of RecruitingBlogs to add comments!

Join RecruitingBlogs


All the recruiting news you see here, delivered straight to your inbox.

Just enter your e-mail address below


RecruitingBlogs on Twitter

© 2020   All Rights Reserved   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service