In 2006, the company I was working for was bought by a group of private equity firms and they decided to outsource IT. My VP kept me in the business and trained me in process improvement. For the longest time, I wondered what I could do to protect my laid-off coworkers. Let's do what I was trained to do.

The Reduce Frictional Unemployment Project

Define Phase

Problem Statement - The United States has 8.3 million unemployed people and 5.4 million job openings. If we can improve how they find each other, we can reduce unemployment.

Goal Statement - My end goal is to reduce frictional unemployment.

Scope - Focus on frictional unemployment. I think I can have more of an effect on frictional unemployment than I could on structural or cyclical unemployment.

Business Case
Logical: It is unknown how many of the 5.4 million job openings go unfilled because of frictional unemployment. It could be 5,000. It could be more. Each one matters.
Emotional: Psychologists found that job loss leads to financial stress, which leads to depression and poor health. There are 8.3 million unemployed potentially exposed to these risks.

Measure/Analyze Phase

Frictional unemployment is caused by search frictions. In awarding the 2010 Nobel Prize in Economics to Peter Diamond, Dale Mortensen, and Christopher Pissarides, the Royal Swedish Academy of Sciences described search frictions as (I added brackets for clarification):

Most real-world transactions involve various forms of impediments to trade, or “frictions”. Buyers [Employers] may have trouble finding the goods [qualified candidates] they are looking for and sellers [job seekers] may not be able to find buyers [employers] for the goods they have to offer. These frictions can take many forms and may have many sources, including worker and firm heterogeneity, imperfect information, and costs of transportation.

To get a better understanding of the search friction known as imperfect information, we need to dive into the field of Information Retrieval. Imperfect information includes:

• Misspellings
• Synonyms - Different words and phrases that share the same meaning.
• Homonyms - Matching words and phrases that have a different meaning.
• Missing information

As far as I can tell, the leading job search engines, candidate search engines, applicant tracking systems, and the government are not doing a lot to improve how the last three issues are handled. Because of this, it is likely that some hard-to-fill jobs are really just hard-to-find.

What's in it for Recruiters? Recruiters may be more focused on getting the most-qualified candidate rather than reducing unemployment. In theory, the tools and processes that make it easier for recruiters to find the best candidate will reduce search frictions and unemployment.

Improve Phase

So, I want to improve how information flows between recruiters and job seekers by improving how recruiters and job seekers use synonyms and homonyms. I decided to focus on job titles. Many search engines allow for searches to be limited to job titles. If recruiters and job seekers knew the synonyms for their job title, they could get better search results. They could see more relevant results by including synonymous job titles and see less irrelevant results by avoiding the working relationships listed in online profiles and job descriptions.

I built a thesaurus of job titles. I needed three pieces of information to build it: a list of job titles, the volume of searches for each, and the volume of usage in job openings. For the list of job titles, I found O*NET's Lay Titles file. For the volume of searches, I used Google Adwords. For the usage in job openings, I used DirectEmployer's XML Feed. Then, I spent hours and hours going through the list and grouping similar job titles together. I have to say that it is difficult to build a thesaurus, and the results are not perfect. I linked the thesaurus to Indeed's Job Trends and Google Trends to make it easy to find where phrases are being used in job titles and searches.

The next step: How can an outsider get recruiters and job seekers to use synonyms more wisely?

Attempt #1

I thought I could educate recruiters about the problem by blogging. I wrote a blog post to show that many job postings do not show up in the relevant searches because the job postings do not include the keywords from synonymous job titles. People search for "Cyber Security Jobs", but many "Information Security Analyst" job descriptions do not include the word Cyber. People search for "Internship", but many "Intern" job descriptions do not include the word Internship.

The implied call to action was to include the right keywords. The results were abysmal. There is currently less than 5 Information Security Analyst jobs that include all 8 keywords out of 900+ jobs. It is safe to say that my advice had no effect on the intern and internship job postings. The "Intern without Internship" jobs in red are missing out on a lot of searches.

Attempt #2

I thought that, if I could get my thesaurus to show up in search results on Google and Bing, the recruiters who are searching for the synonyms would get a better answer and improve their job postings. I applied some search engine optimization tactics. I built a sitemap and submitted it to Google and Bing. I received links to my site from the previous blog post, so that helped. Unfortunately, the process of building links is slow. I received 800 hits to my site in the past month. It is good, but it is not enough to really drive change. I am guessing that most recruiters do not search for synonyms that often.

Attempt #3

It seems to me that synonym problem will only be fixed if a thesaurus is built into the tools and processes that recruiters and job seekers use. A thesaurus of job titles could improve how recruiters create job postings and search for candidates. When a recruiter creates a job posting, the search engine could show the recruiter which set of synonyms the job title matches (prototype). If it does not match the job title to a synonym set, it could warn the recruiter and suggest a job title. Until the job search engines implement a thesaurus, applicant tracking systems could help the job posting show up in the relevant searches by identifying keywords that should be in a job description but are missing. When a recruiter searches for a candidate, the search engine could give the recruiter the option to automatically include all related synonyms in the results.

I would like to see Indeed and LinkedIn build a thesaurus into their systems. How can an outsider force them to implement a thesaurus?

For attempt #3, I am making it an open source thesaurus of job titles.

Indeed may resist. First, the quality of my thesaurus may not be good enough. Second, any thesaurus does not scale well across countries, because countries have different languages. Third, implementing a thesaurus would increase processing cost, since the system is looking up multiple synonyms instead of just one.

Ultimately, Indeed will implement some type of thesaurus, because an open source thesaurus of job titles represents the most significant threat to their US business. Job seekers will use the search engine that provides them with the best results. Information Retrieval states that the best search results include all the relevant items and exlude all the irrelevant items. By including synonyms, additional relevant items are returned in the results. By focusing a search on a job title, more irrelevant items are excluded from the results. Indeed has many competitors who are fighting to provide the most value to job seekers. The last thing Indeed wants is to be replaced as the industry leader by one of its competitors. Therefore, Indeed will implement a thesaurus.

Without a strong competitive threat, it is hard to tell if LinkedIn will implement a thesaurus. We will see what happens.

We put a man on the moon. We decoded the human genome. Can we make it easy for job seekers and recruiters to provide and find the right information? Some believe that it will always be difficult to find the right candidates or job openings (a realistic view). I believe it should be easy (an optimistic view).

Thank you for all the shares of my previous posts. It tells me that I'm onto something. I think that, as more people share this post, it will be more likely that the search engines and applicant tracking systems build better ways of handling synonyms into their systems.

John Carty
Follow me on Twitter @JohnPCarty