Table of Contents
LMIC and Vicinity Jobs have teamed up to deliver labour market information on online job postings from across Canada via our interactive dashboard, also referred to as the Canadian Online Job Posting Dashboard. You can look up the number of online job postings in near real time by geographic region and by occupation. The online job postings are collected, cleaned and structured by Vicinity Jobs to identify the work requirements detailed by employers in the job postings, including the skills, knowledge domains, tools and technologies. These data have been collected and updated monthly since January 2018.
You can find out here how to use the dashboard and what data are covered. We have answered some frequently asked question (FAQs) as well, but if you do not find the answer you are looking for, please email us at firstname.lastname@example.org.
Data and Insights
The data offer up-to-date information related to job openings posted online and their work requirements, including skills. Each month the data, collected from thousands of websites and job boards across Canada, identify new, unique online job postings. Besides being very timely, the data cover the vast majority of job openings posted online.
There are some important caveats to be mindful of when using and interpreting the data, however. For example, not all job openings are advertised online. Those advertised are usually for jobs in urban centres and in service-oriented occupations. In addition, the work requirements associated with each online job posting are based on what the employer stated in the job being advertised. This information cannot determine the relative importance of the work requirement for the job, but it can tell us how often that work requirement appears in similar job postings (e.g., those in the same city and occupation). Similarly, some work requirements for the job may not have been made explicit by the employer if they feel that “it goes without saying.”
The Labour Market Information Council (LMIC) strives to improve the timeliness, reliability and accessibility of labour market information (LMI) to facilitate decision-making by employers, workers, job seekers, academics, policy makers, educators, career practitioners, students, parents and under-represented populations. This interactive dashboard provides accessible, clear, useable information about the links between work requirements and online job postings across Canada.
Starting in 2018, LMIC conducted a series of targeted surveys asking Canadians about their labour market information needs. We found that work and skill requirements were the most sought-after type of LMI by almost every group surveyed — second only to wage and salary information.
Vicinity Jobs is a Canadian company that deals with Big Data analytics and Internet search technologies. Its mission is to deliver labour market information technology solutions. Vicinity Jobs collects and analyzes a variety of online job postings each month and uses a machine learning technique called natural language processing (NLP) to extract occupation and work requirements from each job posting.
How Should I Use This Information?
The interactive dashboard allows users to explore online job postings and work requirements from across Canada, beginning from 2019 to the present. In this way, the dashboard offers a close-up of what Canadian employers say they are looking for in potential new hires by occupation and geography.
What Can I Find on the Dashboard?
The dashboard offers two ways to view the online job posting data. In the “Job Title” view, users search by occupation. They can either perform a general search (where the job title search field is left blank) or a specific search (where a job title is entered into the search field). Users must then choose a location (e.g., province or city) and a time period (e.g., month, year, or quarter) to search by. If a general search is performed, the user will see a list of all job titles with the total number of online postings and the annual growth rate for each occupation in the selected location and time period. If a specific search was performed, the user will see only the specified occupation in the resulting list. Clicking on a job title in this list will then show the work requirements associated with it for the same location and time period.
In the “Work Requirement” view, users search by work requirement rather than by job title. Users can perform a general or a specific search by typing a work requirement into the search field. Then, after selecting the location and time period, they will be shown either a full list of work requirements or a single work requirement (depending on the type of search performed), along with the number of online job postings that specify that same work requirement and the annual change. Clicking on a work requirement from the list will then show the job titles associated with the selected work requirement for the same location and time period.
Example 1: Job Title view, general search
- Leave the Job title search field blank.
- Under Location, set Province to “Nova Scotia” and location to “Cape Breton.”
- Under Time period, set to Year to 2019 and Quarter to “Q1.”
- Click “Search.”
- The “Job Posting” table populates with two occupations: Registered nurses and Retail salespersons. We are shown that in Cape Breton during 2019 Q1, there were 90 online job postings for registered nurses and 60 online job postings for retail salespersons. Moreover, this represents a 12.5% increase from 2018 Q1 for registered nurses and a 25% decrease for retail salespersons.
- Clicking on “Registered nurses,” the second table, “Work Requirements” populates. We note, among others, that communication skills, leadership, problem solving, and teamwork are the top 4 skills listed in 53%, 35%, 34%, and 30% of all registered nurse postings in Cape Breton during 2019 Q1.
Example 2: Work Requirement view, specific search
- In the search field, enter “Microsoft PowerPoint” to conduct a specific search by work requirement.
- Under Location, set Province to “Québec” and location to “All.”
- Under Time period, set to Year to 2019 and select “Year Only” to retrieve results for the entire year.
- Click “Search.”
- The “Work Requirements” table populates with one item: Microsoft PowerPoint. We note that this requirement is classified under “Tools and Technology.” In 2019, there were 7,400 online job postings in Québec that listed Microsoft PowerPoint as a requirement. Since the data for the dashboard begin in 2019, there is no value for the annual change.
- Clicking on “Microsoft PowerPoint,” the second table, “Job Requirements” populates. We note, among others, that job postings for Administrative assistants (7,700), Administrative officers (3,300), Professional occupations in advertising, marketing, and public relations (2,700), and Corporate sales managers (2,400) listed “Microsoft PowerPoint” as a work requirement more frequently relative to other occupations for Québec in 2019.
Note that on the dashboard you can also select the level of occupational detail – the “NOC Level”. Canada’s National Occupational Classification (NOC) system consists of a 4-level hierarchy, with the greatest detail available for the 500 “unit group” occupations (4-digit NOC). Further details about the NOC are available in our FAQ.
What Else Should I Know?
Remember, data always has some important caveats and limitations including, but not limited to the following:
- Online job postings are related to, but distinct from, job vacancies. For example, employers (such as governments) may search for staff before there is a formal job opening to fill in order to create a “pool” of applicants from which to hire. Alternatively, many open jobs are never posted in the first place; rather, they are filled internally or by word of mouth recruitment.
- The list of work requirements shown does not indicate how important the requirement is for that occupation. Sometimes employers list “nice to have but not essential” qualifications without specifying them as such. As well, sometimes a requirement falls into the “goes without saying” category and is therefore not listed.
See our Caveats and Limitations section for an elaboration of these and additional considerations that are merited in using this dashboard.
How Do I Use the Dashboard?
- Select “Occupation” or “Work Requirements” tab to choose how the online job postings are to be grouped
- Select the geography of interest (e.g. province, territory and/or city) using the dropdown menu.
- Select the time period of interest using the dropdown menu (e.g. year, quarter or month).
A list of online job postings organized by occupation or work requirements (depending on the tab selected in step 1), ranked by the number of postings in the geography and time period chosen, will populate the search window. You can then use the “job category”1or “work requirement category”2 dropdown menu or search function to refine the list.
- Click on an occupation or work requirement in the first table to display the work requirements or occupations most frequently associated with your selection for that geography and time period.
What Information is Available?
The interactive dashboard allows users to explore timely, granular labour market information related to online job postings. Within any time period (starting in January 2019 and geographic region, the user can explore the type and frequency of work requirements (details that employers include within each job posting). Users can view the range of online job postings by occupation. Within each occupation, the percentage of online job postings linked to specific work requirements is also included.
LMIC classifies these work requirements into four categories based on the Skills and Competencies Taxonomy from Employment and Social Development Canada (ESDC): Skills, Knowledge, Tools and Technology, and Other. Note that ESDC’s taxonomy encompasses seven categories of skills and competencies. We have collapsed Interests, Personal Abilities and Attributes, Work Activities and Work Context into the “Other” group.
How is the Information Collected?
Online job posting data are provided by Vicinity Jobs, a Canadian company that deals with Big Data analytics and Internet search technologies. They collect and analyze job postings found on various websites and job boards and link each posting to a unique occupation and set of work requirements. These work requirements — over 40,000 are possible, but only 2,500 appear regularly — are defined by Vicinity’s proprietary taxonomy for categorizing free text descriptions in online job ads.
The dashboard data begins in January 2019 and runs to the present. This near-real time information on posted jobs and their work requirements is available by detailed occupation (4-digit NOC) as well as more aggregated occupational groups; month, quarter or year; and job location. Currently, there are 78 sub-provincial or sub-territorial locations available, which are based on Statistics Canada’s Economic Regions with some exceptions.
Caveats and Limitations
The data shown on our dashboard, drawn from Vicinity Jobs, offers an expansive set of near-real time information on online job postings and their associated work requirements. As with any dataset, it is important to bear in mind the caveats and limitations associated with the information. Broadly, there are two types of closely connected caveats and limitations: 1) data interpretation and 2) data collection. For further discussion of how online job postings can complement official labour market statistics, see our joint report with ESDC and Statistics Canada here.
Job postings vs job vacancies
The dashboard presents a sample of online job postings provided by Vicinity Jobs. While this sample is large, by definition, it is only a subset of all job postings. Even though many employers actively recruit online, job postings do not precisely represent job vacancies.
Job vacancies refer to the number of available job openings that an employer wants to fill. The primary data source for measuring vacancies in Canada is the Job Vacancy and Wage Survey (JVWS) for which a vacancy is defined as a job that is or will become vacant during the month, and for which the employer is actively recruiting outside the organization.
Even though many employers actively recruit via online job postings, the two concepts are distinct. First, not all vacancies are posted online. Second, a count of job postings (online and offline) may underestimate the number of actual vacancies because employers may seek to fill multiple vacancies via a single job posting. Third, counting job postings may overestimate vacancies if, for example, the employer maintains a job posting that they are not currently seeking to fill (in which case the posting does not technically represent a vacancy as defined in the JVWS). In general, it should be expected that the number of job postings in the dashboard versus a complete count of job vacancies across Canada would differ.
Gross vs net changes in employment demand
When evaluating a growing sector or occupation, one is (often implicitly) thinking about the net change in employment demand. For example, “How many new web design jobs will there be this year?” is a question about net employment changes. Conversely, gross changes in employment demand include these new jobs plus turnover (i.e., where the previous person left the position).
The distinction matters because online job postings can only be used, with some caution, as proxies for gross changes in employment demand — not net. With online job postings, there is no way to know if the position results from the organization’s growth or to fill an existing position left vacant. As such, the economic health of different occupations should not be estimated simply from the growth in the number of online job postings — growth here might reflect either economic dynamism or particularly high turnover.
Work requirement frequencies do not necessarily indicate their importance
Every month, Vicinity Jobs collects, cleans and structures hundreds of thousands of online job postings from across Canada, extracting details such as occupation, location and work requirements. A key benefit is that the work requirements extracted from online job postings reflect the language used by employers.
This real-world use of language, however, should be interpreted with caution. First, there is no guarantee that employers explicitly state all work requirements in job postings. In many cases, they assume that certain requirements are implicitly obvious to prospective job candidates. Second, there is no way to tell which work requirements are critical for the position, or the proficiency that is needed to perform the job successfully. The data allow us only to observe that certain requirements are more frequently stated by employers across online job postings.
For example, Microsoft Excel was associated with 22% of online job postings for “Economists and economic policy researchers and analysts” (NOC 4162) in 2019. Does this mean that the other 78% do not need Excel? Probably not. Even for the 22% of postings mentioning Excel, there is no information about the level of expertise required by the employer.
Work requirements and skills
It is common to talk about the “skills” identified in online job posting data. However, the word “skills” does not refer to all work requirements; it has a specific definition. The data in the dashboard is organized into four work requirement categories based on ESDC’s Skills and Competencies Taxonomy:
- Skills: The developed capacities that an individual must have to be effective in a job, role, function, task or duty.
- Knowledge: The organized sets of information used to execute tasks and activities within a particular domain.
- Tools and technology: The categories of tools and technology used to perform tasks.
- Other: The work requirements not captured in the other three categories: Work Activities, Work Context, Interests, and Personal Abilities and Attributes.
The work requirements of any particular position is a combination of skills (e.g. critical thinking, or problem solving), knowledge of a subject area (e.g., budgeting, or electrical systems) and use of a particular tool or technology (e.g., Excel or diodes).
Given the nature of the inter-connected, yet distinct, categories of work requirements, care should be exercised when extrapolating the online job posting data in the dashboard to draw conclusions on which skills are required by employers or what is lacking in the job market.
In processing the content of online job postings, the data may be skewed towards certain industries, occupations, regions, firm sizes, and education level requirements. While Vicinity Jobs strives to capture all verifiable online job postings, this cannot be guaranteed.
Hidden job market
Many employers hire internally or through informal means such as word of mouth. These sources of employment demand cannot be captured from online job posting data. They also may not be included in vacancy survey data.
Data are collected from online job postings via scanning algorithms that seek to deliver a comprehensive set of job postings information. Organizations that collect data in this manner typically develop proprietary algorithms to clean and structure raw data. The information collected, therefore, is structured differently across different providers, even though they acquire the data from, essentially, the same set of online jobs postings. Despite these differences, quality assurance remains an essential part of the process. To this end, Vicinity Jobs frequently tests and revises its algorithms for collecting, cleaning and structuring the raw data, based both on internal quality assurance checks and ongoing feedback from partners, including LMIC.
Duplicate job postings
One of the biggest data-quality issues associated with postings pulled from online sources is that many employers post the same job on many different websites — in fact, an estimated 80% are duplicates. To avoid double (or multiple) counting of job postings, de-duplication — removing job postings that appear on multiple websites — is essential. The process, however, is not perfect. Vicinity Jobs removes an estimated 95% of duplicate job postings each month. But small differences in the same posting, including website layout, means that a few are still missed.
For most objectives, the primary data of interest when collecting and analyzing raw text from online job postings are work requirements and job titles. To that end, the resulting data should link job postings to an occupation (or job title) and to a set of work requirements. The process for doing so can be divided into four broad steps: 1) collecting, 2) cleaning, 3) structuring, and 4) extracting.
Although the four steps proceed in sequence, refining and optimizing each step often requires reviewing the results of one part while iteratively testing another part. For example, Steps 3 and 4 (structuring and extracting) may produce valuable information that can be used to increase the accuracy of de-duplication algorithms used in Step 2 (cleaning).
The four-step method to transform online job postings into useable labour market information and insights are described below.
Step 1: Data Collection
Data collection refers to the process of acquiring the text from online job postings, such as those available on job boards, like Service Canada’s Job Bank, job aggregator websites or directly from corporate websites. Data collection typically involves a variety of approaches, including web scraping (the process of downloading raw text from websites) and accessing website content through the host’s dedicated API connection. In all cases, the data collected are publicly and freely available. The technique often employed for this purpose is called Document Object Model (DOM) parsing, which allows a user to extract portions of a web page based on its underlying HTML structure. Specifically, data collection programs can retrieve the dynamic content of a web page by referencing, for example, the Cascading Style Sheet (CSS) selectors associated with specific parts of the page.
Importantly, quality assurance protocols are implemented even in this initial step through the selective identification of which websites should be monitored. Since websites vary widely in terms of the reliability of jobs posted (e.g., a corporate job board versus jobs posted on Kijiji), Vicinity Jobs identifies their job posting sources through careful review and vetting prior to starting the data collection process.
Step 2: Data Cleaning
After the raw text is downloaded, it must be processed into a form conducive for further statistical analysis. This process is known as “feature extraction.” With respect to text data, this processing includes tasks such as simplifying punctuation, converting words from plural to singular, replacing abbreviations, and collapsing to lower case letters.
Another phase of the cleaning process focuses on removing duplicate or fake job posts. Approximately 30% of web pages are near-duplicates (Henzinger, 2006) and, in the case of online job postings, companies often advertise their jobs on multiple sites, increasing the estimated rate of duplicate postings to as high as 80% (Jijkoun, n.d.). Duplicated content can skew the results when trying to identify the most frequently demanded work requirements or estimate the number of job postings in the labour market.
De-duplication — the process of removing duplicate job postings — is achieved by using a natural language processing algorithm to train statistical models, known as “classifiers,” to predict whether two postings refer to the same position. With respect to Vicinity Jobs’ data, approximately half of all raw job postings are identified as duplications and are removed.
Step 3: Data Structuring
Once data from the job postings have been collected and cleaned, they are ready to be structured and organized into a specific set of classifications or categories.
One particularly important exercise is the classification of job postings into a standard taxonomy of occupations, such as the Canadian National Occupational Classification (NOC) or the American Standard Occupational Classification (SOC) systems. Most algorithms that create this mapping between raw text and occupational classifications do so via examining the job title in the posting. This approach is limited due to the high variability of job titles used by employers. For instance, a “Programmer” might refer to NOC category 2173 “Software engineers and designers” or to 2174 “Computer programmers and interactive media developers,” depending on the actual requirements of the position. Similarly, NOC categories might be assigned differently based on industry. For example, a sales representative job would be allocated to different 4-digit NOC codes depending on if the job is in the financial services, retail or wholesale industry. More advanced approaches use both the job title and job description to inform the classification process.
One method for mapping job titles to occupations requires a reference list of known job titles and a corpus of terms used to describe occupations. One such resource is the UK Office for National Statistics (ONS) 2010 index, which identifies nearly 30,000 alternative job titles across all occupational unit groups. Similarly, one could use the NOC profiles, which list example titles for each occupation. These reference lists are used to further clean the job title text by removing words in the posting title that do not exist within the list of known titles. Afterwards, these job titles are matched to a standard taxonomy of occupations.
A more recent approach is to use machine learning techniques to build a text classifier for associating job postings with occupational codes (i.e., NOC). After cleaning the titles as described above, the text is converted into vectors of numbers (a process called word embedding). One of the simplest and most flexible models for word embedding is the Bag-of-Words (BoW) model, which relies on a method called one-hot encoding to transform the text into mathematical vectors. Once the text has been converted into vector representations, a supervised learning model is implemented. This requires a set of data (online job posting titles and descriptions) for which the occupational classification is already known. In the case of the European WoLMIS system, a team of experts selects websites from which postings are downloaded weekly over several months and then manually labelled according to the European occupational classification system. The labelled job postings are then trained on the data using an SVM Linear classifier. Once the classifier is built and its effectiveness measured, it can be used on the collected data to assign occupational codes.
Another objective of using data from online job postings is to draw insights on the work requirements (e.g., skills) demanded by employers. This often requires structuring the raw job description text into a more meaningful format. As with the classification of postings into occupational units, the raw skills text can be classified into to a predefined taxonomy.
Step 4: Data Extracting
The final step in the process is to extract insights from the resulting structured data. With respect to work requirements, this may take the form of calculating the most frequently requested work requirements by occupation and geography, for example. In addition, since “work requirements” is a broad category, it is informative to further organize these into a hierarchical classification. To that end, we categorize work requirements into four categories based on ESDC’s Skills and Competencies Taxonomy: skills, knowledge, tools and technology, and other. Their full taxonomy includes seven categories, but we collapsed interests, personal abilities and attributes, work activities, and work context into “other.”
Vicinity Jobs’ Approach for Linking Job Postings to NOC
Vicinity Jobs uses an iterative process that first attempts to match job postings to NOC based on their job titles, then further refines the allocation using additional anchors from the content of each job posting and known information about the employer (e.g., industry). The process relies on Vicinity Jobs’ proprietary knowledge base that encapsulates tens of thousands of job titles. This knowledge base was originally compiled from available NOC specification sources (e.g., ESDC’s occupational profiles), then refined using historic job postings. Further refinements in linking the job posting to a NOC are then made by incorporating contextual knowledge (e.g., certain job titles can only be allocated to certain occupations in certain specific contexts).
To identify the appropriate contexts, Vicinity Jobs’ algorithm attempts to allocate postings to industries. It does so, in part, by matching employer data (such as employer name and URL) to known employer profiles stored in a vast, up-to-date database of known Canadian employers.
After postings have been organized into 4-digit NOC codes, a set of unassigned job postings remains whose content and job titles are not sufficiently specific, or the nature of the jobs does not allow for an allocation to a single 4-digit NOC. Instead of forcing or guessing, the algorithm attempts to assign as many of them as possible to the less specific broad occupational category (1-digit NOC).
Vicinity Jobs’ Work Requirements Taxonomy
To ensure that Vicinity Jobs’ work requirements and certifications reporting meets their quality standards while remaining representative of the unique aspects of Canadian labour markets, they have established a taxonomy for work requirements (including skills) that combines extensive input from their clients and partners with information from publicly available sources such as the O*NET taxonomy of technologies and tools, structured skills keywords published by certain websites, and other work requirements data assembled from Canadian online job postings and research.
After compiling an initial version of their work requirements taxonomy, Vicinity Jobs uses a proprietary algorithm to enrich and refine their knowledge about work requirements and the linguistic constructs that represent them. This process involves a sequence of iterative steps, where each iteration includes testing the knowledge base against a broad set of non-structured job postings content to identify work requirements, then manually review the resulting data and provide feedback to fine-tune the algorithms.
The resulting knowledge base encompasses tens of thousands of work requirements and certifications and includes not only labels and identifying keyword combinations, but also a complex framework of underlying interdependencies and context definitions needed to ensure accurate identification of work requirements. For example, the system knows that certain work requirements and certifications could apply to any job, while others are only relevant to specific occupations.
LMIC’s Categorization of Skills and Other Work Requirements
Vicinity Jobs’ data provides a list of work requirements extracted from online job postings that have been cleaned according to a proprietary list of pre-validated requirements. In the LMIC dashboard, we further refine this list to distinguish skills from other work requirements. This is important because there is a long-standing ambiguity over the notion of skills and how they are defined. Employment and Social Development Canada’s Skills and Competencies Taxonomy groups job/worker characteristics into seven domains: 1) skills, 2) personal abilities and attributes, 3) knowledge, 4) interests, 5) work context, 6) work activities, and 7) tools and technology. With these domains serving as a reference, we reclassified Vicinity Jobs’ skills into four distinct groups: 1) skills, 2) knowledge, 3) tools and technology, and 4) other. Table 1 shows examples of the differences among the four categories.
Table 1. Examples of Work Requirements Classified Into LMIC’s Four Domains
|Skills||Knowledge||Tools and Technology||Other|
|1. Ability to learn
2. Communication skills
3. Critical thinking
|1. 5S Methodology
2. Dutch Language
3. Digital Marketing
|1. Fertilizer spreaders
2. Fetal monitors
3. Google Docs
2. Public Relations
3. Project Management
Henzinger, M. (2006). Finding near-duplicate web pages: A large-scale evaluation of algorithms. Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’06), 6–11 August 2006, Seattle, WA, USA (pp. 284–291). New York: ACM. doi:10.1145/1148170.1148222
Jijkoun, V. (n.d.). Online job postings have many duplicates. But how can you detect them if they are not exact copies of each other? Amsterdam, The Netherlands: Textkernel.
- There are 10 job categories each associated with the first number in an occupation’s 4-digit code.
- There are 4 work requirement categories which are based on ESDC’s Skills and Competencies Taxonomy. They are “Skills”, “Knowledge”, “Tools and Technology” and “Other”.