Mirage or Reality? The Limitations of Online Job Postings
Introduction
Online job postings (OJP) are everywhere—on LinkedIn, platforms like Indeed, and company websites. But as more researchers, policymakers, and educators turn to OJP for real-time insights, one important question remains: How reliable are they as a signal of actual labour demand?
OJP data are widely used to fill in very real gaps in Statistics Canada labour market data, especially where granular or timely information is missing. But like any emerging data source, OJP data have limitations when it comes to representativeness. That’s why LMIC set out to test whether OJP data could be made more accurate. We benchmarked these data against Statistics Canada’s Job Vacancy and Wage Survey (JVWS).
The result? An encouraging demonstration project that offers tools to help analysts and decision-makers interpret this potentially promising data source in an informed, responsible, and appropriate way.
Why does data from online job postings matter?
Governments, educators, and researchers increasingly rely on data from OJP to make decisions. LMIC stakeholders often cite the benefits: it’s fast, timely, and more granular than Statistics Canada’s sources.
Perhaps ironically, the biggest limitations of Statistics Canada's data—like lag time—stem from its greatest strengths.
Statistics Canada’s data sources undergo rigorous statistical validation and apply complex weighting procedures to ensure the end results are representative across regions, industries, and populations. Strict confidentiality rules are also in place to protect individuals and prevent the misinterpretation of small sample sizes.
These safeguards are crucial. Without them, it would be too easy to draw misleading conclusions, especially when analyzing smaller communities or underrepresented populations. However, this statistical rigour comes with trade-offs: less granularity and slower data releases.
In contrast, OJP data are available almost immediately and offer more detail. But they lack the above protections, raising questions about how representative or reliable the data are for decision-making.
So, where do online job postings fit into this?
OJP data complement Statistics Canada’s data sources by offering more frequent, flexible access to information about labour market trends. This is especially valuable for Canadians living in small and remote communities, where other data may be sparse or outdated.
Unlike Statistics Canada's data sources, OJP data typically come with fewer restrictions on use and confidentiality. The data are high-frequency and dynamic, providing a near real-time temperature check of labour demand, especially during periods of dynamic change in the labour market.
Online job posting data isn’t perfect. What are the trade-offs?
In the same way that the drawbacks of Statistics Canada’s data stem from strengths, the strengths of OJP data also bring trade-offs.
The first is that OJP are not weighted to reflect representativeness. This has implications when using OJP as a proxy for job vacancies—which are sometimes used to gauge labour market demand. Job vacancies are an economic construct with strict parameters defined by Statistics Canada that allow for robust, reliable parameters of interpretation. Because OJP involve a different sampling approach, certain occupations can be under- or over-represented. Bias and confusion can arise if people misunderstand or misuse the data.
However, over- and under-sampling are not unusual in data sources and do not have to be a weakness. In fact, these techniques are regularly used in Statistics Canada’s surveys to get more reliable information about smaller communities and populations. They are a critical part of data collection. To make a dataset representative, it is weighted.
In deliberate over- and under-sampling, "weighting" creates an adjustment so the analysis will reflect the actual, real-life population—not the artificially over- or under-sampled dataset.
Why weighting data matters
Let’s imagine a province where a particular community represents just 5% of the provincial population. To ensure confidentiality and collect enough data for a meaningful and trustworthy analysis, researchers might deliberately oversample this community such that it makes up 10% of the overall survey sample—twice its actual share of the population. When we use the resulting dataset, we apply weighting so that it once again represents 5% of the population in the analysis. If we did not weight the dataset, we would have a skewed, incorrect view of that population, and possibly the rest of the province’s population as well.
In robust statistical datasets, multiple weights may be applied to account for different factors, such as age, region, or income. This is why weighting is not just important—it's crucial for reliable analyses. This is one of the biggest current limitations of OJP data, which typically lack weighting.
Can we make online job posting data more accurate?
LMIC tested a combination of advanced statistical and machine learning models and applied various weighting functions to evaluate whether OJP data could be made more accurate—specifically, by benchmarking against Statistics Canada’s JVWS.
The results were promising. We found that combining certain methods reduced prediction errors by 15%, showing strong potential to enhance the reliability of OJP data. While our current model did not apply industry- or region-specific weights, we suspect that refining this approach could lead to greater accuracy, making OJP data a more reliable source for data-driven decision-making.
Why improving online job posting quality is in our collective interest
OJP data are already a useful complement in the broader toolkit of data that broadens the landscape of the labour market. The flexibility and availability of these data open up new possibilities for innovation—particularly when statistical methods are applied to improve their accuracy.
As Canada’s labour market enters a potentially tumultuous time, it’s in everyone's interest to ensure that OJP data are not only accessible, but accurate and trustworthy. The consequences of poor data interpretation are not theoretical—they are real, and they affect real people.
Who stands to benefit?
The implications of this work are far-reaching:
- Governments need accurate, timely vacancy data to guide workforce and economic strategies, address skills shortages, and navigate shifting economic conditions.
- Workforce development organizations, which directly support jobseekers and employers by interpreting labour demand to make informed decisions, rely on vacancy estimates to connect jobseekers with real opportunities.
- Post-secondary institutions use vacancy trends to shape programs and make curriculum decisions. Improved OJP data interpretation can help align training with employer demand. In an era of declining enrolment and financial uncertainty, accurate insights into employer demand are critical for both post-secondary institutions and the local economies they support.
Across all sectors, better interpretation of OJP data leads to better guidance, smarter planning, and more effective outcomes for Canadians.
A demonstration—Not the end, but the beginning
This report was a demonstration: a proof of concept showing that it is, indeed, possible to create a reliable and trustworthy weighting method for OJP data. But there is more to do.
We hope this work offers a practical framework to begin the task of making OJP more accurate and trustworthy. The combined method we tested reduced prediction errors by 15%, offering real promise. With further refinement, these methods can help bridge the gap between high-frequency, real-time job posting data and statistically sound labour market estimates.
This work is particularly valuable for governments, educators, and researchers looking to supplement existing data sources. To learn more about the methods, results, and applications, read the full report and stay tuned for our upcoming OJP guide, designed to help users interpret and apply OJP data responsibly and effectively.