LinkUp

LinkUp spiders, parses, processes, stores, and delivers jobs data. 
Spider: Crawl job listings pages and collect all the URLs to the individual job Parse: Capture the contents of each page and extract the individual job elements Process: Clean up formatting, map locations, assign SOC | O*NET classification Store: Save job information in search engine and relational databases Deliver: Distribute jobs through feeds, API, reports, and platforms.

LinkUp’s dataset includes job listings in 195 countries, with a significant overweight to the United States. 

LinkUp’s historical dataset contains 110 million job records dating back to 2007, with a significant increase to coverage starting in 2012. The data available in Data Monster begins in 2012.  

LinkUp’s data is updated every 24-48 hours based on the complexity of a company scrape.

● Analyze sum totals of monthly unique active, new, and deleted openings  
● Identify locations with high workforce demand within specific industries and companies  
● Compare job counts by company, state, and industry  
● Use analytics on duration to draw insights about turnover and churn  
● Measure macro U.S. job openings from employers with the most job openings 

Jobs can temporarily disappear from LinkUp’s dataset due to the following:  
● A career site could go down temporarily for maintenance 
● Jobs could be formatted incorrectly rendering them "unscrapable" for a time 
● LinkUp’s scrape system could have an issue with a certain company ID  
● The company could have changed its ATS or career portal software  
These changes affect job count data at the company level, and the dataset overall experiences some drift as a result, which stabilizes after a period of about 10 days at the macro level. LinkUp’s proprietary scrape technology and dedicated full-time team of scrapers are notified of these changes and respond swiftly to maintain the integrity of the dataset by updating scrapes, which are prioritized by client need.  
 As mentioned above, approximately 80% of job listings in LinkUp’s index are located in the U.S. Because LinkUp scrapes directly from company career pages, the index does not include jobs from employers who post job listings exclusively on third-party sites or by other means. Small private businesses and some hourly, temporary, or contract job postings normally found in store windows or sites such as Craigslist will not be included in LinkUp’s dataset unless they are also on the company’s career page.  

LinkUp’s Data Dictionary for Job Records and Company Records include the following components: 
 Job Records:  
● Unique hash 
● Title ● Company ID  
● Company Name 
● City 
● Region/State 
● Zip Code 
● Country 
● Created Date 
● Last Checked Date 
● Updated Date 
● Deleted Date 
● SOC | O*NET Code 
● Full-text Description 
● URL  

Company Records: 
● Unique Hash 
● Company ID  
● Company Name 
● Homepage URL  
● Legal Entity ID  
● PermID  
● Ticker  
● NAICS Code 

LinkUp’s dataset covers all companies, all sectors, and all job types and appends the North American Industry Classification System (NAICS) to all companies. The dataset includes 30,000 companies globally, including all eligible companies in the major U.S. exchanges as well as private companies. 
Additionally, LinkUp applies Standard Occupational Classification (SOC) codes and the Occupational Information Network (O*NET) system to all current and historical job openings in 
its dataset. The SOC | O*NET system features over 1,000 standardized occupational categories with detailed descriptions.