How I Built a Company Career & ATS Hiring Signal Scraper + AI Markdown RAG Extractor in Node.js

When building LLM-powered RAG pipelines or B2B outbound intelligence systems, two data bottlenecks come up repeatedly:

As part of my suite of 8 production Node.js scrapers on the Apify Store, I built two focused actors to solve both problems using fast HTTP/Cheerio pipelines in Node.js:

Here is a practical walkthrough of the architecture and implementation behind both tools.

In AI Website to Clean Markdown & RAG Chunk Extractor, the pipeline runs in three deterministic stages:

Before converting anything to Markdown, we strip noisy tags and common boilerplate selectors while preserving semantic headings ( h1 – h4 ), paragraphs, lists, code blocks, and tables:

function chunkMarkdownWithHeadings ( markdown, maxWords = 300 ) { const blocks = markdown. split ( / \n{2,} / ); const chunks = []; let currentHeading = ''; let buffer = []; let wordCount = 0; for ( const block of blocks ) { const trimmed = block. trim (); if (! trimmed ) continue; if ( /^# {1,3}\s +/. test ( trimmed )) { currentHeading = trimmed; } const words = trimmed. split ( / \s +/ ). length; if ( wordCount + words > maxWords && buffer. length > 0 ) { chunks. push ({ headingContext: currentHeading, content: buffer. join ( ' \n\n ' ), wordCount, estimatedTokens: Math. ceil ( wordCount * 1.33 ) }); buffer = currentHeading? [ currentHeading ]: []; wordCount = currentHeading? currentHeading. split ( / \s +/ ). length: 0; } buffer. push ( trimmed ); wordCount += words; } if ( buffer. length > 0 ) { chunks. push ({ headingContext: currentHeading, content: buffer. join ( ' \n\n ' ), wordCount, estimatedTokens: Math. ceil ( wordCount * 1.33 ) }); } return chunks; } Enter fullscreen mode Exit fullscreen mode Each output item includes the page metadata (title, canonical URL, OpenGraph/JSON-LD summary), full cleaned Markdown, word/token estimates, and the ready-to-embed ragChunks array for OpenAI, Claude, or LangChain.

When enriching B2B leads, a company with 8 open engineering roles on Ashby or Greenhouse is a very different prospect from a dormant brochure site.

In Company Career Page, ATS & Hiring Signal B2B Lead Scraper, we start from a company's root domain, discover its /careers or /jobs links, and fingerprint the underlying Applicant Tracking System (ATS):

const ATS_SIGNATURES = [ { name: ' Greenhouse ', pattern: /boards \. greenhouse \. io|job-boards \. greenhouse \. io|greenhouse \. io \/ embed/i }, { name: ' Lever ', pattern: /jobs \. lever \. co|api \. lever \. co/i }, { name: ' Ashby ', pattern: /jobs \. ashbyhq \. com|ashbyhq \. com/i }, { name: ' Workable ', pattern: /apply \. workable \. com|workable \. com/i }, { name: ' BambooHR ', pattern: / \. bamboohr \. com \/( careers|jobs ) /i }, { name: ' BreezyHR ', pattern: / \. breezy \. hr/i }, { name: ' Personio ', pattern: / \. jobs \. personio \.( de|com ) /i }, { name: ' Teamtailor ', pattern: /teamtailor \. com/i }, { name: ' SmartRecruiters ', pattern: /jobs \. smartrecruiters \. com/i }, { name: ' Workday ', pattern: /myworkdayjobs \. com/i } ]; function detectAtsPlatforms ( html, outboundLinks ) { const combined = [ html,... outboundLinks ]. join ( ' ' ); return ATS_SIGNATURES. filter ( sig => sig. pattern. test ( combined )). map ( sig => sig. name ); } Enter fullscreen mode Exit fullscreen mode Prioritizing Talent & Recruiting Contacts Alongside open role titles and external ATS board URLs, the actor extracts emails and prioritizes recruiting-specific inboxes ( careers@, hr@, talent@, jobs@, people@ ) ahead of generic support addresses:

function prioritizeHiringEmails ( emails ) { const priorityRegex = /^ ( careers|jobs|talent|hr|recruiting|people ) @/i; return [... new Set ( emails )]. sort (( a, b ) => { const aPri = priorityRegex. test ( a )? 0: 1; const bPri = priorityRegex. test ( b )? 0: 1; return aPri - bPri; }); } Enter fullscreen mode Exit fullscreen mode Try the Full Suite of 8 Actors & Browser Extensions Both actors are live on the Apify Store alongside 6 additional B2B, e-commerce, and technical SEO scrapers:

If you have feedback on chunking heuristics or ATS edge cases, drop a comment below!