Q: 2 questions regarding capabilities
Hi Sean, AgenticFlow looks very interesting! I had two specific use cases I’d love your input on:
1) Job Scraping at Scale
Would AgenticFlow be capable of scraping job postings (specifically Salesforce-related) from a list of around 200 company job pages and exporting the results in a standardized format (URL, job title, company name)? Ideally without having to define CSS or Xpath selectors or pagination for each individual site?
2) LinkedIn Contact Research
Can AgenticFlow be used to automatically identify relevant contacts on LinkedIn for a given list of companies – based on filters like job title, function, and seniority?
I'm only looking to extract basic public profile data (first name, last name, job title) – no messaging or contact scraping involved.
Happy to clarify or expand if needed – thanks in advance!
SeanP_AgenticFlowAI
May 30, 2025A: Hey Sumoling07061981!
AgenticFlow looks like a great fit for these use cases!
This is good tutorial: https://youtu.be/ZdLY7EVh3PM?si=QmNUNw9F1N2MVtoN
Let's dive into the specifics:
1) Job Scraping at Scale (Salesforce Jobs from ~200 Company Pages):
Yes, this is achievable, with the right tools connected!
Capability: AgenticFlow can orchestrate this.
How it Works:
- Input: You'd provide the list of ~200 company job page URLs (e.g., in a Table Dataset or triggered via API).
- Scraping (Key Part):
Manually defining CSS/XPath for 200 different sites is indeed a nightmare and not what AgenticFlow does natively.
Best Approach: You'd use an AI-powered scraping tool that can understand page structure without explicit selectors.
Firecrawl MCP (Extract Action): (https://agenticflow.ai/mcp/firecrawl) – This is ideal. You can provide the URL and a prompt like, "Extract all job postings related to Salesforce from this page. For each job, return the job title, company name, and the direct URL to the job posting. Format as JSON." Firecrawl's "Extract" is designed for this.
Apify MCP: (https://agenticflow.ai/mcp/apify) – You could use a generic "Web Scraper" Actor on Apify and try to guide it with smart instructions, or find/build an Apify Actor specifically for job postings that uses AI to identify common job listing patterns.
LLM with Raw Scrape (Less Reliable for Scale): You could use our basic web_scraping node to get HTML, then an LLM node to parse out job details, but this will be less reliable across 200 diverse sites than a specialized AI scraper like Firecrawl Extract.
- Standardizing Output: The LLM used by Firecrawl Extract (or an LLM step you add after Apify) can be prompted to return the data in your desired standardized format (URL, job title, company name).
- Exporting: Save the structured data to a Google Sheet (via MCP: https://agenticflow.ai/mcp/google_sheets) or export as a CSV/XLSX file using the "Export Data to File" node.
"Without defining CSS/Xpath/pagination": This is precisely what AI-powered extraction (like Firecrawl's Extract feature) aims to solve. You rely on the AI to understand the page structure. Pagination might still need some handling (e.g., instructing the scraper to click "next" if the tool supports it, or iterating through page numbers if URLs are predictable).
2) LinkedIn Contact Research (Public Profile Data):
Yes, for publicly available data via API, within limits.
How it Works:
Input: Your list of target companies.
Identify Contacts (via MCP):
Apollo.io MCP (Recommended): (https://agenticflow.ai/mcp/apollo_io) This is a B2B database. You can use its "Search Contacts" action, filtering by company name, job titles (e.g., "Sales Manager," "VP of Enablement"), function, and seniority level. It's designed for this.
LinkedIn MCP: (https://agenticflow.ai/mcp/linkedin) You can use actions like "Search Organization" to find company URNs, and then potentially try to find employees or use its limited search capabilities. However, the LinkedIn official API is very restrictive about broad employee searching and bulk data extraction to prevent abuse. It's not designed as a mass prospecting scraping tool. You'd typically get basic profile info for connections or publicly searchable individuals.
Extract Basic Public Data: Once a relevant contact is identified (especially via Apollo.io), the data returned usually includes first name, last name, and job title.
"No messaging or contact scraping": This aligns with what official APIs generally permit. AgenticFlow operates through these official channels.
In Short:
- Job Scraping: Yes, very possible and efficient using AgenticFlow to orchestrate an AI-powered scraper like Firecrawl (via its "Extract" action using prompts) or Apify.
- LinkedIn Contact Research: Yes, for identifying relevant profiles and getting basic public data, primarily by integrating with B2B databases like Apollo.io via MCP. Direct, large-scale "scraping" of LinkedIn profiles for contacts is generally limited by LinkedIn's API policies.
For both use cases, leveraging our Multi-Agent System Add-On (if you have T3/4) could be beneficial for handling the ~200 sites in parallel for job scraping or managing different stages of contact research and data formatting.
This looks like a solid plan, and AgenticFlow is well-equipped to be the central nervous system for these automations!
— Sean