Anakin

Author: Anakin
Version: 0.0.2
Type: Tool Plugin

Description

Anakin is a powerful web scraping and AI-powered search plugin for Dify. Built on the AnakinScraper API, it enables your AI applications to extract data from any website, perform intelligent web searches, and conduct deep research using advanced AI pipelines.

Key Features

• Anti-detection - Proxy routing across 207 countries prevents blocking
• Intelligent Caching - Up to 30x faster on repeated requests
• AI Extraction - Convert any webpage into structured JSON
• Browser Automation - Full Playwright/headless Chrome support for SPAs
• Session Management - Authenticated scraping with encrypted session storage (AES-256-GCM)
• Batch Processing - Submit multiple URLs in a single request

Setup

1. Get Your API Key

1. Sign up at anakin.io
2. Go to your Dashboard
3. Copy your API key (starts with )

2. Configure in Dify

1. Install the Anakin plugin in your Dify workspace
2. Go to Plugins → Anakin → Configure
3. Enter a name for the authorization (e.g., "Production")
4. Paste your API key
5. Click Save

Tools

1. URL Scraper

Scrapes a single URL, returning HTML, markdown, and optionally structured JSON.

Response includes:

• Raw HTML
• Cleaned HTML
• Markdown conversion
• Structured JSON (if enabled)
• Cache status
• Timing metrics

2. Batch URL Scraper

Scrape up to 10 URLs simultaneously in parallel.

3. AI Search

Synchronous AI-powered web search returning results with citations and relevance scoring.

Response includes:

• Array of results with URLs, titles, snippets
• Publication dates
• Last updated timestamps

Note: This is a synchronous endpoint - results are returned immediately without polling.

4. Deep Research (Agentic Search)

Multi-stage automated research pipeline combining search, scraping, and AI synthesis.

Response includes:

• AI-generated comprehensive answers
• Summaries
• Structured findings
• Citations with source URLs
• Scraped source data
• Processing metrics

Note: This is an async operation that typically takes 1-5 minutes.

5. Custom Web Scraper

Execute pre-configured scraper templates for domain-specific structured data extraction.

Response: Structured JSON matching the scraper's defined schema.

Usage Examples

In a Workflow

1. Add a Tool node to your workflow
2. Select Anakin → Choose your tool
3. Configure parameters:
   • For URL Scraper: Enter the URL, enable for structured output
   • For AI Search: Enter your query
4. Connect to the next node for processing

In an Agent

1. Create an Agent app
2. Add Anakin tools to the agent's toolset
3. The agent will automatically use scraping/search based on user queries

Example: Scraping with AI Extraction

Returns structured product data automatically extracted by AI.

Example: Authenticated Scraping

Scrapes pages that require login using your saved browser session.

Processing Times

Error Codes

Country Codes

Proxy routing supports 207 countries. Common codes:

• - United States (default)
• - United Kingdom
• - Germany
• - France
• - Japan
• - Australia

Support

• Source Code: GitHub Repository
• Website: anakin.io
• Documentation: anakin.io/llms-full.txt
• Dashboard: anakin.io/dashboard
• Support: [email protected]

License

This plugin is provided by Anakin. Usage is subject to Anakin's Terms of Service and Privacy Policy.