GitHub Datasource Plugin

Access GitHub repositories, issues, pull requests, and wiki pages as a datasource for Dify with comprehensive authentication support.

Features

Repository Access: Browse and download files from public and private repositories
Issues & Pull Requests: Access issue and PR content with comments
Multiple Authentication: Support both Personal Access Token and OAuth
Rate Limit Handling: Automatic rate limit detection and handling
Content Processing: Automatic markdown processing and content extraction
Multi-Content Types: Support for various file formats and GitHub content types

Supported Content Types

Repository files (Markdown, code, documentation)
GitHub Issues with comments
Pull Requests with comments
Various file formats (JSON, YAML, Python, JavaScript, etc.)
README files and repository metadata

Setup and Installation

Requirements

Dify platform version >= 1.9.0
Python 3.12+
Valid GitHub account with appropriate permissions

Installation

Install the plugin in your Dify instance
Configure authentication credentials (see Authentication section below)
Test the connection and start using GitHub as a datasource

Authentication

Option 1: Personal Access Token (Recommended for Development)

Go to GitHub Settings > Developer settings > Personal access tokens
Click "Generate new token (classic)"
Select the following scopes:
- - Full control of private repositories
- - Access user email addresses
- - Read user profile data
Copy the generated token
In Dify, configure the datasource with your token

Example Configuration:

Option 2: OAuth (Recommended for Production)

Create a GitHub OAuth App:
- Go to GitHub Settings > Developer settings > OAuth Apps
- Click "New OAuth App"
- Fill in application details:
  - Application name: "Dify GitHub Integration"
  - Homepage URL: "https://your-dify-domain.com"
  - Authorization callback URL: "https://your-dify-domain.com/console/api/oauth/callback"
Note the Client ID and Client Secret
Configure in Dify system settings
Users can then authorize through OAuth flow

System Configuration Example:

Usage Workflows

1. Repository Content Access

Add GitHub datasource to your Dify knowledge base
The plugin will automatically discover your accessible repositories
Select repositories, files, issues, or PRs to include
Content is automatically processed and indexed

2. Issue Tracking Integration

Configure GitHub datasource with appropriate permissions
Issues and PRs are automatically discovered and indexed
Search and query issue content using Dify's AI capabilities
Get insights from issue discussions and resolutions

3. Documentation Management

Connect repository with extensive documentation
README files and markdown documents are automatically processed
Create AI-powered documentation search and Q&A
Keep documentation knowledge base synchronized

Configuration Examples

Basic Datasource Configuration

Provider Configuration

Rate Limits and Performance

GitHub API Limits

Personal Access Token: 5,000 requests per hour
OAuth: 5,000 requests per hour per user
GitHub Apps: Higher limits available

Optimization Features

Automatic rate limit detection and handling
Intelligent request batching
Content caching to reduce API calls
Graceful degradation when limits are reached

Troubleshooting

Common Issues

"Invalid access token" Error

Problem: Authentication fails with token error

Solution:

Verify token is valid and not expired
Check token has required scopes (, , )
Ensure token hasn't been revoked
Generate a new token if needed

"Rate limit exceeded" Error

Problem: Too many API requests in short time

Solution:

Wait for rate limit reset (shown in error message)
Reduce the number of repositories being accessed
Consider using GitHub Apps for higher limits
Implement request batching in your usage

"Repository not found" Error

Problem: Cannot access specific repository

Solution:

Verify repository name is correct (case-sensitive)
Check if repository is private and token has access
Ensure token has scope for private repositories
Verify you have read access to the repository

Debug Mode

Enable debug logging to troubleshoot issues:

FAQ

Q: Can I access private repositories?

A: Yes, with proper Personal Access Token with scope or OAuth authorization.

Q: Are GitHub Enterprise repositories supported?

A: Currently supports GitHub.com only. Enterprise support may be added in future versions.

Q: How often is content synchronized?

A: Content is fetched in real-time when accessed. No background synchronization is performed.

Q: What happens if my token expires?

A: Personal Access Tokens don't expire unless manually revoked. OAuth tokens are handled automatically.

Q: Can I limit which repositories are accessible?

A: Yes, you can configure specific repositories in the datasource configuration.

Security Considerations

Token Management

Store tokens securely using Dify's encrypted credential storage
Regularly rotate Personal Access Tokens
Monitor token usage in GitHub settings
Use minimal required scopes

Data Privacy

Plugin only accesses explicitly authorized content
No data is stored outside of Dify platform
All API communications are encrypted with HTTPS
Follows GitHub's data handling policies

Support and Community

Documentation

Issues and Feature Requests

Report issues through Dify support channels
Feature requests can be submitted via Dify community

Version History

v0.3.0: Complete OAuth support, Issue/PR access
v0.2.0: Enhanced authentication methods
v0.1.0: Basic repository file access

Version: 0.3.0

This plugin implements all features up to v0.3.0 including comprehensive OAuth support, advanced content access, and enhanced security measures.