GitHub Repository Markdown Scraper

This actor scrapes GitHub repositories and converts their contents into markdown format using the gitingest.com service. It's useful for documentation, analysis, or creating searchable content from GitHub repositories.

Features

• Process multiple GitHub repository URLs
• Configurable file inclusion/exclusion patterns
• Adjustable maximum file size limit
• Converts repository content to markdown format

Input Parameters

• githubRepoUrls (required): Array of GitHub repository URLs to process
• patternType (optional): Whether to "include" or "exclude" files matching the pattern (default: "exclude")
• pattern (optional): Glob pattern for files to include/exclude (e.g., "*.md", "src/")
• maxFileSizeKb (optional): Maximum file size in kilobytes to include in output (default: 50)

Output

The actor outputs JSON objects with the following structure for each processed repository:

1{
2    "url": "https://github.com/user/repo",
3    "markdownContent": "# Repository Content..."
4}

Example Usage

1{
2    "githubRepoUrls": [
3        "https://github.com/username/repository",
4        "https://github.com/username/another-repo/tree/main"
5    ],
6    "patternType": "include",
7    "pattern": "*.md",
8    "maxFileSizeKb": 100
9}