Crawl4AI Cloud

github-file

Files of a public GitHub repository, one row per file: one file, or every file under a folder (sub-folders included, in path order). A row holds the path, ref, blob sha, size in bytes, a language guessed from the extension, is_binary, the UTF-8 text cut at max_chars (null for a binary file) and the raw and GitHub URLs. url is a github.com /blob/ or /tree/ URL, a raw URL, or owner/name (the root folder); ref and path replace the URL's; max_files caps a folder. 2-4 units, plus 1 per text file of a folder.

stagedraw.githubusercontent.comWASM module11 fields2-4 units, +1 per text file in a folderfresh 5 min
Run it belowAPI reference
‹ Recipes
…json
POST api.crawl4ai.com/recipes/…

$ crawl recipe

              
Live · your key · counts against your quota✓ 200 OK

              
# Hit Run - the rows render here as a table, with the cost

            

            
            
          

        

      

    





Fields
11 columns on every row. null means the site does not state it.


fieldtypemeaning
repositorystringThe repository as owner/name.
pathstringThe file's path from the repository root.
refstringThe branch, tag or commit the file was read at: the one asked for, else the repository's default branch.
shastringThe file's git blob sha at the commit the ref pointed to when the row was read, the same commit as content; never computed from a cut text; null only when GitHub's API refused the one call that gives it.
sizeintegerThe file's full size in bytes, also when the text is cut.
languagestringA language guessed from the file name or extension with a fixed table (Rust, Python, Markdown, ...); null when the table has no entry.
is_binarybooleantrue when the file is not UTF-8 text (a NUL byte or invalid UTF-8 in what was read); in a folder a known binary extension (png, zip, pdf, fonts) counts and is not downloaded; null when the file could not be read.
contentstringThe file's text, cut at max_chars characters; null for a binary file or a file that could not be read.
is_truncatedbooleantrue when content is shorter than the file (cut at max_chars, or a file over 8 MB).
raw_urlstringThe file on raw.githubusercontent.com at this ref.
urlstringThe file's page on github.com at this ref.



null means the site does not state it.
repositorystringThe repository as owner/name.
pathstringThe file's path from the repository root.
refstringThe branch, tag or commit the file was read at: the one asked for, else the repository's default branch.
shastringThe file's git blob sha at the commit the ref pointed to when the row was read, the same commit as content; never computed from a cut text; null only when GitHub's API refused the one call that gives it.
sizeintegerThe file's full size in bytes, also when the text is cut.
languagestringA language guessed from the file name or extension with a fixed table (Rust, Python, Markdown, ...); null when the table has no entry.
is_binarybooleantrue when the file is not UTF-8 text (a NUL byte or invalid UTF-8 in what was read); in a folder a known binary extension (png, zip, pdf, fonts) counts and is not downloaded; null when the file could not be read.
contentstringThe file's text, cut at max_chars characters; null for a binary file or a file that could not be read.
is_truncatedbooleantrue when content is shorter than the file (cut at max_chars, or a file over 8 MB).
raw_urlstringThe file on raw.githubusercontent.com at this ref.
urlstringThe file's page on github.com at this ref.


How it behaves
Cost. 2-4 units, +1 per text file in a folder. A failed run costs 0.
Freshness 5 min. The same input within 5 min is served from cache at 0 units.
How it answers. Answers at once for one input; a large run goes as a job you poll.














Questions about this recipeQuestions
Short answers. The API reference has the long ones.


Call POST /recipes/github-file with your key and url and ref and path. You get rows with the 11 fields above. The recipe is tested against the live site every 6 hours.







Need more from GitHub?
Extra fields, higher volume, or a private copy with a freshness SLA. We build it on Enterprise.

Request a change





More for developers and open source
All 7 recipes →


github-repoGitHub
verified
One GitHub repository's facts, then its latest releases: the first row is the repository (stars, forks, open issues, language, license…
Startup and investor intel17 fields
3 units · fresh 10 minTry it →
github-trendingGitHub
verified
The repositories trending on GitHub today: name, description, language, stars and stars gained today.
Startup and investor intel7 fields
2 units · fresh 30 minTry it →
package-infonpm and PyPI
verified
One row of facts about a package from its registry's public JSON: latest version and when it was published, description, license, homepage…
Developers and open source11 fields
2 units · fresh 1 hTry it →












For developers
Get a free API key and run your first crawl.
1,000 credits every month. No credit card. Your self-hosted setup keeps working.


Get a free API key



For teams
Talk to us about volume, regions and an SLA.
A 30-minute call with the team that writes the crawler. DPA on request, invoice billing, your choice of US, EU or Singapore.


Book a callhello@crawl4ai.com