I need to get the contents of a file hosted in a GitHub repo. I'd prefer to get a JSON response with metadata along with it. I've tried numerous URLs with cURL with to only get a response of {"message":"Not Found"}. I just need the URL structure. If it matters, it's from an organization on GitHub. Here's what I think should work but doesn't:
http://api.github.com/repos/<organization>/<repository>/git/branches/<branch>/<file> As the description (located at http://developer.github.com/v3/repos/contents/) says:
/repos/:owner/:repo/contents/:path
An ajax code will be:
$.ajax({ url: readme_uri, dataType: 'jsonp', success: function(results) { var content = results.data.content; }); Replace the readme_uri by the proper /repos/:owner/:repo/contents/:path.
https://raw.githubusercontent.com/:owner/:repo/master/:path to get raw (binary, not Base64)
Accept header to application/vnd.github.v3.rawThis GitHub API page provides the full reference. The API endpoint for reading a file:
https://api.github.com/repos/{username}/{repository_name}/contents/{file_path} { "encoding": "base64", "size": 5362, "name": "README.md", "content": "encoded content ...", "sha": "3d21ec53a331a6f037a91c368710b99387d012c1", ... } Reading https://github.com/airbnb/javascript/blob/master/package.json using GitHub's API via curl:
curl -H 'Accept: application/vnd.github.v3.raw' https://api.github.com/repos/airbnb/javascript/contents/package.json Accept: application/vnd.github.v3.raw to get raw file response (thanks jakub.g)Reading https://github.com/airbnb/javascript/blob/master/package.json using GitHub's API in Python:
import base64 import json import requests import os def github_read_file(username, repository_name, file_path, github_token=None): headers = {} if github_token: headers['Authorization'] = f"token {github_token}" url = f'https://api.github.com/repos/{username}/{repository_name}/contents/{file_path}' r = requests.get(url, headers=headers) r.raise_for_status() data = r.json() file_content = data['content'] file_content_encoding = data.get('encoding') if file_content_encoding == 'base64': file_content = base64.b64decode(file_content).decode() return file_content def main(): github_token = os.environ['GITHUB_TOKEN'] username = 'airbnb' repository_name = 'javascript' file_path = 'package.json' file_content = github_read_file(username, repository_name, file_path, github_token=github_token) data = json.loads(file_content) print(data['name']) if __name__ == '__main__': main() GITHUB_TOKEN before running
'Accept: application/vnd.github.v3.raw' request header: curl -H 'Accept: application/vnd.github.v3.raw' 'https://api.github.com/repos/airbnb/javascript/contents/package.json' (no need to pipe to | jq -r ".content" | base64 --decode).Here's an alternative, more contemporary solution using fsspec in Python. This example includes grabbing a zip file from github and unzipping it locally.
GitHubFs uses PyGitHub under the hood, which is a Python library to access the GitHub REST API.
Goes without saying you will need to keep your token secure.
Requires fsspec with github and libarchive to be installed.
from pathlib import Path from fsspec import filesystem def get_remote_github_obj( org: str = "my-org", repo: str = "my-repo", branch: str = "my-branch", username: str = "my-username", token: str = "my-github-pat" ): git_fs = filesystem( "github", org=org, repo=repo, branch=branch, username=username, token=token, ) # Create a new dir Path("/path/to/desired/local/download/location/").mkdir(parents=True, exist_ok=True) # Get the zip archive from URL git_fs.get( rpath="relative/path/to/remote/zip_file.zip", lpath="/path/to/desired/local/download/location/zip_file.zip" ) # Use libarchive to unarchive libarchive_fs = filesystem( "libarchive", fo="/path/to/desired/local/download/location/zip_file.zip", ) libarchive_fs.get( rpath="/", lpath="/path/to/desired/local/download/location/example_unarchived_dir", recursive=True, ) 