This library is used on GitHub.com to detect blob languages, ignore binary or vendored files, suppress generated files in diffs, and generate language breakdown graphs.

• How Linguist works
• Change Linguist's behaviour with overrides
• Troubleshooting
• Contributing guidelines

Install the gem:

gem install github-linguist

Linguist is a Ruby library so you will need a recent version of Ruby installed. There are known problems with the macOS/Xcode supplied version of Ruby that causes problems installing some of the dependencies. Accordingly, we highly recommend you install a version of Ruby using Homebrew, rbenv, rvm, ruby-build, asdf or other packaging system, before attempting to install Linguist and the dependencies.

Linguist uses charlock_holmes for character encoding and rugged for libgit2 bindings for Ruby. These components have their own dependencies.

1. charlock_holmes
   • cmake
   • pkg-config
   • ICU
   • zlib
2. rugged
   • libcurl
   • OpenSSL

You may need to install missing dependencies before you can install Linguist. For example, on macOS with Homebrew:

brew install cmake pkg-config icu4c

On Ubuntu:

sudo apt-get install build-essential cmake pkg-config libicu-dev zlib1g-dev libcurl4-openssl-dev libssl-dev ruby-dev

Linguist can be used in your application as follows:

require 'rugged' require 'linguist' repo = Rugged::Repository.new('.') project = Linguist::Repository.new(repo, repo.head.target_id) project.language #=> "Ruby" project.languages #=> { "Ruby" => 119387 }

The github-linguist executable operates in two distinct modes:

1. Git Repository mode - Analyzes an entire Git repository (when given a directory path or no path)
2. Single file mode - Analyzes a specific file (when given a file path)

A repository's languages stats can be assessed from the command line using the github-linguist executable. Without any options, github-linguist will output the language breakdown by percentage and file size.

cd /path-to-repository github-linguist

You can try running github-linguist on the root directory in this repository itself:

$ github-linguist 66.84% 264519 Ruby 24.68% 97685 C 6.57% 25999 Go 1.29% 5098 Lex 0.32% 1257 Shell 0.31% 1212 Dockerfile

The --rev REV flag will change the git revision being analyzed to any gitrevisions(1) compatible revision you specify.

This is useful to analyze the makeup of a repo as of a certain tag, or in a certain branch.

For example, here is the popular Jekyll open source project.

$ github-linguist jekyll 70.64% 709959 Ruby 23.04% 231555 Gherkin 3.80% 38178 JavaScript 1.19% 11943 HTML 0.79% 7900 Shell 0.23% 2279 Dockerfile 0.13% 1344 Earthly 0.10% 1019 CSS 0.06% 606 SCSS 0.02% 234 CoffeeScript 0.01% 90 Hack

And here is Jekyll's published website, from the gh-pages branch inside their repository.

$ github-linguist jekyll --rev origin/gh-pages 100.00% 2568354 HTML

The --breakdown or -b flag will additionally show the breakdown of files by language.

You can try running github-linguist on the root directory in this repository itself:

$ github-linguist --breakdown 66.84% 264519 Ruby 24.68% 97685 C 6.57% 25999 Go 1.29% 5098 Lex 0.32% 1257 Shell 0.31% 1212 Dockerfile Ruby: Gemfile Rakefile bin/git-linguist bin/github-linguist ext/linguist/extconf.rb github-linguist.gemspec lib/linguist.rb …

The --strategies or -s flag will show the language detection strategy used for each file. This is useful for understanding how Linguist determined the language of specific files. Note that unless the --json flag is specified, this flag will set the --breakdown flag implicitly.

You can try running github-linguist on the root directory in this repository itself with the strategies flag:

$ github-linguist --breakdown --strategies 66.84% 264519 Ruby 24.68% 97685 C 6.57% 25999 Go 1.29% 5098 Lex 0.32% 1257 Shell 0.31% 1212 Dockerfile Ruby:  Gemfile [Filename]  Rakefile [Filename]  bin/git-linguist [Extension]  bin/github-linguist [Extension]  lib/linguist.rb [Extension]  …

If a file's language is affected by .gitattributes, the strategy will show the original detection method along with a note indicating whether the gitattributes setting changed the result or confirmed it.

For instance, if you had the following .gitattributes overrides in your repo:

*.ts linguist-language=JavaScript *.js linguist-language=JavaScript 

the output of Linguist would be something like this:

100.00% 217 JavaScript JavaScript:  demo.ts [Heuristics (overridden by .gitattributes)]  demo.js [Extension (confirmed by .gitattributes)]

The --json or -j flag output the data into JSON format.

$ github-linguist --json {"Dockerfile":{"size":1212,"percentage":"0.31"},"Ruby":{"size":264519,"percentage":"66.84"},"C":{"size":97685,"percentage":"24.68"},"Lex":{"size":5098,"percentage":"1.29"},"Shell":{"size":1257,"percentage":"0.32"},"Go":{"size":25999,"percentage":"6.57"}}

This option can be used in conjunction with --breakdown to get a full list of files along with the size and percentage data.

$ github-linguist --breakdown --json {"Dockerfile":{"size":1212,"percentage":"0.31","files":["Dockerfile","tools/grammars/Dockerfile"]},"Ruby":{"size":264519,"percentage":"66.84","files":["Gemfile","Rakefile","bin/git-linguist","bin/github-linguist","ext/linguist/extconf.rb","github-linguist.gemspec","lib/linguist.rb",...]}} 

NB. The --strategies flag has no effect, when the --json flag is present.

Alternatively you can find stats for a single file using the github-linguist executable.

You can try running github-linguist on files in this repository itself:

$ github-linguist grammars.yml grammars.yml: 884 lines (884 sloc)  type: Text  mime type: text/x-yaml  language: YAML

This flag has no effect in Single file mode.

When using the --strategies or -s flag with a single file, you can see which detection method was used:

$ github-linguist --strategies lib/linguist.rb  lib/linguist.rb: 105 lines (96 sloc)  type: Text  mime type: application/x-ruby  language: Ruby  strategy: Extension

If a file's language is affected by .gitattributes, the strategy will show whether the gitattributes setting changed the result or confirmed it:

In this fictitious example, it says "confirmed by .gitattributes" since the detection process (using the Filename strategy) would have given the same output as the override:

.devcontainer/devcontainer.json: 27 lines (27 sloc)  type: Text  mime type: application/json  language: JSON with Comments  strategy: Filename (confirmed by .gitattributes)

In this other fictitious example, it says "overridden by .gitattributes" since the gitattributes setting changes the detected language to something different:

test.rb: 13 lines (11 sloc)  type: Text  mime type: application/x-ruby  language: Java  strategy: Extension (overridden by .gitattributes)

Here, the .rb file would normally be detected as Ruby by the Extension strategy, but .gitattributes overrides it to be detected as Java instead.

Using the --json flag will give you the output for a single file in JSON format:

$ github-linguist --strategies --json lib/linguist.rb {"lib/linguist.rb":{"lines":105,"sloc":96,"type":"Text","mime_type":"application/x-ruby","language":"Ruby","large":false,"generated":false,"vendored":false}}

NB. The --strategies has no effect, when the --json flag is present.

If you have Docker installed you can either build or use our pre-built images and run Linguist within a container:

$ docker run --rm -v $(pwd):$(pwd):Z -w $(pwd) -t ghcr.io/github-linguist/linguist:latest 66.84% 264519 Ruby 24.68% 97685 C 6.57% 25999 Go 1.29% 5098 Lex 0.32% 1257 Shell 0.31% 1212 Dockerfile

$ docker build -t linguist . $ docker run --rm -v $(pwd):$(pwd):Z -w $(pwd) -t linguist 66.84% 264519 Ruby 24.68% 97685 C 6.57% 25999 Go 1.29% 5098 Lex 0.32% 1257 Shell 0.31% 1212 Dockerfile $ docker run --rm -v $(pwd):$(pwd) -w $(pwd) -t linguist github-linguist --breakdown 66.84% 264519 Ruby 24.68% 97685 C 6.57% 25999 Go 1.29% 5098 Lex 0.32% 1257 Shell 0.31% 1212 Dockerfile Ruby: Gemfile Rakefile bin/git-linguist bin/github-linguist ext/linguist/extconf.rb github-linguist.gemspec lib/linguist.rb …

Please check out our contributing guidelines.

The language grammars included in this gem are covered by their repositories' respective licenses. vendor/README.md lists the repository for each grammar.

All other files are covered by the MIT license, see LICENSE.