Linguist concludes a JS/markdown repo with a single perl file is somehow "46.2% Perl"

[Pomax]

https://github.com/Pomax/BezierInfo-2 consists of 117 javascript files, 93 markdown files, 21 html files, 3 php files, 2 css files, and a single perl file. Yet somehow, the breakdown is:

Perl 46.2%
JavaScript 42.4%
HTML 9.7%
PHP 1.7%

Rather than the expected:

JavaScript 49.3%
Markdown 39.2%
HTML 8.8%
PHP 1.2%

so something seems to be going very wrong here =)

[lildude]

so something seems to be going very wrong here =)

Not from a Linguist perspective 😉 Maybe you missed the following when you read How Linguist Works:

Linguist starts by going through all the files in a repository and excludes all files that it determines to be binary data, vendored code, generated code, documentation, or are defined as data (e.g. SQL) or prose (e.g. Markdown) languages, whilst taking into account any overrides

This means all files in your docs dir will be ignored by default as they're considered documentation and all Markdown files will be ignored because they're considered prose.

Thus leaving only the files in src which contains your Perl file and brings us to the next point in that document:

The percentages are calculated based on the bytes of code for each language as reported by the List Languages API.

That single Perl file contains more bytes of code than all your JavaScript combined:

$ curl -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/Pomax/BezierInfo-2/languages
{
  "Perl": 62200,
  "JavaScript": 57149,
  "HTML": 13045,
  "PHP": 2247
}

So all in, Linguist is working as expected. If you wish to change this behaviour, you'll need to implement the relevant overrides to suit your desires.

[Pomax]

Ah, fantastic: that's the missing piece.

Can that please be added as short section to the README.md so that the fact that that file exists is immediately discoverable when folks follow the link that github supports gives them? It doesn't need to be a full explanation, just something along the lines of "if you need to change how linguist parses your repository, you can do this using a .gitattributes file, click here for detailed information"?

Maybe even under the "Usage" heading, since it's basically using linguist by proxy.

[lildude]

🤔 we do this already, though not in quite so many words, with links to the documentation right at the top of the README with the second link being pretty clear to me.

[Pomax]

hm, I don't see any mention of .gitattributes here? https://github.com/github/linguist/blob/master/README.md

The links to "the docs" are clear enough, but this is a classic case of "you don't know you even want to look for this in the docs unless you already know that's something you'll find in the docs": mentioning that Github users (rather than Linguist users) can configure how linguist works through a .gitattribtues file, directly in the README.md (without saying how, of course, just at least surfacing that as a thing they can do with links to the relevant section(s) in the docs), would have very high value, in addition to the "Application usage" and "Command line usage" sections.

[lildude]

We used to have all the documentation in the README however I moved it because it was cumbersome to maintain and not consistent with how you'd document the basic installation and usage of a Ruby library.

I'm apprehensive about re-introducing any mention of overrides etc back into the README as it will likely lead to further requests to "just add this" and soon we'll be back to square one. If anything is going to be documented for GitHub users (keep in mind GitHub isn't the only user of Linguist), it should be in the GitHub docs; after all, most GitHub users wouldn't know to come looking at Linguist unless referred here by support or the docs.

I admit the language coverage/config/stats info in the GitHub docs is lacking. I'm not responsible for those docs but can certainly bring it up with our docs team.

[Pomax]

I didn't know, so I contacted support, and they sent me here. So this entire conversation is evidence that it'd be good to at least mention there's a config file people can create. None of this conversation would have happened if that'd been at least mentioned in the README =)