gb2312 / GB18030 encoding detection conflict

[wesinator]

Prerequisites

• Put an X between the brackets on this line if you have done all of the following:
  • Reproduced the problem in Safe Mode: http://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
  • Followed all applicable steps in the debugging guide: http://flight-manual.atom.io/hacking-atom/sections/debugging/
  • Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
  • Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
  • Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Description

A text file with encoding detected as GB18030 on Github is auto-detected as gb2312 in atom.

Steps to Reproduce

https://github.com/malice-plugins/yara/blob/17a4fc946febe8b002e285f591bcb21b92a99e9e/rules/userdb_panda.yar

1. Edit this file on Github
   
2. Open in Atom
3. Select "Auto Detect" encoding,

Expected behavior: Atom detects the encoding of the file as GB18030.

Actual behavior: Atom auto detects the encoding as gb2312, 'undefined encoding'

iconv fails to convert from GB2312, but works with GB18030:

iconv -f GB2312 -t UTF-8 userdb_panda.yar
iconv: illegal input sequence at position 29230

Works: iconv -f GB18030 -t UTF-8 userdb_panda.yar

Reproduces how often: Always

Versions

apm 2.1.2
atom 1.32.0 x64
Ubuntu 18.04

Additional Information

[rsese]

Thanks for the report! I can reproduce as described with macOS 10.12.6 with 1.34.0-nightly5.

[maxbrunsfeld]

Here is the library that we're using for encoding detection: https://github.com/aadsm/jschardet.

[wesinator]

@maxbrunsfeld Do I need to open a new issue there ?

[maxbrunsfeld]

@wesinator Yeah, that's what I would recommend.