Update libmagic; link system ICU on Darwin; use RbConfig CFLAGS, etc.

[geoff-nixon]

This pull request includes changes intended to allow the resolution of #29, #40, #42 (partial), #43, #49, #50, #51, #52.

• file/libmagic (and patch) updated to 5.16.
• extconf.rb modified to:

  • enforce RbConfig compiler and related flags.

  • fix for non-GNU 'make' (ie, no make -C).

  • remove 'dst' build directory after compiling.

  • Build and link against system ICU on Darwin.
    I've included a minimal subset of the ICU headers in darwin-icu-headers.tar.gz,
    taken from the Apple ICU source (Apple doesn't distribute these headers because they prefer you use their client frameworks like Core Text, etc., but they're available freely at the link above). These headers are from the version bundled with 10.7 (10.6 was too old, so I've retained the check for homebrew icu4c on older versions).

    I've tested that they work on 10.9, so presumably it should work on 10.7 and 10.8 as well.
    If anyone can confirm that, that would be great.

    I think perhaps this is a better solution than relying on brew install icu4c?

[brianmario]

Wow awesome thank you! I'll try and pull this down and play with it later today or this weekend. Looking at which versions of ICU are including in the various OSX versions - it indeed looks like we'll only support 10.7+ (this is probably due to the use of the Transliteration API I recently added).

I've been considering removing libmagic and replacing it with a much simpler, but consequently less accurate function that does the following:

1. Check if there's a BOM at the beginning of the data we're testing, if so return that as the encoding
2. If there's no BOM, check if there's a null byte within the first 1024 bytes (this is what git does for it's binary detection). If a null byte was found, return that the data is binary.
3. If there's no BOM and no null byte, pass it along to ICU for detection.

This roughly what libmagic is doing today, but with a much more complex heuristic. The issue with libmagic is that it makes 2 copies (well, at least one) of the input text which can be pretty massive in some cases which sucks. Also a lot of people have had issues with libmagic for whatever reason.

Anyway, thanks for this. I'll do what I can to get it merged soon.

[geoff-nixon]

Awesome. Hopefully this will result in fewer headaches for some people.

Before you remove libmagic: take a look at this issue and pull for me.

Never mind that, I think I figured it out.

~ Geoff

[geoff-nixon]

So apparently you can't really use RbConfig::MAKEFILE_CONFIG['CFLAGS'] since I guess some installs (including the system Ruby on 10.7) use a makefile expansion in that variable $(cflags).
So I switched it to -O2 -fPIC -std=gnu99, which I think should work just about anywhere. (Pushed and squashed.)

But with that tweak, I can confirm it does indeed work on 10.7 and 10.8.

[geoff-nixon]

Closed as libmagic was removed in 0.7.0. The other half of this pull is now in #66.