[For discussion] Automated Travis CI (OS X) and Docker (Linux) builds

[chriskilding]

I'm looking at starting an iOS app which makes heavy use of a Haskell library. To ensure some degree of sanity in the build process (i.e. avoiding having a temperamental process that only works on my dev box), and to take some of the work out of keeping up with the Haskell library as it gets updated over time, I want to automate at least part of the process of cross-compiling it for iOS. I expect others who are thinking of doing Haskell + iOS projects are thinking along the same lines.

Basically it would be nice (if possible) to have all the Haskell side of it hosted on a build server such that I can periodically grab an ARM/iOS build of the Haskell library off GitHub Releases, and stuff it in my Xcode project. (There are obviously prettier ways of doing this with dependency managers, but this would be a solid first step that would remove a lot of manual work.)

With this in mind there are 2 major target environments for ghc-ios automated builds:

• Travis CI, which has an OS X build environment specifically for Mac and iOS apps (easier to support, and the more important of the two, because a downstream Haskell+iOS project would be likely to also use Travis CI to run its own test suite)
• Docker, which is your choice of Linux distro + extra bits as defined in the Dockerfile (harder work because it's Linux, but the preliminary work I've done suggests at least parts of this could work)

The first step is to get ghc-ios compiling and running in either or both of these environments, such that it can compile a 'hello world' Haskell program. Which is what this PR contains.

I would appreciate comments, ideas, fixes etc on how exactly to proceed with this.

You can check out the current state of the automated builds at:

• https://travis-ci.org/themasterchef/ghc-ios-scripts
• https://registry.hub.docker.com/u/themasterchef/ghc-ios-scripts/

Note: a Travis CI build would be beneficial for you guys, as you could automatically validate submissions to this repository to see if they work, without applying the diff to a copy of ghc-ios on your own dev boxes manually.

[chriskilding]

Finding 1: The clang/llvm binary is OS-specific so it can't be assumed in the shell script that we will always use the Mac binary. Therefore I chose to extract LLVM installation from the shell script, and to run this platform-specific step in the Travis and Docker config files.

[chriskilding]

Finding 2: There is a bizarre illegal instruction failure on OS X in the make install_packages routine deep in the compilation of both the device and simulator binaries of ghc-ios. The error is as follows:

"inplace/bin/ghc-cabal" copy libraries/ghc-prim dist-install ":" '' '/usr/local' '/usr/local/lib/i386-apple-darwin11-ghc-7.8.3' '/usr/local/share/doc/ghc/html/libraries' 'v p'  
make[1]: *** [install_packages] Illegal instruction: 4
make: *** [install] Error 2

If you try to use the (incomplete) built version of ghc-ios after this, you get this error:

$ ghc-ios .travis/helloworld.hs
/Users/travis/build/themasterchef/ghc-ios-scripts/ghc-ios: line 31:  2723 Illegal instruction: 4  arm-apple-darwin10-ghc $ghcargs -threaded -staticlib -outputdir build/arm -o build/arm/$baseFileName -pgmlibtool libtool-quiet $targetFile -stubdir .
The command "ghc-ios .travis/helloworld.hs" exited with 132.

Full trace available at https://travis-ci.org/themasterchef/ghc-ios-scripts/builds/54758122

[marciok]

Cool, good job !
I'm going to try to make that build work on travis.

[chriskilding]

A little digging via here suggests that Illegal instruction: 4 means there is a compiler problem:

"Illegal instruction: 4 usually indicates that there is a platform / compiler mismatch. [library] must be compiled with clang++ and not g++ on OS X."

Sure enough, looking down the log of this failed build, as soon as it starts building ghc for iOS devices we see that GCC/G++ is being used somewhere...

GHC build  : arm-apple-ios
GHC host   : arm-apple-ios
GHC target : arm-apple-ios
checking for perl... /usr/bin/perl
checking for a BSD-compatible install... /usr/bin/install -c
checking whether ln -s works... yes
checking for gsed... sed
checking for gcc... /usr/bin/gcc
checking for ld... /usr/bin/ld
checking for gcc... /usr/bin/gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes

...although curiously GNU ld is not being used:

checking whether ld is GNU ld... no
checking whether ld understands --build-id... no
checking whether ld understands -no_compact_unwind... yes
checking whether ld understands -filelist... yes 

If I have understood what the make process is doing correctly, ghc-ios-scripts might need some more overrides to force the use of clang in the Travis CI environment?

[chriskilding]

After more hackage, I have specifically removed the jobs: ncpus line in the travis environment which is normally the cause of the make install / illegal instruction error. The error still happens on travis. So I'm guessing that this known issue in the cabal config is not the reason for the error, and that something else is at work.

It is the same error as in this issue so could be related.

[chriskilding]

More build system information...

The Travis CI environment (where ghc-ios fails) gives us:

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.9.5
BuildVersion:   13F34

My own Mac Mini (where ghc-ios works) gives us:

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.10.2
BuildVersion:   14C1514

Perhaps either the ghc cross compiler, or the bottom level ghc being used to compile it, are compiling using the OS X 10.10 Xcode SDK available on travis, and then trying to execute on 10.9 - which will fail. In which case a -mmacosx-version-min=10.9 flag may be in order.

[chriskilding]

Good news:

• the bootstrap GHC is using clang after all, when it compiles its own Haskell programs
• brew's package list is really old out of the box so we need to update it to get a (much newer) bootstrap GHC
• the Xcode default is to use an LLVM based on 3.5svn. This is much newer than the 3.1 release that used to generate bad code from Haskell input (or so says the readme) so we probably don't need to install an old version of LLVM any more!

Bad news:

• the min OS X version flag does nothing
• we still get the Illegal instruction 4 error on Travis

[chriskilding]

@marciok how's your work on enabling a travis build getting along? I'm pretty much out of ideas on why this stubborn illegal instruction failure keeps popping its head up, as I don't know the ins and outs of GHC's build system particularly well.

[marciok]

@themasterchef I haven't made any progress so far 😞
This weekend I will but more effort to see if I can solve.

[chriskilding]

Good news - it looks like the Travis team may be introducing OS X Yosemite on its Mac build system very soon (maybe even next week - see http://blog.travis-ci.com/2015-04-09-this-week-in-travis-ci/ for information), which will allow us to identify whether this is a Mavericks specific issue.

(If indeed the illegal instruction comes from 10.10 targeted code executing on 10.9, as opposed to say ARM targeted code on x86_64.)

[chriskilding]

@marciok I've had another go at this today, electing this time to build from the vanilla GHC source as the customised iOS GHC source is now far out of date. I then inject certain required things like build.mk from the installghcios.sh script after downloading the source.

By the looks of travis, it gets much further through compiling the cross-compiler now, though there are some weird bugs:

• build.mk is set to quick-cross or perf-cross, which should have set the Stage1Only env variable. Somewhat defying all logic, when we run this on Travis it still tries to compile stage2 and beyond, which obviously can't work due to known issues with stage2 and iOS. Which is probably why the build falls over in the end.
• Lots of warnings near the final stage of the build about not being able to find an LLVM version - possibly related to this bug https://ghc.haskell.org/trac/ghc/ticket/7143#comment:18 where OS X oh-so-helpfully doesn't like the -fllvm flag?
• Lots of warnings about not being able to find .hi files (although it ignores all these warnings and ploughs on with compiling the cross-compiler regardless)

Would be useful to know if I am just independently rediscovering the raft of errors you guys have already discovered when compiling vanilla GHC for iOS, or if I have found new and more interesting (promising?) errors.

[chriskilding]

Closing due to inactivity