Ticket #374 (closed bug: fixed)

Opened 10 years ago

Last modified 10 years ago

compile error on powerpc64-unknown-linux-gnu

Reported by: visteya Assigned to:
Priority: minor Milestone:
Component: generic Version: FFADO SVN (trunk)
Keywords: debian powerpc64 predefs.h ELFCLASS64 ELFCLASS32 Cc:
The device the bug applies to:

Description

My system: PowerMac? G5 running Debian wheezy.

scons
scons: Reading SConscript files ...
Checking for a working C-compiler yes
Checking for a working C++-compiler yes
Checking for pkg-config (at least version 0.0.0)... yes
Checking jackd version... 0.122.0
Installed Jack Audio Connection Kit (JACK) supports FFADO setbuffersize API
Checking for libraw1394 (2.0.5 or higher)...    yes
Checking for libconfig++ (0 or higher)...       yes
Checking for libxml++-2.6 (2.13.0 or higher)...         yes
Checking for libiec61883 (1.1.0 or higher)...   yes
Checking for lrint(3.2) in C library m... yes
Checking for lrintf(3.2) in C library m... yes
Checking whether 'which pyuic4' executes yes
Checking for the python module 'dbus' (cached) yes
Checking for the python module 'PyQt4' (cached) yes
Checking for the python module 'dbus.mainloop.qt' (cached) yes
Checking whether 'xdg-desktop-menu --help' executes (cached) yes
Checking for dbus-1 (1.0 or higher)...  (cached) yes
Checking for dbus-c++-1 (0 or higher)...        (cached) no
Checking for alsa (0 or higher)...      (cached) yes

One of the dbus-headers, the dbus-c++-headers and/or the application
'dbusxx-xml2cpp' where not found. The dbus-server for ffado will therefore not
be built.

Trying to find the system triple: (cached) powerpc64-unknown-linux-gnu
Doing a DEBUG build
Detected DIST_TARGET = powerpc64
Doing a 64-bit PowerPC build
scons: done reading SConscript files.
scons: Building targets ...
building 'config_debug.h' from 'config_debug.h.in'
g++ -o src/DeviceStringParser.os -c -Wall -g -m64 -fPIC -DDEBUG -DENABLE_BEBOB -DENABLE_FIREWORKS -DENABLE_OXFORD -DENABLE_DICE -DENABLE_RME -DENABLE_GENERICAVC -I. -Isrc -I/usr/include/libxml2 -I/usr/include/glibmm-2.4 -I/usr/lib/powerpc-linux-gnu/glibmm-2.4/include -I/usr/include/sigc++-2.0 -I/usr/lib/powerpc-linux-gnu/sigc++-2.0/include -I/usr/include/glib-2.0 -I/usr/lib/powerpc-linux-gnu/glib-2.0/include -I/usr/include/libxml++-2.6 -I/usr/lib/libxml++-2.6/include src/DeviceStringParser.cpp
In file included from /usr/include/inttypes.h:26:0,
                 from src/ffadotypes.h:31,
                 from src/debugmodule/../fbtypes.h:27,
                 from src/debugmodule/debugmodule.h:30,
                 from src/DeviceStringParser.h:27,
                 from src/DeviceStringParser.cpp:24:
/usr/include/features.h:323:26: fatal error: bits/predefs.h: No such file or directory
compilation terminated.
scons: *** [src/DeviceStringParser.os] Error 1
scons: building terminated because of errors.

Writing a small C source reveals the problem to be the -m64 option to gcc:

#include <inttypes.h>
int main(int argc, char* argv[])
{
        return 0;
}

$ g++ -c foo.c -Wall -m64
In file included from /usr/include/inttypes.h:26:0,
                 from foo.c:4:
/usr/include/features.h:323:26: fatal error: bits/predefs.h: No such file or directory
compilation terminated.

I changed the SConstruct:

--- SConstruct  (revision 2426)
+++ SConstruct  (working copy)
@@ -577,7 +577,7 @@
     # build for 64-bit userland?
     if env['DIST_TARGET'] == "powerpc64":
         print "Doing a 64-bit PowerPC build"
-        env.MergeFlags( "-m64" )
+        env.MergeFlags( "-mpowerpc64" )
     elif env['DIST_TARGET'] == "x86_64":
         print "Doing a 64-bit x86 build"
         env.MergeFlags( "-m64" )

and it now compiles.

Attachments

SConstruct (33.5 kB) - added by visteya on 12/07/13 05:58:12.
Revised SConstruct which detects 32-bit userspace

Change History

10/12/13 02:51:01 changed by jwoithe

I would very much like to hear the thoughts and experiences of others running the ppc64 architecture. The manpage of gcc 4.7.1 says this about the "-m64" option:

Generate code for ... 64-bit environments of Darwin and SVR4 targets (including GNU/Linux). ... The 64-bit environment sets int to 32 bits and long and pointer to 64 bits, and generates code for PowerPC64, as for -mpowerpc64.

From my reading, this means that "-m64" is the option which should be used for your environment since you are running Linux. Of course it's clear that it doesn't work for you and I am very interested to know why. Having said all that, I've never run anything on a G5 mac so my experience with the architecture is practically non-existent. My initial reaction is to suggest that maybe there's something wrong with the system header files, or that the system header files for some reason don't like "-m64" even though the environment is one of those explicitly covered according the gcc manpage. Is the gcc manpage inaccurate in this regard?

Can others shed any light on this?

10/12/13 06:54:04 changed by visteya

What prompted me to build ffado from source were these experiences:

I wanted to have ardour running on my G5 with an Echo Audiofire 8. I installed Debian's packages for jackd2, jackd2-firewire, libffado, and qjackctl.

Debian's jackd2 package does not even work on ppc64. It crashes with a Bus Error, which means unaligned memory access. I investigated this, and it is due to the jackd2 developers apparently having been unaware of architectures which are not x86. They use packed structures, which happen to work OK on x86, but cause alignment errors on almost any RISC architecture. There are some tickets about this on the jackd2 github site, mainly from someone who experienced the same crash on ARM architecture. The developers made a patch to fix this for ARM:

#if defined(__arm__)
    #undef POST_PACKED_STRUCTURE
    #define POST_PACKED_STRUCTURE
#endif /* __arm__ */

but no patch for any other RISC architecture!

I added my own similar patch:

#if defined(__powerpc__)
    #undef POST_PACKED_STRUCTURE
    #define POST_PACKED_STRUCTURE
#endif

and was able to build a jackd2 for G5 which does not crash.

However jackd2 still didn't work properly. I think this was actually due to problems with libffado, but that wasn't obvious to me at the time.

After reading the jackd2 tickets and discussion about the use of packed structures, I began to be concerned about the jackd2 developers. So I uninstalled the jackd2 I had built, and installed Debian's jackd1 package. I still could not make any sound, and concluded the problem could be with libffado.

Debian's libffado package is from svn r2171. I decided to try the current svn libffado. That is when I discovered it would not compile, figured out the -m64 problem, and made this ticket.

I think this story might indicate there is no one else except me who ever tried to use jackd2 and libffado on powerpc64. There definitely are a few people in the world running Linux on powerpc64, but maybe none of them ever installed jackd2.

I would agree with you that there is something wrong with the system header files. The 3-line test .c which fails to compile with -m64 demonstrates that. I suppose what would help is if someone else with a G5 but running some other distro and/or gcc/glibc version could say if their system exhibits the same problem with -m64. But I rather doubt such a person will stumble on this ticket, if he exists at all. :)

predefs.h exists on my system, as /usr/include/powerpc-linux-gnu/bits/predefs.h. It is unclear to me why /usr/include/powerpc-linux-gnu is not in the built-in paths which gcc searches for headers. It is also unclear how -m64 affects the inclusion of the predefs.h file.

I am curious to know how the Debian developer was able to compile libffado for ppc64. I checked the source for the Debian package, and don't see any patch to remove the -m64 for ppc64. Maybe the developer used a different version of gcc/libc to build the package, one which doesn't exhibit the -m64 header file problem. The default gcc in wheezy is 4.6.3 which is what I used.

My main goal is to just make sound come out of the Audiofire, something I have not yet achieved. It will take some more hours to figure out the answers to all these questions and problems.

10/15/13 04:45:22 changed by jwoithe

Thank you for your detailed description.

Since r2171 has been packaged by Debian for your system one would assume that it compiled for the packager. Since r2171 SConstruct has only had a handful of changes (detection of jack versions in r2176, r2189 and r2383, revising the minimum libraw1394 version in r2203, updating the reported version number in r2215 and r2218). From this I can only conclude that the "-m64" option was in place for the ppc-linux architecture in r2171 and that at the time of packaging it compiled correctly.

r2171 dates from June 2012, so it's possible that the package was compiled on a slightly different system to what you have now. Even so, "-m64" should still work on your system and the fact that it clearly has trouble related to header files suggests that something is wrong at the system level. Perhaps - as you suggest - the problem is with the gcc package in that the build in search path is insufficient. Again, it would be good to hear from the packager about these issues.

I'm not convinced that changing the flag in scons is the correct answer; my reading of the gcc manpage suggests that "-mpowerpc64" is in fact wrong for Linux, even though it seems to produce a working libffado for you. Having more information about these flags on the ppc Linux architecture would be handy at this point, and would allow us to determine the best way of addressing the problem. In the meantime, what are your thoughts as to how we should approach the issue?

12/04/13 21:24:17 changed by visteya

  • keywords changed from powerpc64 predefs.h to debian powerpc64 predefs.h ELFCLASS64 ELFCLASS32.

I have managed to get sound out of my Audiofire8 connected to the PowerMac? G5.

The compiler problem where the -m64 option to gcc caused a header file to not be found ( bits/predefs.h ) was resolved by simply installing the Debian packages g++-multilib and gcc-multilib. I reverted my above-mentioned alteration to SConscript ( -mpowerpc64 reverted back to -m64 ).

However, though the build proceeded past the point where it previously failed due to the missing header file, and compiled all the sources, the link failed spectacularly with hundreds of errors such as:

/usr/bin/ld: powerpc:common64 architecture of input file `src/devicemanager.os' is incompatible with powerpc:common output
/usr/bin/ld: powerpc:common64 architecture of input file `src/ffado.os' is incompatible with powerpc:common output
...
/usr/local/src/ffado-svn/src/libavc/descriptors/avc_descriptor.cpp:105: undefined reference to `operator new(unsigned long)'
src/libavc/descriptors/avc_descriptor.os: In function `AVC::AVCDescriptor::load()':
/usr/local/src/ffado-svn/src/libavc/descriptors/avc_descriptor.cpp:313: undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> >::substr(unsigned long, unsigned long) const'
...

And fortunately a big clue at the very end:

/usr/bin/ld: src/devicemanager.os: file class ELFCLASS64 incompatible with ELFCLASS32
/usr/bin/ld: final link failed: File in wrong format
collect2: ld returned 1 exit status

So, it turns out that scons guesses (via admin/config.guess) that it should build a 64-bit binary:

Trying to find the system triple: (cached) powerpc64-unknown-linux-gnu
Doing a DEBUG build
Detected DIST_TARGET = powerpc64
Doing a 64-bit PowerPC build

I suppose this is a somewhat reasonable guess, given that uname -m returns ppc64. However, in fact Debian powerpc64 is a 32-bit userland and a 64-bit kernel. All the packaged libraries which ffado attempts to link to are 32-bit, e.g.:

$ file /usr/lib/powerpc-linux-gnu/libraw1394.so.11.0.1
/usr/lib/powerpc-linux-gnu/libraw1394.so.11.0.1: ELF 32-bit MSB shared object, PowerPC or cisco 4500, version 1 (SYSV), dynamically linked, BuildID[sha1]=0xa8f27cf08b376d0100e88eb2c30304c6f986c5d3, with unknown capability 0x41000000 = 0x13676e75, with unknown capability 0x10000 = 0xb0401, stripped

Fortunately, one can override the guessing of SConstruct by passing an explicit DIST_TARGET to force a 32-bit build:

scons DIST_TARGET=powerpc

This incantation, I am happy to report, resulted in a successful build.

The next challenge was that the Debian package of ardour2 does not work with jackd/libffado I built because ardour passes an incorrect command line option to jackd: -d ffado instead of -d firewire. This problem has been fixed in ardour git, so I had to build ardour from source. I decided to build ardour3 from git checkout. This turned out to not work out of the box, because the ardour3 configure script does not even consider ppc64 as a possible output of uname -m. It ends up thinking it is building for i686 and tries to build SSE optimizations which cause errors and fail. But I fixed the ardour3 wscript and got it to build, and run.

I suppose there's some lessons of this story. All I wanted to do was make sound on a PowerMac? G5 running plain Debian. This wish turned out to be quite difficult to achieve.

* Without some years of software development experience, I would not have been able to figure out how to fix all these problems I ran into.

* The whole world is an x86. Most software developers don't think, or maybe don't even know about other architectures. So if you want things to just work with minimum trouble, just get an intel machine like everybody else in the world.

Probably that's what I should have done. I question whether the time I spent on this will benefit anyone else. I should have just gotten an intel machine instead...

I don't know if anyone sees value in fixing the SConstruct script so it does the right thing on Debian ppc64. I don't even know what would be the correct approach, as I don't know what other Linux distros still support powerpc64 nor whether they have 32-bit or 64-bit userlands.

So maybe at least adding a sentence to the README would be a good idea:

"If you are building for powerpc64 on Debian, you must invoke scons with DIST_TARGET=powerpc"

Thanks for reading all this.

12/05/13 15:17:04 changed by jwoithe

Thank you for your extensive work and feedback about this issue. As it turns out another ppc64 user has been recently working with the source. They encountered some slightly different issues to those you report (most notably the "-m" flag was not being passed to gcc at all in their environment). The practical upshot of their experience lead to the same conclusion as you reached: issues surrounding the fact that this distribution is running 32-bit userland on a 64-bit kernel. As you pointed out, the output from uname certainly seems to imply a 64-bit environment. This is clearly a trap for the unwary.

I'd like to respond to a few specific statements in your post.

Without some years of software development experience, I would not have been able to figure out how to fix all these problems I ran into.

I am happy to acknowledge this and express my gratitude that you were willing and able to work through the issues.

The whole world is an x86. Most software developers don't think, or maybe don't even know about other architectures. So if you want things to just work with minimum trouble, just get an intel machine like everybody else in the world.

It is probably a little harsh to say that "Most software developers don't think, or maybe don't even know about other architectures." On the contrary, I expect that many developers are very much aware of other architectures and try their best to accommodate them. However, at least in the case of FFADO and JACK it's important to realise that all the developers are, as far as I know, volunteers: we do this in our spare time and no one pays us to do it. The practical effect of this is that we can't afford to invest in a large collection of platforms to test every single combination of hardware and software. Finding the time to run all these tests would also be a problem. Speaking for myself, I would dearly like to have FFADO support everything that's out there, but I simply don't have the money to purchase hardware just to facilitate this testing. Most others - if not all - are in a similar situation. This is why we rely on others such as yourself to test on the hardware you have access to and provide feedback so any difficulties might be addressed. Again, we are extremely grateful that you have been able to identify the problems on your particular setup.

Returning to the issue at hand, I do see value in addressing the problem within SConstruct. Unfortunately many people gloss over README files, and a comment in there about DIST_TARGET - while completely valid - will likely be missed by most people. If SConstruct could handle this automatically it would be a good thing.

In order to do this, it seems we need some sort of test which reveals whether the default userland on a ppc64 system is 32 bit or 64 bit. I'm thinking of something which perhaps checks the type of a selected default library and switches based on that. Using a combination of "file" and "grep 32-bit" on /lib/libc.so.6 or /lib/ld-*.so would probably work but I'm sure there's a more elegant way using one of the binutils. Assuming the existence of something like this, we could then put a test in for a 32-bit environment in the case where a ppc64 was detected and override DIST_TARGET accordingly. Given a sufficiently robust test, adding something like this would not be difficult and I would be happy to do so.

I will look into the feasibility of this in the next few days unless you happen to be in a position to propose a solution in the meantime.

Aside from this SConstruct issue, are there any other things we ought to address to make the situation on ppc64 easier?

12/05/13 18:15:39 changed by visteya

I didn't intend my comment about developers as a criticism of FFADO developers. I'm sorry for the harsh tone.

I agree it would be good to fix SConstruct. I would be willing to work on this. Your suggestion of file | grep was also my initial idea. But for Debian, at least, I am not sure the solution can be that simple. On 64-bit processors, Debian supports what they call "multiarch", meaning both 64-bit and 32-bit libraries and headers can be installed and one can run/build either 64-bit and 32-bit programs.

https://wiki.debian.org/Multiarch

I am not greatly educated on all the details of how this works in Debian, having only done some superficial reading, but I will read more.

On amd64 (x86_64), 64-bit is the "default" and 32-bit is the "alternate" you get when you install the (optional) multiarch support. Whereas on powerpc64 it seems to be the other way around. I'm sure there is some historical reason for powerpc64 being this way but I don't know what it is.

It will be more than "a few days" before I have something tested enough to propose. I do have a regular job which eats a lot of time, but I will be glad to work on it nonetheless. If you would prefer to do something about it sooner yourself, just let me know.

12/05/13 18:29:53 changed by jwoithe

Thanks for the reply and no need to apologise. My comments were as much to document the current situation for future readers of this ticket.

Since you are willing to work on this I am more than happy for you to do so. It will be much easier for you to do this since you have the hardware and software environment which is affected.

It is true that multiarch complicates things, not so much because it exists but because the default width is different depending on the architecture (x86, ppc). In the first instance I think the default behaviour of our build environment should target the default width, but rather than assume what this might be we should try to come up with a test which can confirm it.

The "file/grep" solution may work so long as there was a definitive file name we could check which was guaranteed to apply to the "default" compilation environment. Failing this we may have to arrange for a small trivial C program to be compiled using default flags; this could then be tested using file/grep (or some binutils utility yet to be identified) to see what width it is compiled for.

Regarding the file/grep, I'm fairly certain that one of the binutils programs has a flag which can be used to report the object type (at least as far as 32/64 bit goes). If this provides a more reliable output compared to "file" it might be worth looking into it. In the meantime, "file" should be fine.

12/07/13 05:58:12 changed by visteya

  • attachment SConstruct added.

Revised SConstruct which detects 32-bit userspace

12/07/13 06:13:16 changed by visteya

I have attached a revised SConstruct which builds ffado properly on a Debian powerpc64 machine and seems to work properly on a x86_64 machine.

(follow-up: ↓ 10 ) 12/08/13 18:14:54 changed by jwoithe

Thanks for sending this though. I will go through it in the next few days and verify that it works as expected on pure 32-bit systems. Your code is certainly far more comprehensive than what we had before; hopefully it does the trick.

One question: you altered the shell at the top of SConstruct from "/usr/bin/python" to "/usr/bin/env python". What was the reason for this? I would have expected "/usr/bin/python" to be valid on pretty much every system. Or is this a "just in case" preemptive move?

(in reply to: ↑ 9 ; follow-up: ↓ 11 ) 12/09/13 06:33:47 changed by visteya

Replying to jwoithe:

One question: you altered the shell at the top of SConstruct from "/usr/bin/python" to "/usr/bin/env python". What was the reason for this? I would have expected "/usr/bin/python" to be valid on pretty much every system.

"/usr/bin/python" would be the Python interpreter as installed by most Linux packaging systems. For most non-Linux systems it is not, nor is it the case for a Linux system where Python is built from source using the default PREFIX.

There are valid arguments to be made for both incantations, but for scripts which are intended to work anywhere Python works, "/usr/bin/env python" is much more likely to do the right thing. When I realized not all the world is Linux, I made a habit of using "/usr/bin/env python" for my own scripts.

However, my change of the first line of the SConstruct script to "/usr/bin/env python" was in fact not an improvement. The proper thing to do would be to remove the line entirely, since the script cannot work when invoked directly, but only when executed by scons itself. Leaving it there does no harm, but it is somewhat misleading.

(in reply to: ↑ 10 ) 12/09/13 14:36:12 changed by jwoithe

Replying to visteya:

There are valid arguments to be made for both incantations, but for scripts which are intended to work anywhere Python works, "/usr/bin/env python" is much more likely to do the right thing. When I realized not all the world is Linux, I made a habit of using "/usr/bin/env python" for my own scripts.

Thanks for the background. I'm not overly familiar with the intricate details of python so the above information is useful. Of course in FFADO's case Linux is the only platform it will be usable on in its current form so the idea of cross platform support is largely moot. Even so, if there is a convention of sorts we should possibly follow it.

However, my change of the first line of the SConstruct script to "/usr/bin/env python" was in fact not an improvement. The proper thing to do would be to remove the line entirely, since the script cannot work when invoked directly, but only when executed by scons itself. Leaving it there does no harm, but it is somewhat misleading.

This sounds fair - removing the line entirely would seem to be the right thing to do. I'll split the removal of the line out from the rest of the changes since they are really independent. Thanks again for the information on the rationale behind this change.

I'll follow up on the remainder of the changes once I get a chance to test them (hopefully this evening).

12/10/13 04:38:54 changed by jwoithe

I've committed the changes as discussed. r2463 fixes the #! line while r2464 is the bulk of your patch. I've put a call out on ffado-devel for this to be tested so we can confirm there are no regressions for other architectures. Note that r2465 is a dummy commit to correct the attribution in r2464 - thanks to a brain fade I accidently attributed the changes in r2464 to the wrong person. Sorry about that.

It would be good if you could obtain r2464 (or later) and verify that the code now in trunk does the right thing on your system. If this proves successful I think this ticket can be closed as "fixed".

12/20/13 01:53:44 changed by jwoithe

  • status changed from new to closed.
  • resolution set to fixed.

Since committing to trunk there have been no regression reports. Tests have been done on x86 32 and 64 bit systems and things seem to be OK there. It therefore seems appropriate to call this as fixed.