Port FFADO to the new firewire stack

From what I've seen (taking juju from some/the? git repo) it wasn't that hard. It looks as if it at least contains a compatibility mode for raw1394 stuff.

And that stuff is only missing the read_cycle_bla function. My try at creating that function failed miserably though...

The main idea of porting is of course that we can avoid the compatibility layer. The programming model of the new stack (should) make more sense for our application.

First tests of Juju on 2.6.27.1 with libraw1394-2.0.0 and ffado r1359:

Ieee1394Service::initialize complains about lacking raw1394_read_cycle_timer support. This isn't really true, it's a matter of initialization. For the old stack, the raw1394handle contains a file descriptor which gets initialized to fopen(/dev/raw1394) from raw1394_new_handle_on_port(). Later, ioctl(handle->fd, RAW1394_IOC_GET_CYCLE_TIMER, &) is called.

For the new stack, things are pretty much the same, except that handle->fd is now handle->iso.fd and initialized by raw1394_iso_xmit_init(). In other words, the raw1394_read_cycle_timer() calls can only succeed after ISO initialization.

With the current code, strace shows:

ioctl(4294967295, 0x8010230c, 0x7fffdeb43bc0) = -1 EBADF (Bad file descriptor)
clock_gettime(CLOCK_MONOTONIC, {8301, 175520540}) = 0
futex(0x7f6dd6b21078, FUTEX_WAKE_PRIVATE, 1083011clock_gettime(CLOCK_MONOTONIC, {8301, 175808565}) = 0
futex(0x7f6dd6b21078, FUTEX_WAKE_PRIVATE, 108301175808: ESC[31mWarning (ieee1394service.cpp)[ 252] initialize:  This system doesn't support the raw1394_read_cycle_timer call.   

Obviously, the fd is invalid (initialized to -1).

Second: the ISO streaming does not start. IsoHandler::prepare expects E_Initialized, but the state is 3, which seems to be E_Running. See the attached log for details, if interested.

Since the new libraw1394-2.0.0 is backward compatible with the old stack, distributions will include it. I'm somewhat involved in the Debian Juju migration, talking to both, the libraw1394 maintainer and to the Juju kernel guys. We've fixed a libraw1394-2.0.0 segfault, I'll also attach the patch until it gets included upstream.

However, moving to the new stack should IMHO be accompanied by switching to the new API. There's no need to hurry, since Juju won't be fully featured before Linux 2.6.29. On the other hand, there's no use in developing for the old stack for years. One could expect 2.6.29 within six months. It might be worthwhile to keep and maintain the 2.0 release branch while changing the trunk to Juju.

I feel, ffado-3.0 is way too late for this. It should be targeted as the follow-up release to the 2.0.

First teststreaming3 run with Juju von 2.6.27.1

patch against libraw1394-2.0.0, fixing a segfault

Replying to adi:

First tests of Juju on 2.6.27.1 with libraw1394-2.0.0 and ffado r1359: Ieee1394Service::initialize complains about lacking raw1394_read_cycle_timer support. This isn't really true, it's a matter of initialization. For the old stack, the raw1394handle contains a file descriptor which gets initialized to fopen(/dev/raw1394) from raw1394_new_handle_on_port(). Later, ioctl(handle->fd, RAW1394_IOC_GET_CYCLE_TIMER, &) is called. For the new stack, things are pretty much the same, except that handle->fd is now handle->iso.fd and initialized by raw1394_iso_xmit_init(). In other words, the raw1394_read_cycle_timer() calls can only succeed after ISO initialization. With the current code, strace shows: {{{ ioctl(4294967295, 0x8010230c, 0x7fffdeb43bc0) = -1 EBADF (Bad file descriptor) clock_gettime(CLOCK_MONOTONIC, {8301, 175520540}) = 0 futex(0x7f6dd6b21078, FUTEX_WAKE_PRIVATE, 1083011clock_gettime(CLOCK_MONOTONIC, {8301, 175808565}) = 0 futex(0x7f6dd6b21078, FUTEX_WAKE_PRIVATE, 108301175808: ESC[31mWarning (ieee1394service.cpp)[ 252] initialize: This system doesn't support the raw1394_read_cycle_timer call. }}} Obviously, the fd is invalid (initialized to -1).

This means that the juju implementation is not backward compatible to the old one. There is no reason to tie the cycle timer read to an ISO descriptor since it's not related to an ISO stream. It's a global function of the card, not the stream.

The current implementation in FFADO also reflects this, and I don't really want to change this.

Second: the ISO streaming does not start. IsoHandler::prepare expects E_Initialized, but the state is 3, which seems to be E_Running. See the attached log for details, if interested.

The juju version doesn't seem to be iterating the handlers. The packet callback handlers are not called.

Since the new libraw1394-2.0.0 is backward compatible with the old stack, distributions will include it. I'm somewhat involved in the Debian Juju migration, talking to both, the libraw1394 maintainer and to the Juju kernel guys. We've fixed a libraw1394-2.0.0 segfault, I'll also attach the patch until it gets included upstream.

Obviously the new implementation is not backward compatible to the old one yet. If that would be true, FFADO would just work. Therefore I think libraw should be fixed instead of us implementing quirks. I hope the linux1394 maintainers think the same. Currently I don't have the time to look into it myself.

However, moving to the new stack should IMHO be accompanied by switching to the new API. There's no need to hurry, since Juju won't be fully featured before Linux 2.6.29. On the other hand, there's no use in developing for the old stack for years. One could expect 2.6.29 within six months. It might be worthwhile to keep and maintain the 2.0 release branch while changing the trunk to Juju.

Something like that is the idea, but priority is on releasing 2.0 at the moment. After all, if libraw is 100% backwards compatible it doesn't matter what stack the user runs. The main issue with switching to juju is that it will take quite some time before it has propagated to the majority of users.

I feel, ffado-3.0 is way too late for this. It should be targeted as the follow-up release to the 2.0.

There is no reason to put juju support in the 2.0 series, since the old style should keep working, even on juju systems. As far as I can assess it, the changes required to make efficient use of the juju native API will be fairly extensive. The main issue is that the current implementation is built around the per-packet callback paradigm, where the new style would be based around a per audio-buffer callback. One of the challenges with that is that the number of packets that constitute one audio buffer is variable. The scheduling of hardware interrupts will have to be done taking that into account, which is not really accounted for with the current implementation.

Once we start doing such extensive changes, we will have a hard time maintaining compatibility with old-style systems. And since not all current distro releases are already including juju, there will be old-style systems for years to come. I don't want to ask people to upgrade their distro to upgrade their FFADO.

BTW: Thanks for looking into this.

Pieter

This is just the stack between user space <-> libraw <-> and the new linux OHCI driver?! ... it will still be limited to supported hardware -- or perhaps it will have a smaller set of supported hardware initially?

Juju run with fixed cycle timers and fixed ARM segfaults in libraw1394 (upcoming 2.0.1)

Small status update: with the current git revision of libraw1394, all things we've addressed so far are fine: read_cycle_timer works, channel allocation works, arm_handler doesn't segfault.

I've attached a new debug log. For now, I wonder why "setSplitTimeoutUsecs" and "getSplitTimeoutUsecs" both fail. To quote the log around line 53:

158771118453: Debug (ieee1394service.cpp)[ 937] getSplitTimeoutUsecs: reading SPLIT_TIMEOUT on node 0x1...
158771118525: Debug (ieee1394service.cpp)[ 541] readNoLock: raw1394_read failed: node 0xFFC1, addr = 0x0000FFFFF0000018, length = 1
158771118556: Debug (ieee1394service.cpp)[ 941] getSplitTimeoutUsecs: read of CSR_SPLIT_TIMEOUT_HI failed

Can't see why this fails on the new stack. (I've attached oldstack.gz for comparison, same read, same address, but working)

Run with old ieee1394 stack, just for comparison

Hello,

I am using a GNU/Debian SID with linux 2.6.29 or 2.6.30. These stock Debian kernels only offer the new firewire stack. I use Ardour and recently invested in a Edirol FA-66.

What is the current state of support of the compatibility layer? I also use Kdenlive/MLT video editor which uses the compatibility layer and works great. I discussed with the developers who told me migration to the compatibility layer took only a few hours. I cannot help as I am not a developer myself.

As a result, in Debian, jackd is still linked against freeBOB. This means that unless you recompile the kernel, jackd and a bunch of other dependant sofwares, you are not going to use FFADO.

To reach a larger audience, IMHO, it would be nice to port FFADO to the new compatibility layer.

What do you think? I lost more than 40 hours trying to make make FFADO work and I can recompile. But some people cannot and are not going to switch to special Ubuntu distros, just for the sake of running FFADO.

Any help or information would be greatly appreciated. When FFADO is compiled against the new kernel interface, I can start FFADO mixer. But when I start Jackd it dies after a few seconds. Do you know how to fix this?

Also, because the old stack may not be supported in future Linux kernel, I took the liberty to switch this ticket to FFADO 2.x. Correct me if I am wrong.

Kind regards, Pascal

Hi!

Pascal (pkinsbourg), please contact me directly via E-Mail (adi@drcomp.erfurt.thur.de). I'll answer your questions, especially those related to Debian.

This ticket is intended for tracking development process regarding the new stack, and all entries are automatically forwarded to the kernel 1394 mailing list. I therefore want to keep it clean, it's not the right place for end-user support.

Thanks.

My experience is that we use the stack in a very different way than the 1394 video people do. The subset used by video is fairly small. For example they rarely use ISO transmit, as the most interesting direction for video is into the computer. Hence not many cameras support recording from 1394. FFADO on the other hand wouldn't be very useful without ISO transmit to the soundcard. It's probably not a big surprise that the ISO transmission is what causes the majority of our headaches with the kernel layer.

Another thing is that last time I checked the new stack doesn't support iso traffic on multiple channels at the same time in the same direction. I have to admit that I haven't checked recently, but that alone makes it unsuited for FFADO.

There are two ways where I see FFADO working with the new stack: 1) the compatibility layer is fixed such that it is effectively 100% transparent 2) we rewrite FFADO to bypass libraw1394 as it's API model doesn't suit realtime audio very well.

For (2) we have two options: either we do this in userspace, or directly in the kernel. At somebody is working on a kernel space FFADO driver that uses the new stack. But that will be 3.0, as a kernel-space new-stack driver is basically the definition of FFADO 3.0.

In the mean time I'm afraid that you'll have to complain to those that broke things.

Hello,

Well, I jump on this thread to tell my story: * I am from Kdenlive team and I consider myself mid-level in administration. * I bought a firewire device on the information "Full support" on FFADO. * Then I discovered support for the new stack was incomplete. * I lost several days trying to compile the Debian Kernel 2.6.29 and 2.6.30 which resulted in a kernel panic on reboot.

FFADO has "perfect" support for some devices, but most distributions lack support for the old kernel stack. So FFADO is a heck to install, especialy when linux sources need patches.

Finally, I think there should be a minimal way to make faado work with the recent kernel stack and this should be a priority. You are loosing audience and GNU/Linux users could be using FFADO. At present, GNU/Linux only supports USB audio and PCI audio, this is the truth.

Now I am thinking about reselling my audio device. What should I do?

Kind regards, Jean-Michel

1) the compatibility layer is fixed such that it is effectively 100% transparent

This could be the best option knowing that several users may encouter severe problems, including Debian GNU/Linux. At present, people willing to use FFADO have to install a custom distribution, with custom kernel, custom jack (firewire plugin), etc ...

In Debian, FFADO packages exist, but FFADO support in jackd is absent. It means no user and it seems useless.

Please find a way to fix this, some of us are in a bad situation. We would appreciate your help.

Kind regards, Jean-Michel

All problems you have can be solved by proper packaging, even the kernel stack issue. So I can only refer you to the people that create the problem, i.e. the people packaging ffado / jackd / the 1394 stack for debian. IMHO this is the wrong place to complain.

I might sound crude here, but we can't take on everything. We don't have the development bandwidth to port FFADO to the new stack at the moment. Our bandwidth is barely sufficient to support FFADO on one distro with one kernel, let alone the moving target of 30 distro's and 2-month kernel release cycles. I'm sorry, but the only way that's going to change is if someone finds significant funding for this project. Until that happens, you'll have to be happy with what you get and might have invest some of your own time into configuring your system such that it works. Or choose a distribution that gets things right with respect to audio+firewire+ffado.

Re comment 7 by pkinsbourg:

I am using a GNU/Debian SID with linux 2.6.29 or 2.6.30. These stock Debian kernels only offer the new firewire stack.

Interesting; until now they enabled only the old drivers, except temporarily in some test kernel packages a while ago.

You could ask the packagers to provide the old drivers too; there are instructions at http://ieee1394.wiki.kernel.org/index.php/Juju_Migration . (The kernel configurator's help text to the new drivers also links there.)

What is the current state of support of the compatibility layer?

The 'compatibility layer' in its narrower sense is provided by libraw1394 v2 and is fundamentally complete. The biggest problem (among other smaller ones) isn't compatibility in a strict sense but rather that streaming through the new drivers stops right after it started. This needs to be fixed; but how and where (in the kernel or outside or both?) is not yet clear.

Also, because the old stack may not be supported in future Linux kernel

The old drivers will remain supported in the mainline kernel until the new drivers are _fully_ capable of what the old ones are. However, distributor kernel packages are a different matter; whether the old or new or both stacks or even none of them is included into distributed kernel packages is each distributor's choice. At the side of the mainline kernel project, we will aim to document better what kinds of hardware are supported by the new drivers and what kinds still require the old drivers, to help distributors choose.

Re comment 9 by ppalmers:

last time I checked the new stack doesn't support iso traffic on multiple channels at the same time in the same direction.

Multichannel IR DMA is not implemented in the new stack. However, multiple IR and IT contexts are of course implemented and available to userspace drivers. At least multiple IR contexts received a little bit of testing by now by the IIDC folks (not much, because other shortcomings in the new drivers until 2.6.29 inclusive made it not quite easy to run multiple streams). The character device file interface requires a 1:1 relationship of iso contexts and file descriptors though (IOW, one open fd for each stream required), but this shouldn't be an issue since I can't see a need for state to be shared between contexts that would demand a shared fd. Correct me if I'm missing something.

In the mean time I'm afraid that you'll have to complain to those that broke things.

That would be mainline kernel driver maintainers (document more clearly what the new kernel drivers can and cannot do in a given release; I posted a Kconfig help text update just this week) and distributors (not providing kernel drivers that are suitable to FFADO, for whatever reasons they might have --- there was also a case a while ago where a switch from old to new only inadvertently happened by accident until somebody noticed).

Re comment 11 by jmpoure:

I lost several days trying to compile the Debian Kernel

Kernel recompilation is difficult, and users should not have to do that if possible. Some special requirements cannot be satisfied by packaged kernels of general-purpose distributions though; then users need to look for 3rd-party packages, a specialized repo, or look for guidance on how to recompile a kernel on their specific distro. However, starting with kernel 2.6.30 in which the new FireWire? drivers became very attractive for a number of use cases (except audio), it should probably be recommended to distributors of compiled kernels that they provide _both_ stacks installed in parallel. Without extra configuration or with explicit configuration to have the new stack active by default, a switch to the old stack will take a user only a few modprobe config file changes instead of kernel recompilation.

most distributions lack support for the old kernel stack.

Correction: So far most distributions provide the old stack only. This starts to change now though.

I think there should be a minimal way to make faado work with the recent kernel stack and this should be a priority.

It is a high priority for me as contributor to the kernel drivers, and as best as I can tell it is also high priority for the FFADO developers. But high priority alone doesn't give fast results; the difficulty and complexity of the matter on one hand and the limited development resources both of the kernel driver project and of the FFADO project make this problem one which will alas not be solved in a short term. Limited resources require triage: Choose items to work on first which can be solved with a known finite and justifiable amount of resources.

JFTR: same with 2.6.31. (just compiled it and saw it failing, no further investigations beyond that)

I.e. asynchronous things work (except set/ getSplitTimeoutUsecs), but isochronous reception doesn't see any packets? Which OHCI controller are you testing BTW?

Exactly. Tests were done with Texas Instruments PCIxx12 OHCI Compliant IEEE 1394 Host Controller.