Last year, I worked on a project
for visomat inc. which involved
interfacing to a LED matrix display so that it could display some
video. We had some latency problems and at one stage we decided to
try out DMX instead of Ethernet as interface techniquce, hoping that
we'd get better results that way. The idea was to feed the LED
matrix, which consisted of 12 power supplies/controllers, with 12
DMX interfaces to make sure that data arrives at the LEDs
synchronously.
DMX is an asynchronous serial protocol running at 250 kbits per
second. I decided to try using an AVR microcontroller running at 16
Mhz and have it bit-bang the data out of a parallel port. For each
bit, 64 machine cycles would be available which should be enough to
set the port bits and also grab incoming data out of the USB
interface that we wanted to use to interface to the host.
I built a prototype of the interface and we made some promising
experiments with it, but in the end we found that the visual latency
problems were not fixed. The problem was really in how the LED
hardware worked and could not be repaired on the feeding side. The
prototype went onto the shelf and we stuck with Ethernet for that
project.
A few weeks ago, I discussed the project with a friend of mine and
we thought that it'd be cool to turn the 16 port DMX interface into a
product. We found someone who would want to use it in an OEM
setting, and I made bringing the hardware to product level my
weekend project.
Good bye AVR, hello ARM
The AVR based solution proved to be problematic, though. Even
though the CPU has enough processing headroom, the lack of RAM
requires that the data is fed into the interface "just in time".
Any variance in the USB input data stream could result in visible
hickups in the DMX streams. Getting it to work seemed like
possible, but require too much work in the end. Thus, I decided to
turn to using more capable hardware which would also give the DMX
interface network connectivity and more local intelligence.
As new platform, I chose the ARM9
based Eddy-S4M
board produced by the Korean
manufacturer SystemBase.
These boards are distributed
by trenz electronic
who I've found to be quick and reliable in the past.
Linux is the standard operating system on Eddy-S4M. I'm not a huge
Linux fan, so I spent some time to find an alternative operating
system, but neither eCos,
nor FreeRTOS
or RTEMS seemed to be ported to the
AT91SAM9260 CPU with a free development system. As I was into
application development, I decided not to port or write an operating
system myself. Running on the bare hardware was not an option
either as I need a working TCP stack in order to interface to the
host.
SystemBase calls their Linux port "lemonix". It appears to be a
relatively plain linux-2.6.20 with real-time
patches, BusyBox and a bunch
of C applications to control startup and configuration. The
userland and kernel source code is available from SystemBase after
registration, but the kernel tree that they make available is
incomplete and needs to be augmented by some files that are
available from
a Google group so
that it can be built. Getting the cross compilation environment to
run on my Ubuntu 9.10 box worked flawlessly, and after removing the
whole Eddy application stuff and most of the SystemBase drivers from
the file tree I ended up with a relatively sane Linux that I could
cross compile and install on the Eddy hardware through tftp.
Making it fast
Back to the DMX interface, I was now faced with a system that gave
me about 400 machine cycles for each DMX bit sent. That should be
plenty of time to get out some bits to the ports. The bit timing
would be achived by a timer interrupt every 4 microseconds. Even
with interrupt handling overhead, this should leave enough headroom
for TCP processing.
The ARM architecture specifies two interrupts. The normal, vectored
interrupt system is used by Linux and mapped to its own, portable
interrupt architecture, ignoring the vectoring facilities that the
hardware provides. The second, fast interrupt system (FIQ) that the
ARM provides is not used by Linux, and it seemed like a good fit to
my requirements. The FIQ is interesting because when the FIQ
handler is entered, the CPU automatically switches six registers to
a distinct bank. These six registers can then just be used without
the need to save or restore them to the stack. Even though the FIQ
is not used by Linux itself, ARM-Linux provides for an interface so
that drivers can use it. FIQ support was missing in lemonix, but it
was trivial to backport.
Writing a FIQ interrupt in ARM assembly was straightforward. The
driver bottom half set up the FIQ register set and enabled the timer
interrupt, the FIQ handler set the port bits and incremented the
pointers accordingly.
Reducing jitter
When looking at the output ports with the logic analyzer, though, I
could see variations in bit edges in the order of one microsecond,
which is beyond what the DMX receivers would be able to tolerate.
These variations were visible even when the FIQ handler just set the
port bits without writing any real data. The source for this
jitter, as it turned out, was virtual address translation. The ARM9
CPU includes a MMU and all software, including interrupt handlers,
uses virtual addressing, even to access I/O ports. The virtual
address maps are stored in SDRAM due to their size, and the hardware
automatically traverses these maps if an address can not be found in
the translation lookaside buffers in the MMU. So in the FIQ
handler, when the output port address was not present in the TLB,
the MMU would access the SDRAM, and SDRAM random access is rather
slow.
Thus, I had to change the FIQ handler so that it never accesses the
SDRAM, either directly (by reading the data buffer) or indirectly
(by causing TLB misses). The AT91SAM9260 CPU has two internal SRAMs
that are accessible in two machine cycles, and I use one of those as
a data buffer. In order to prevent TLB misses, indivdual TLB
entries can be locked down so that they're never removed by the MMU
automatically. Thus, my driver locks down the I/O and the SRAM
buffer addresses that the FIQ handler accesses.
Reducing jitter even more
Even with the TLBs locked down, I still saw some jitter in the
leading edges of the DMX bytes. The cause for this were variations
in interrupt response times, as that depends on the instruction
being interrupted. The variations were below one microsecond, but
that was long enough to bring the serial decoding routine in my
logic analyzer out of sync. Wanting to play it safe, I decided to
remove that jitter as well by synchronizing on the timer value
inside of the FIQ handler: Instead of just banging out the next bit
as fast as possible, the FIQ handler now waits until the
free-running timer that triggered the interrupt reaches a certain
value. That way, the effective FIQ response time is made mostly
constant.
Wow, 2010!
In the last few years, I have been using various AVR CPUs for my
embedded projects. It is a great CPU that is easy to use and fast,
and the advent
of LUFA and
Teensy made my life a lot
easier, as I was freed from USB serial dongles and driver
installation. But then, being able to process serious amounts of
data is nice, too, and this is were 8 bits just don't suffice. ARM
based boards are cheap nowadays, and I'm looking forward to
embedding JavaScript or maybe even Common Lisp in one of my future
projects. When speed is needed, I can always fall back to C and
assembler.
