SONiVOX Ports EAS Portable Audio Package to Tensilica’s HiFi 2 Audio Engine

SONiVOX, formerly known as Sonic Network, focuses on audio synthesis and related applications for portable products, where the overarching goal is low memory footprint and minimal power dissipation (high performance at low clock rates). The company offers its AudioINSIDE EAS (Embedded Audio Synthesis) technology for several leading microprocessors and DSPs and has ported EAS to Tensilica’s HiFi 2 Audio Engine and the Diamond 330HiFi audio processor to achieve a small memory footprint and low operating clock rates while keeping its EAS source code in C.

The EAS package supports a broad range of audio file formats including:

• General MIDI Level 1 (GM1)
• MIDI Type 0 and MIDI Type 1
• General MIDI Lite (GML)
• Scalable Polyphony MIDI (SP-MIDI)
• Synthetic Mobile Application Format (SMAF), including ADPCM
• Downloadable Sounds (DLS) and Mobile Downloadable Sounds (mDLS)
• Mobile eXtensible Media Format (mXMF)

The package’s core is a high-quality MIDI synthesis engine (shown in Figure 1), which is based on the Downloadable Sound Level 2 (DLS2) MIDI standard. The EAS’ MIDI synthesis engine features a MIDI parser; pitch, volume, and filter modulation; MIDI performance controllers; and several other sound-modification features. The synthesis engine handles a broad range of sample from 8 to 44 kHz to maximize the audio performance while meeting various memory-cost and power-level targets.

Figure 1. SONiVOX's MIDI synthesis engine.

The DLS synthesizer uses a number of wavetables for generating sound. SONiVOX offers several wavetables, based on recorded source material that now amounts to more than a terabyte of data. The company records the monaural and stereo source material for its wavetables using a 48-kHz sample rate at 16-24 bit resolution. Because its wavetables are produced in house, SONiVOX owns all of the copyrights to the wavetables, an important consideration when selecting a synthesis vendor.

SONiVOX’ primary wavetable is the General MIDI (GM) wavetable, which consists of 128 melodic instruments and 47 drum instruments, organized into one drum kit. This GM wavetable complies with General MIDI Level 1 and SP-MIDI. It is available in EAS (SONiVOX), DLS, and Soundfont formats in multiple sample rates and sample resolutions. In addition, SONiVOX offers regionalized wavetables (Chinese, Korean, Japanese, Latin American, Middle Eastern, and African) and stylized, genre-specific wavetables (FM Synth, Techno-Dance, and Industrial Rock). The regionalized and stylized wavetables can be used instead of or in conjunction with the GM wavetable.

In addition to the wavetable-based MIDI synthesizer, the EAS package includes a number of audio-processing effects:

• Sonic Booster – adds variable amounts of compression, expansion, and loudness to inject energy into the sound and to improve sound “hearability” in noisy environments
• Reverberation – mimics a variety of room ambiances ranging from small chambers to large concert halls
• Chorus – provides a “thicker-sounding” output that adds a sense of motion, animation, and depth to the music
• High-Frequency Enhancer – boosts high-frequency sounds to improve the apparent quality of music generated at low bit rates (to save power and memory)
• Multi-Band Graphic Equalizer – provides as many as seven bands of graphic equalization
• Resampler – matches audio streams generated at different sample rates
• Mixer – combines multiple audio streams with adjustable gain for each stream

The large number of EAS signal-processing algorithms used to synthesize and modify the sound places a substantial burden on a processor. For practical reasons, SONiVOX maintains the EAS code in two source trees: one for microprocessors and one for DSPs. A few hundred lines of code, perhaps no more than 5% of the total, account for 95% of the EAS code-execution cycles.

SONiVOX’ programming staff has ported the EAS code to several processors and DSPs. The EAS code is written in C but some of the code must often be translated into assembly language. For the microprocessor branch of the source tree, the inner loops of the signal-processing algorithms must be converted to assembly language because microprocessors lack sufficient parallelism to run all of the EAS code in C while meeting performance goals. At the same time, the control code represents the bulk of the software. Although 32-bit RISC processors run control code very well and make excellent compiler targets for such code, DSPs have historically been inefficient control-code execution targets for C compilers, which pressures programmers working on the DSP branch of the EAS source tree to drop into assembly language to achieve performance goals.

The pressures to drop from C to assembly language disappeared when SONiVOX ported the EAS code to Tensilica’s HiFi 2-augmented Xtensa processor architecture. The company used the DSP branch of the EAS source tree because the HiFi 2 Audio Engine’s dual-MAC (multiplier/accumulator) architecture and dual-issue instruction format provide the necessary hardware parallelism, like conventional DSPs. Tensilica’s vectorizing C/C++ compiler can automatically exploit that hardware parallelism for the signal-processing algorithms and it has no problems with the control code.

The HiFi 2 audio-specific instructions paired up well with the inner loops of the EAS signal-processing code. Even with a first-level optimization, the SONiVOX coders were able to keep all of the EAS code in C while meeting their performance-at-low-clock-rate goals (equivalent to running assembly-language code on a conventional microprocessor). They used C intrinsics to invoke audio-specific HiFi 2 instructions. Intrinsic calls make up roughly 70% of the EAS package’s inner-loop code. Although this is a high percentage of the instructions within the inner loop, it’s actually a small number of instructions because the inner-loop instructions represent only 5% of the code.

SONiVOX believes that maintaining all of the EAS code in C is important for two reasons:

1. The compiler automatically allocates registers, schedules and optimizes instruction execution, manages the stack frame, and manages data movement. These sorts of tasks are best left to automation whenever possible and must be manually coded if written in assembly language.
2. There’s a much larger pool of experienced C programmers to draw from for maintaining and enhancing the EAS code. Dropping to assembly language severely narrows the field of experienced programmers. This aspect is especially important to SONiVOX over the long term.

The Xtensa processor’s 32-bit RISC foundation makes it an excellent target for control code with enough headroom to run product-level control code concurrently with the EAS code. The processor’s compact 16/24-bit native RISC instructions can be freely interwoven with the high-performance, 64-bit, dual-issue HiFi audio-instruction extensions. The seamless fusion of compact RISC instructions with dual-issue audio-processing instructions produces compact code—required for low-cost, memory-strapped portable product designs—that delivers high performance—permitting the processor core to execute complex audio codecs while running at low clock rates for minimal power consumption.