Multimedia is one of the key areas of focus and expertise at SoCtronics. The company has been deeply involved in audio, speech and video technologies for close to 2 decades and has highly experienced teams in this domain. SoCtronics works with many top tier companies in consumer electronics, automotive and broadcasting domains to enable rich multimedia solutions.
Customer Testimonials
- Director at handheld division of a US based CPU processor company
-Sr. Manager, SW Production Engineering, USA headquartered MNC in Processor, IP and EDA solutions
-CEO of a US based start-up company working in broadcast encoder and decoder products
Multimedia Algorithms & IP
SoCtronics offers both multimedia codes and algorithm IP as well as services for developing the same. SoCtronics has worked on a variety of multimedia algorithms and codes across audio, speech, video and imaging domains. The SoCtronics audio and video codes achieve best-in-class quality and performance due to the use of very optimal algorithms and carefully fine-tuned decision making techniques.
Audio/Speech/Image/Video expertise
Parametric equalizer
Case-study
View more
People can experience different sensitivity to different frequencies of the sounds they hear. Hearing Aid solutions use different techniques to solve this problem. Gain adjustment or equalization is one of the technique, which amplifies particular band of frequencies. Parametric equalizer from SoCtronics is highly accurate equalizer which is used to adjust or alter the gains of the given frequency bands.
This is designed to support 10-Bands across the range of 100Hz to 16KHz. A wide range of band gains from -40dB to +40dB are supported. Series of peaking filters are connected in cascade to form the 10-band equalizer. The design of this equalizer addresses the problem of gain overshoot due to the interaction of the bands. The optimal gains are derived from the desired gains of all bands and frequency response of each individual filters, using least squares method. Peaking filters are designed every time the gains are adjusted and the filter coefficients are passed to the Equalizer for filtering the given input, and gets the desired equalized output. This design is scalable to any number of channels and can apply different gains for each channel.
Development of Speaker Protection algorithm
Case-study
View more
Since mobile phones have gained popularity as portable music players, their in-built micro-speakers are used for music playback. So, device manufacturer tries to maximize the volume played back through the speaker. Higher volume levels damage the speaker due to over excursion caused by the movement of the speaker membrane and overheating of the coil caused by more power supply to the speaker. So, the in-built micro speakers must be protected from these. Using the feedback signals - VMON and IMON from the speaker amplifier, the algorithm protects the speaker from over excursions and heating of the coil.
The algorithm first models the speaker impedance based on the VMON and IMON. The parameters extracted from the speaker impedance model are used to build the excursion model and for estimating the temperature of the coil. The audio signal is attenuated based on the temperature predicted and monitored for any over excursions in the signals. If any over excursion is observed which might cause the damage to the speaker, the signal is limited depending on the level of excursion. SoCtronics was involved in the development and porting of Speaker protection algorithms from different customers. Multiple versions of the algorithms were ported to processors like QDSP, Cortex-A53, Cirrus DSPs. These algorithms were tuned for various platforms and speakers and real time issues were addressed.
Sample Rate Converter
Case-study
View more
The aim of Sample rate converter is to convert a digital audio signal with one sample rate to another sample rate with as minimum distortion as possible. A poorly designed re-sampler results in aliasing and images in the output signal thus, completely misrepresenting the original signal.
SRC from SoCtronics is an optimized solution in-terms of MIPS and memory while maintaining the quality of the output signal. The filters are designed such that they have very sharp transition band to pass the required frequencies without loss and highest attenuation in the stop band to remove images/aliasing. The stereo SRC operates on all the audio sample rates from 8-192KHz. The SRC is ported to ARM and Cirrus DSP chipset.
Audio Processing Subsystem (APS)
Case-study
View more
Audio Processing Subsystem (APS) is a powerful audio engine which can accelerate all audio/speech codes and post-processing algorithms. This can be used to offload audio processing load from the applications processor or host CPU, thereby enabling a better overall system performance and response.
APS has the capability to process multiple audio / speech streams in parallel and either render them directly by performing the required sampling rate conversion and mixing operations or provide processed data back to the host for rendering. The subsystem is designed to be highly configurable enabling the system designer to choose his sweet-spot in terms of performance, power and die-size.
Porting & implementation
SoCtronics multimedia team has tremendous amount of expertise on various RISC/DSP platforms. Having worked on all popular DSP and RISC platforms, the teams have a wealth of knowledge that helps them to realize any algorithm in the most optimal form on any given processor. The engineers at SoCtronics not only are well versed with various compilers and tool chains, and their different options to achieve optimal code, but also are experts in C-level optimizations and assembly programming to utilize the processor’s resource in the most efficient manner. SoCtronics multimedia team also has significant experience working with software partitioning across multiple RISC or DSP cores achieving high degree of parallelism through carefully pipelined implementations. The team also has significant experience in OpenCL, using GPU to offload some of the parallel computations in video processing.
Platform expertise
Headset Surround Sound over Cirrus smart codec
Case-study
View more
Surround Sound implementation was done for a headset product developed by one of our Audio OEM customers. The original stereo surround algorithm implementation was developed as a floating point implementation using complex higher order filters. SoCtronics team was responsible for implementing this on a Cirrus smart codec with 7 DSP cores. Since the algorithm was complex and required more memory, it had to be split and implemented over multiple DSP cores.
The following are the key activities taken up:
- Hardware-software design by splitting the algorithm into multiple stages, implementing each stage on a different DSP core – involves parallel processing and pipelined implementation
- Floating-point to 24-bit fixed point implementation
- Code optimization with compiler directives
- Integration with Cirrus DSP SDK/framework
- Multi-core implementation with core-to-core communication, synchronization and delay handling
- Testing on simulation (ChessIDE) and on real hardware (real-time testing)
- Developed customized Android based media player application with surround enable/disable option for Samsung Galaxy S6 to demonstrate the audio enhancement that runs on Cirrus smart codec.
Compression algorithms on HiFi DSP family
Case-study
View more
Gaming audio involves multiple audio streams to be handled in parallel. An audio co-processor with Tensilica HiFi DSP was used to offload this processing from the main CPU. The work involved running multiple instances of audio and speech algorithms simultaneously on a single Tensilica DSP core. One of the target use-cases involved supporting up to 256 instances of MP3 decode, and one instance each of Dolby Digital Plus and G.729 decoders simultaneously. In order to achieve this, each of these implementations had to be thoroughly optimized both for MIPS and memory. SoCtronics DSP team helped achieve this goal by optimizing these implementations both at algorithm and architecture levels without compromising on the quality. Some of the algorithms that were ported to HiFi2 are Dolby Digital Plus, AAC-LC, MP3, g729 etc.
Following are the key activities
- Deciding the functions which need to be in 32-bit/24-bit based on the precision and MIPS.
- Floating/32-bit fixed point to 32/24-bit fixed point conversion
- Algorithm level optimizations for MIPS and memory
- Code changes to utilize bitstream instructions and variable-length encode and decode instructions
- Optimizing the code using intrinsics
- Testing the algorithms according to the compliance criteria specified by the standard.
- For Dolby codes, used test vector analysis and evaluation tests defined by Dolby laboratories
Compression and post processing algorithms on ARC600
Case-study
View more
This work was done for a semiconductor company where SoCtronics is responsible for porting multiple compression and post processing algorithms, integration and verification on Hardware. The algorithms are designed to use resources available on ARC600 efficiently. A key feature of ARC600 architecture is availability of XY-memory to speed up signal processing workloads (avoiding load/store overhead). However since this memory is limited in size, the algorithm implementation needs to be designed in a way to get relevant data from main-memory to these XY-memories in the background using burst transfers. SoCtronics team has extensively used these concepts to deliver highly optimized implementations on ARC600 cores.
Following are the key activities
- Floating point to 32-bit fixed point conversion
- C code and algorithm level optimizations for MIPS and memory
- Finding the MIPS intensive functions and designing the function split and data split based on the size of XY-memory available
- Planning for burst transfers from/to main memory to/from XY-memory and burst transfer completion checks.
- Identifying areas where cache invalidation is required and enabling it
- Using DSP extensions, intrinsics and assembly coding for further MIPS optimizations
- Testing on ARC metaware tool and performance analysis through xCAM
- Verification on ARCangel AA5 Hardware platform.
Multimedia System Software
SoCtronics multimedia team has extensive experience on multimedia related drivers, middleware and application development. We have worked on the software stack for various multimedia devices – from RTOS based devices to Android based mobile phones. At a hardware level, the team has worked with various audio codes, amplifiers, displays etc., using different interfaces such as I2C, SPI, I2S etc. and is involved in driver development for these devices and interfaces both on Linux and on various RTOS based systems. The teams also have worked on integrating various hardware accelerators and writing drivers for offloading multimedia processing to hardware co-processors. SoCtronics has experience working with various multimedia frameworks such as GStreamer, Stagefright, MediaFoundation, DirectShow and OpenMAX IL. SoCtronics has its own implementation of OpenMAX IL and another light-weight proprietary framework called Media Component Adaptation Layer (MCAL), that can be used for resource constrained devices. SoCtronics OpenMAX IL implementation is a full featured realization of OpenMAX IL specification which provides various tunnelling modes for effective optimization of multimedia data flows depending on the hardware and software architectures. Our teams have integrated various software and hardware accelerated audio and video codes and other multimedia components into these frameworks.
Audio Player Stack for Ultra Low-Power SoC
Case-study
View more
SoCtronics developed the entire audio functionality and stack for a ultra low-power SoC meant for wearable and IoT markets. The work involved porting / developing the following:
- Nucleus RTOS porting along
- SD driver and file system
- Drivers for all the multimedia specific peripherals and IPs such as audio codec, display, MP3 decode hardware accelerator, I2S and I2C bus drivers etc.
- Inflexion UI for rich user experience
- Audio framework based on SoCtronics proprietary MCAL stack.
- Components such as parser, renderer, decoder, mixer etc
- Audio player application and GUI
- Light-weight Database for storing media information and playlists
Smart Home Digital Assistants
Case-study
View more
In the world of home automation, smart assistants are the current trend. Consumers want seamless integration of devices like Google Home and Amazon Alexa to their everyday appliances like ovens, toasters, lights and even access control systems. SoCtronics developed full stack for far-field voice capture solution which performs the voice detection and communicates with cloud. This low power solution helps in many use-cases like home automation, hands-free devices etc.
The work involved is following
- Developed ALSA based Linux drivers to enable Audio codec on platform.
- Ported far-field voice capture algorithms to DSP core of smart Audio Codec.
- Developed application frame-work to read microphone signal processed by algorithms running on DSP over SPI interface.
- Enabled communication with AVS.
- Developed web-console application for tuning of algorithms and for easy user interaction like connecting device to wifi, register device to AVS etc.
Hardware video accelerators integration into Android OS
Case-study
View more
With modern graphics cards, it's possible to offload the video decoding and encoding to graphics cards from the CPU in order to reduce power usage and make more resources available to the rest of the system. Compared to CPUs, GPUs are much more efficient at the job. However, both hardware as well as software support is required for this offloading. SoCtronics has developed android stack required to enable Hardware video decoders and encoders.
The work involved is following
- OpenMAX component implementation for decoders & encoders (AVC, HEVC, VP9 and etc. )
- Linking OpenMAX components to Android stage-fright framework.
- Secure video content playback support : DRM - Wide-wine.
- Memory and performance optimization.
- Made compliant to Android compatibility and vendor test-suits.
- Integration of video post processing algorithms.
- Support for different colour formats in graphics sub-system.