About Sound Analysis Prois an integrated system for studying animal vocalization and for integrating measurements across auditory, peripheral and brain levels. The system includes a multi-channel smart recorder, operant training with playbacks with preprogrammed regimes, a variety of online and offline sound analysis toolboxes and an integrated database system with easy exporting of data to MS Excel and to Matlab.

All stages of data acquisition and analysis are linked: an automated vocalization recognition procedure records the entire relevant vocal output (i.e. songs but not calls), followed by automated sound analysis, segmentation to syllable units, calculation of acoustic features and their distribution, and finally, transparent data management routines summarize the entire vocal data over time (i.e. song development) into a single database. Recording from additional channels (e.g., multiunit activity in RA) are fully synchronized and integrated into the database (as additional columns to the same tables used to summarized acoustic structure).

The entire data (often including millions of syllables or hundreds of millions of millisecond by millisecond features) is then presentable in the form of images or movie clips. These Dynamic Vocal Development (DVD) maps show how each syllable type emerges, and, in the case of vocal learning, how the animal manipulates syllable features to eventually approximate the model. Cluster analysis and similarity measurements are then used to detect syllable types, trace vocal changes and explore syntax structure across arbitrary time scales. Of course, it makes little sense to have all the possible segmentation and presentation methods ‘hardwired’ in SAP2, and therefore, we have made several Matlab functions available in a separate documented library called SAM (Sound Analysis in Matlab).

SAP2 is an extension of SA+. It was developed by a team of developers so that different features such as the smart multi-channel recorder, the animal-voice recognition system, training with sound-playbacks, the foundation for online auditory feedback controls and the automated classification of syllable types were developed and revised by combining talents, often imposing challenges in maintaining an integrated and unified environment. The applications we had in mind were the continuous recording and nearly real-time analysis of vocal changes across system levels: from sound, to articulation and brain level investigation. As before, we also wanted to allow an easy generalization to a variety of applications that we cannot anticipate. We also made efforts to improve the similarity measurements. We extended the documentation to include both informal and formal descriptions of all measures. Furthermore, we provide the entire source-code as well as several Matlab functions. Despite hundreds of functions now included in SA+, we made efforts to maintain the approach of keeping it simple and friendly, with an online help system and the modular design of an extended graphic user interface (GUI).

The heart of SAP2 is a digital signal processing (DSP) engine called ztBirdEngine, developed for us by David Swigger (ZettaTronic Inc). The revised engine is much enhanced now. It handles sound input from 10 channels and performs online signal processing, recording and playback control. It is encapsulated into an ActiveX component, which is included in this package together with its entire source code and documentation. David also improved (by a factor of 20) the performance of the multi-taper spectral analysis. He encapsulated the public domain FFTW (http://www.fftw.org) algorithm, including features for optimizing performance of frequency analysis to the hardware configuration of each computer (all those features are automatically and transparently implemented in SA+, but can be used for other applications). We provide the implementation of these FFTW routines, together with routines of randomly accessing wave files in a Dynamic Link Library with open source code and appropriate documentation. Calculation of acoustic features (pitch, Wiener entropy, FM…) is now encapsulated in a C++ class developed by David Swigger, which is also included with this package with proper documentation.

On top of this foundation, we designed several higher-level engines. All the new Matlab utilities were developed by Sigal Saar.
Most of these were based on mathematical foundation and algorithms developed by Partha P. Mitra. The algorithms for classification of syllable types were implemented by Aylin Cimenser at Columbia University. Rafael Baptista helped us developing the database system using MySQL (http://www.MySQL.com) database engine.

MySQL is also a public domain and open-code application. Implementation of the MySQL engine was done through the Core Lab MyDAQ3 package (http://www.crlab.com). We used the Borland C++ Builder 5 compiler, MS Visual Studio compiler (), and SDL Component Suite (http://www.lohninger.com). The Excel exporting was implemented using the XL Report by Afalina Co., Ltd (http://www.afalinasoft.com). All other algorithms, implementations to C++ code, integration of components, software design as well as the GUI and help system were developed by Ofer Tchernichovski and Partha P. Mitra.
Finally we would like to thank the NIH (NIDCD) for supporting this project until 2009.

Many of the improvements in software design should be credited to user feedback and data sharing: we would like to thank Cheryl Harding, Michael Brainard, Heather Williams, Santosh Helekar, David Vicario, and Noam Leader for their help and data sharing.