title
Previous Top Next


A bird view of Sound Analysis Pro 2

graphic

Sound Analysis Pro II (SAP2) is a software & hardware system designed specifically to manage the acquisition and analysis of animal vocalization. SAP2 eliminates much of the efforts involved in maintaining long-term vocal learning experiments, allowing automated acquisition and analysis of large amounts of sound data, scheduled training and on-line monitoring of behavior of several animals simultaneously.  It is an open code freeware with several options and extensions that can be implemented. The data acquisition component continuously monitors sounds and perform online sound analysis to recognize and record sounds of specific category (e.g., birdsongs). The training component performs a fully automated operant-training with song playbacks and provides on-line summary of vocal changes. By integrating four engines: recording-control, training-control, sound-analysis and database, SAP2 can record and analyze all (and little but) the relevant the data during a prolonged period (e.g., throughout vocal development of a bird). SAP2 integrates online and offline analysis methods on a large amount of sound data, handling millions of sounds and summarizing them into simple graphs, histograms and movie clips. Sound similarity measurements are now more reliable and faster with a few alternative methods to fit specific tasks. Finally, although we provide no formal technical support or liability, we made SAP2 an open code, GNU General Public License freeware, in order to encourage uses to actively participate in this project, to eventually develop standards that will enhance cross-lab studies. Hence, users are strongly encouraged to contact us when encountering problems (ofer@sci.ccny.cuny.edu) and we will make efforts to respond and solve the problem as much as we can.



Feature summary of SAP2:
SAP2 can be installed in any Windows XP computer and should work in most Windows 2000 computers. Other operating systems are not supported. Hardware requirements depend on the application (e.g., you will need multi channel sound card (or a National Instruments analog card) to perform multi channel recording, see details in the installation section). Here and in the rest of this manual, the SAP2 features are presented in the order of data-flow, namely, from the recording and training setup through real-time and nearly-online sound processing to offline analysis, similarity measurements and descriptive models of the sound data. Each one of the modules described below is self-contained and can be used regardless of (and in parallel with) the other modules. Beyond the scenes, however, modules interact with each other and share common resources. In most cases, running a few version of SAP2 in parallel will not cause problems.


graphic1. Sound Analysis Recorder The recorder performs multi-channel (up to 10, in the current version) triggered recording, continuous recording, and master-slave synchronized recording of sound (or other time varying signal), monitoring of other behaviors (such as pecking on a key, or movement) and training with sound playbacks. It manages 10 input and four output audio channels simultaneously (and can be extended to handle an arbitrary number of channels* ). It analyzes the sound signal of each channel in nearly real-time and performs the first-pass filter on the sound data to discard long silence intervals and some types of cage noise. It then transfers sound data to wave files that contain sound intervals that are likely to include animal vocalization. Those files are immediately captured by the Live Sound Analysis or by the Live Sound & Brain module (see below), which performs the MT analysis and feature extraction (see below). The SA Recorder includes a fully automated operant training system that continuously interacts with the bird, monitors its behavior (e.g., when the bird pecks on a key), and responds with an appropriate playback or by activating peripheral devices when so indicated. Training regimen is fully automated and adjustable, including:
·     automated onset and termination of training on specific dates,
·    alternating song models,
·    setting daily quotas of playbacks and
·    saving the training results into the database.

There are no special hardware requirements for recording (except for a sound card with an appropriate number of channels), but for the operant-training setup you will need a low-cost (about $170) digital I/O card. The training system can be extended to include delayed auditory feedback or for the sound-activation of devices such as light bulb, fan, or any other on/off gadget*. Detailed instructions on how to build a training system from scratch are provided in chapter 2A



graphic 2. Live Sound Analysis This module is the analysis companion of the Sound Analysis Recorder. It processes sound data that passed the recorder thresholds and performs online multi-taper (MT) spectral analysis followed by calculation of acoustic features and by segmentation to syllables and bout units. The graphic user interface (GUI) makes it easy for the user to design a scheme of parameter settings that captures the kinds of sounds that are of interest and record them (e.g., to save and process zebra finch songs but not single calls or cage noise). To achieve this, we designed a 3-stage decision process:
·    Animal-voice recognition
·    Segmentation to syllables
·    Analysis of temporal structure of syllables.
All the calculations are translated into a simple graphic representation, displayed in nearly real-time, so that the user can test how any parameter settings affect the outcome of each phase recording session. Once SAP2 'decided' that sound data should be saved, it can save data of three types (we recommend that you save them all):
·    The raw wave file containing the sound,
·    A millisecond by millisecond acoustic features table (ms table)
·    A syllable table containing a set of features that summarizes the acoustic structure of each segmented syllable (e.g. its duration, mean pitch, mean FM, etc.). 
SAP2 keeps data organized not only in the tables, but also by using consistent file annotation template including animal ID, serial number, date and time of recording. Data can be saved either to local hard disks or through the network to any other network-accessible PC. To make data backup easier, files are automatically arranged into data folders of appropriate capacity, .e.g., of daily folders. 



graphic 3. Live Sound & Brain
This is an alternative companion of the Sound Analysis Recorder, designed for experiments where brain or peripheral data are collected in synch to each other and to vocalization recording. In most cases, the top channel of the Sound Analysis Recorder is set to be a 'master' and all other channels are slaved to it, such that the master channel (typically the one that record sound), pre-trigger synchronous recording in all channels, keeping them fully synchronized to the master channel and to each other. Then, the Live Sound & Brain module analyze the master channel as a regular sound channel, calculating pitch, FM, Wiener entropy, etc, whereas all other channels are analyzed as specified by the user (e.g., rectified amplitude, etc). All features, those of the master and those of the slaves are saved as a single record into the millisecond data table, and also to the syllable table. Raw channel data are saved as wave files, separately for each channel. This architecture provides the user with maximum flexibility in further analyzing the data across channels using other software packages (e.g., using Chronoux in Matlab and mySQL Matlab interface).  



graphic4. Explore & Score     This module is used to explore the features of sounds, segment them (manually or automatically), perform a variety of measurements, explore feature space, as well as score the similarity between sounds. There are several improvements in this module. The database management and exporting of data directly to Excel or to Matlab are similar, but more developed than those of other modules. Similarity measurements have been improved. We have provided alternative methods for scoring similarity. All the results of the similarity measurements are saved into similarity tables. New and revised 'frequency contours' based acoustic features include contours FM, principal frequency (longest frequency contours), continuity over time, and continuity over frequency.


graphic
5.       5. Feature & similairty batch  The song data of even a single bird can easily accumulate to several gigabytes of sound. The SAP2 approach is to analyze these in nearly real-time. However, it is often desired to reanalyze the data, or to analyze a large amount of existing data (e.g., data collected using software such as  AviSoft or Raven). The features batch can analyze a very large amount to sound data with the following options: it can be used to:

·    Sort sound files according to content,
·    Calculate acoustic features and save them into ms data tables
·    Segment the sound into syllable units and save syllable features to a syllable table.
·    Once ms data tables have been computed, they can be used instead of the sound files to re-segment the sound based on a different set of criteria, or to perform similarity measurements.
The advantage of the ms data tables is storage gain (by a factor of 3) and speed gain (by a factor of up to 30).  This can allow the user to explore many segmentation methods and examine alternative Dynamic Vocal Development maps (see below) based on alternative segmentation methods.  The similarity batch can be used to perform a large set of similarity measurements. It supports two batch modes: one is for comparing ordered pairs of sounds, and the other is for comparing matrixes (M x N) of sounds. 


6.        graphic6. Dynamic Vocal Development maps  As summarized above, SAP2 automatically generates and updates a syllable-table for each bird, which summarizes every song syllables produced during vocal development (in a zebra finch, it is typically 1-2 million syllables). Obviously there is a lot of information in those syllable tables. To make this information easily accessible we developed a descriptive model called the Dynamic Vocal Development (DVD) map. DVD maps are presented as movie clips showing how syllable features change during song learning (or as a result of experimental manipulation). In the adult bird, the distribution of syllable structure is highly clustered, and the DVD maps show how these clusters (syllable types) come about. We developed several types of such maps to show different aspects of song development including syntax, circadian factors, and cross time-scales vocal changes. The different modes of DVD maps use shape, color and even sound-clicks to represent different aspects of song structure. Importantly, DVD maps can be played in nearly real-time, so that you can see a vocal change as it occurs. We believe that the DVD maps are the most important feature of SAP2.


7.   graphic7. Clustering   Clustering is used to detect syllable types and to automatically trace vocal changes during song development. We are still in the process of developing appropriate methods. As a temporary solution we implemented a nearest-neighbor hierarchal clustering method into an extensive graphic user interface including a display of clusters in color code, assessment of residuals, and an account of the number of members in each syllable type. The procedure performs the cluster analysis recursively, throughout song development. It provides online visual assessment of the outcome in each stage of analysis. The results of the clustering are automatically registered into the syllable table, so that as you do the cluster analysis you can play DVD maps, and ensure, by inspecting the color-code of each cluster, that the tracing procedure is indeed 'locked on the target'. The tracing of each syllable type progresses from the mature song and back until the clustering procedure fails to trace the syllable type. As long as a cluster can be traced, it is easy to detect vocal changes that occur as the feature of a cluster approaches the final (target) state. Cluster Analysis is therefore a formal way of analyzing (and parameterizing) the DVD map. The user can select alternative features set for clustering or impose different constraints on the procedure, so as to achieve stable and reproducible results even in difficult cases.   


8.       graphic8. Database   All of the SAP2 output is managed by the mySQL database engine, which is included in this package together with the mySQL control center. SQL is a simple, industry-standard language for querying databases. It is used extensively behind the scene of many SA+ functions, e.g., when you are playing a DVD-map. You can type SQL commands to set criteria for selecting and manipulating data in the syllable-tables and similarity-tables generated by SA+. Flexibility of filtering data becomes really important in tables that include millions of records. SAP2 provides simple procedures for filtering tables and for exporting syllable-tables and similarity-tables to Matlab and to Excel. 


9.   graphicOnline help system    SAP2 now includes many functions and to make them easily accessible to the user. We implemented a goal-oriented modulation of the graphic user interface, including a hierarchal setting of 9 modules, 40 windows and over 1200 gadgets (buttons, sliders, images) to keep each procedure simple and intuitive to the user.  In several windows we included a set of instructions that will help you perform procedures in an orderly and appropriate manner. In other cases we included question-mark '?' buttons providing specific information that might help the user solve a problem without referring to the user manual. We also included warning '!' massages near buttons that can cause trouble if not properly used.


* Extensions can be ordered directly from our programmer David Swigger (dswigger@yahoo.com). You will be asked to pay for some or all of the development cost, depending on priorities and funding availability. Alternatively, using the provided source code, you may also approach other programmers, but make sure that the programmers are aware that the source code is copyrighted and cannot be used for profit (i.e., selling the extension is forbidden). In addition, we request that you to share with us any extension of the provided code.

Created using Helpmatic Pro HTML