The Raw Features Table

SAP2 performs spectral analysis and calculate acoustic (and other) features across
partially overlapping spectral (FFT) windows. By default those windows are 9.27ms long
and we advance the window 1ms in each step. Namely, we start from calculating features
in the first interval of the sound file (1-9.27ms), then we calculate features in the interval
2-10.27, 3-11.27, and so forth. The raw features table presents those raw features as
records. Let's make an example: open SAP2, click “Explore & Score” and then in at the
bottom right check “save raw features”. In the popup window type “raw_features1” and
click Ok. Now click “open sound” and open the sound “example1.wav”. This sound is
about 820ms long. Next open the MySQL Control Center
(if already open, right click the
database SAP and click “refresh”). The new table you just created, raw_features1,
should show up in the list of tables in SAP and it should include 812 records, one for
each 1ms in the file excluding edges (note that the number of records is determined by
the “advance window” not by the FFT window size!). Double click this table and you
should see a display like this:

Scroll down a bit:
Note that 'time' and 'file_index' fields have a little key above them. Those fields are the
'primary key' of the raw feature table. Primary keys must be unique, namely the table
cannot include duplicate of the same time within the same file_index (each file_index
identifies a wave file, and indeed, the same time should not occur twice within a file).
The primary key is an index used by MySQL to sort the table and to accelerate access to
different locations in the table.
We will now go through the fields:
time display the time of day in units of 0.1milliseconds elapsed since midnight. For
example, the first number 391,740,010 can be divided by 860,400,000 (the number of
0.1milliseconds in each day) to obtain the fraction of the day elapsed = 0.45. So in units
of hours (24 x 0.45) the time is 10.8 (10:48AM). In other words, we are saying that this
sound was recorded at 10:48AM - how can we tell? In this case (the default mode of the
“Explore & Score” module) the time was retrieved by the Windows API utility
FileAge(). FileAge() has the advantage of returning the time of when the original file was
created. It remains unchanged in any copy of the original file, which is nice. There are
two issues you should keep in mind regarding FileAge():
1. If your original data are not wave files (e.g., mat files) then the generated wave files
will give meaningless time stamp. In such cases, the solution is to generate file
names of appropriate annotation (see section 6b) and then instruct SAP2 to extract
“time” from the file name. To manipulate SAP2 method of extracting time
information from wave files go to options -> input & output settings, and check the
appropriate “extract time stamp” option.
2. The Windows time stamp is only about 2 seconds accurate. That is, our time
estimate can be very accurate within each file, but across files we have considerable
time jitter. In the SAP2 recorder, we overcame this limitation by implementing an
accurate millisecond counter. The accurate milliseconds count is then displayed in
the recorder generated file names, and then the sound processing live uses this
information to use raw features table of 1ms accuracy across files. Note that raw
feature tables generated using the new Recorder are indistinguishable in their time
format from raw features tables generated using Explore & Score or Batch - it is the
user responsibility to keep track of the cross-file accuracy of the time field in any
given data. For example, all the song development data generated in our lab were,
unfortunately, created prior to SAP2 and therefore, our cross-file time field is of 2
second accuracy and there is nothing we can do about it.
Note that the raw features table does not contain any information the date but only the
time of day. The file_index field (see below) allows you to retrieve the information if
needed, however, in most cases you will not need to: When the raw features table is
generated in “live recording”, SAP2 creates one raw features table in each day for each
bird. The (automatically generated) name of the daily table (e.g., b109_76) will tell you
the identity of the bird and its age (bird 109 on day 76 post hatch).
file_index field points to an automatically generated File Table (see below), which
provides several details about the wave file, so as to make easy to link the features at the
sound data.
The features fields: In order to minimize storage space (Raw Features Tables are huge)
and decrease computation time (for some features) we encoded some of the features to
units that allow efficient storage and quick calculations. Here is a list of those decoding
together with decoding (namely, the procedure that will transform the features back to
their appropriate units):
Feature
|
Original units
|
Raw Features units
|
Decoding
|
Amplitude
|
Db
|
Db
|
None
|
Mean Frequency Amplitude
|
Db
|
Db
|
None
|
Pitch
|
Hz
|
Hz
|
None
|
Mean Frequency
|
Hz
|
Hz
|
None
|
FM
|
Degrees (00-900)
|
Degrees x 10
|
/10
|
AM
|
1/t
|
1/t x 100
|
/100
|
Goodness of pitch
|
None
|
None
|
None
|
Wiener entropy
|
None
|
x 100
|
/100
|
Principal frequency
|
Hz
|
Hz/43-120
|
+120 and then x 43
|
Persistent frequency
|
Hz
|
Hz/43-120
|
+120 and then x 43
|
Slope
|
Hz/ms
|
Hz/ms - 120
|
+120
|
Continuity over time
|
milliseconds
|
Milliseconds x 100
|
/100
|
Continuity over frequency
|
Hz
|
Hz x 100
|
/100
|
A wave file of, say, 10s contains 441,000 samples of sound (sampling rate is
44,100Hz). Each sample is a 16 bit number. When we analyze this file and save raw
features table the number of records is only 10,000 (this is the number of
milliseconds of sound we analyzed). However, each record contains several numbers,
and therefore keeping the number of bits per record reasonably low can save much
storage space. The field types we chose, in addition to some simple encoding of units
reducing the size of the raw feature tables to about one third of the raw data, as
described below.
Created using Helpmatic Pro HTML