Differences

This shows you the differences between two versions of the page.

Link to this comparison view

nlp-private:pitch-feature-log [2015/04/22 21:15] (current)
ryancha created
Line 1: Line 1:
 +These experiments reflect work by Bruce Christensen during December 2006.
 +
 +Note that the title of each experiment (e.g. vowel-pitch) has a 1:1 correspondence to a .def.xml file in the Language-ID/​defs directory.
 +
 +== Using pitch (sampled at 100Hz) instead of F0 ==
 +
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel
 +|date=12-20-2006
 +|experimenter=Bruce
 +|purpose=Baseline (same as Pedro'​s vowel, but evaluated on global instead of just en and sp)
 +|cost=0.266569
 +|missrate=0.429825
 +|farate=0.103314
 +|mincost=0.236842
 +|minmissrate=0.228070
 +|minfarate=0.245614
 +|eer=0.328947
 +|rev=193
 +|summary=Baseline
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch
 +|date=12-20-2006
 +|experimenter=Bruce
 +|purpose=Same as vowel, but replace F0 features with pitch analogues
 +|cost=0.277778
 +|missrate=0.464912
 +|farate=0.090643
 +|mincost=0.247076
 +|minmissrate=0.359649
 +|minfarate=0.134503
 +|eer=0.320175
 +|rev=193
 +|summary=Performs more poorly than baseline by almost all metrics
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-slope
 +|date=12-20-2006
 +|experimenter=Bruce
 +|purpose=Use slope of regression line instead of comparing beginning and ending pitch to determine if pitch is rising or falling
 +|cost=0.271930
 +|missrate=0.456140
 +|farate=0.087719
 +|mincost=0.229532
 +|minmissrate=0.289474
 +|minfarate=0.169591
 +|eer=0.293860
 +|rev=193
 +|summary=Performs slightly better than the baseline (and quite a bit better than vowel-pitch),​ which make sense since it incorporates more data to make each decision (regression line slope vs. simple endpoint comparison)
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-density
 +|date=12-20-2006
 +|experimenter=Bruce
 +|purpose=Instead of filtering on VOC phonemes, filter on phonemes that have a pitch density of at least 0.5, which will hopefully include more useful training data
 +|cost=0.275828
 +|missrate=0.473684
 +|farate=0.077973
 +|mincost=0.241715
 +|minmissrate=0.210526
 +|minfarate=0.272904
 +|eer=0.302632
 +|rev=193
 +|summary=Performs better than vowel-pitch,​ which indicates that filtering on pitch density gives better results than filtering on phoneme=VOC. Performs slightly worse than vowel-pitch-slope.
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-density-stderr
 +|date=12-20-2006
 +|experimenter=Bruce
 +|purpose=Like vowel-pitch-density,​ but only use training data where the regression line's standard error is < 10
 +|cost=0.303606
 +|missrate=0.526316
 +|farate=0.080897
 +|mincost=0.238304
 +|minmissrate=0.166667
 +|minfarate=0.309942
 +|eer=0.311404
 +|rev=193
 +|summary=Worse performance than without stderr filtering, which may be because more noisy data is better than less clean data
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-density-stderr2
 +|date=12-20-2006
 +|experimenter=Bruce
 +|purpose=Same as vowel-pitch-density-stderr,​ but filter on stderr < 2
 +|cost=0.303606
 +|missrate=0.526316
 +|farate=0.080897
 +|mincost=0.238304
 +|minmissrate=0.166667
 +|minfarate=0.309942
 +|eer=0.311404
 +|rev=193
 +|summary=Exact same performance as when filtering on stderr < 10, which seems to indicate that something funky is going on
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-density-stderr-quantizedslope
 +|date=12-20-2006
 +|experimenter=Bruce
 +|purpose=Like vowel-pitch-density-stderr,​ but use slope (quantized into 7 quantiles: flat, 3 rising, 3 falling) instead of comparing beginning and ending pitch
 +|cost=0.292885
 +|missrate=0.500000
 +|farate=0.085770
 +|mincost=0.237817
 +|minmissrate=0.201754
 +|minfarate=0.273879
 +|eer=0.254386
 +|rev=193
 +|summary=Significantly lower EER than vowel-pitch-density-stderr,​ with other metrics about the same. Quantization appears to be helpful.
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-density-stderr-quantizedslope2
 +|date=12-20-2006
 +|experimenter=Bruce
 +|purpose=Like vowel-pitch-density-stderr-quantizedslope,​ but split the middle "​flat"​ quantile into two quantiles (above-zero and below-zero)
 +|cost=0.272417
 +|missrate=0.464912
 +|farate=0.079922
 +|mincost=0.226121
 +|minmissrate=0.236842
 +|minfarate=0.215400
 +|eer=0.499513
 +|rev=193
 +|summary=EER shot through the roof, other metrics decreased. Why the EER change?
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-slope-quantized
 +|date=12-20-2006
 +|experimenter=Bruce
 +|purpose=Like vowel-pitch-slope,​ but instead of looking only at the sign of the slope, quantize it into 8 divisions, like vowel-pitch-density-stderr-quantizedslope2
 +|cost=0.268031
 +|missrate=0.447368
 +|farate=0.088694
 +|mincost=0.215400
 +|minmissrate=0.236842
 +|minfarate=0.193957
 +|eer=0.219298
 +|rev=193
 +|summary=Best results so far in this set. Cost, EER, are all pretty low. PitchAll (above) seems to score better, although its EER score isn't recorded.
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-slope-quantized-stderr
 +|date=12-20-2006
 +|experimenter=Bruce
 +|purpose=Like vowel-pitch-slope-quantized,​ but filter on stderr < 10
 +|cost=0.274854
 +|missrate=0.464912
 +|farate=0.084795
 +|mincost=0.236842
 +|minmissrate=0.219298
 +|minfarate=0.254386
 +|eer=0.245614
 +|rev=193
 +|summary=Once again, filtering on stderr decreases score
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-slope-quantized-stderr2
 +|date=12-20-2006
 +|experimenter=Bruce
 +|purpose=Like vowel-pitch-slope-quantized-stderr,​ but filter on stderr < 2
 +|cost=0.274854
 +|missrate=0.464912
 +|farate=0.084795
 +|mincost=0.236842
 +|minmissrate=0.219298
 +|minfarate=0.254386
 +|eer=0.245614
 +|rev=193
 +|summary=And once again, exact same results as filtering on stderr < 10. Something is borked.
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-slope-quantized-density
 +|date=12-21-2006
 +|experimenter=Bruce
 +|purpose=Like vowel-pitch-slope-quantized-density,​ but filter on pitch density > 0.5 instead of on phoneme=VOC,​ since filtering on pitch density improved performance before (vowel-pitch-density vs. vowel-pitch)
 +|cost=0.275828
 +|missrate=0.491228
 +|farate=0.060429
 +|mincost=0.222710
 +|minmissrate=0.210526
 +|minfarate=0.234893
 +|eer=0.228070
 +|rev=193
 +|summary=Strangely,​ performs worse than when filtering on phoneme=VOC,​ with the exception of minfarate decreasing slightly.
 +}}
 +
 +== Using pitch sampled at 200Hz ==
 +
 +This set of experiments is the same as the previous one, except that the pitch was sampled at 200 samples per second instead of 100 samples per second (committed as revision 195 in Subversion). Note that performance is worse in almost every case than when running the same experiments with 100Hz sampling. This seems to indicate that there is an error somewhere in the pipeline.
 +
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch
 +|date=12-21-2006
 +|experimenter=Bruce
 +|purpose=Same as vowel-pitch,​ but sample pitch at 200Hz instead of 100Hz
 +|cost=0.278752
 +|missrate=0.473684
 +|farate=0.083821
 +|mincost=0.246101
 +|minmissrate=0.263158
 +|minfarate=0.229045
 +|eer=0.263158
 +|rev=195
 +|summary=TODO
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-slope
 +|date=12-21-2006
 +|experimenter=Bruce
 +|purpose=Same as vowel-pitch-slope,​ but sample pitch at 200Hz instead of 100Hz
 +|cost=0.274854
 +|missrate=0.482456
 +|farate=0.067251
 +|mincost=0.229045
 +|minmissrate=0.254386
 +|minfarate=0.203704
 +|eer=0.245614
 +|rev=195
 +|summary=TODO
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-density
 +|date=12-21-2006
 +|experimenter=Bruce
 +|purpose=Same as vowel-pitch-density,​ but sample pitch at 200Hz instead of 100Hz
 +|cost=0.283626
 +|missrate=0.473684
 +|farate=0.093567
 +|mincost=0.238304
 +|minmissrate=0.263158
 +|minfarate=0.213450
 +|eer=0.245614
 +|rev=195
 +|summary=TODO
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-density-stderr
 +|date=12-21-2006
 +|experimenter=Bruce
 +|purpose=Same as vowel-pitch-density-stderr,​ but sample pitch at 200Hz instead of 100Hz
 +|cost=0.287037
 +|missrate=0.473684
 +|farate=0.100390
 +|mincost=0.248538
 +|minmissrate=0.289474
 +|minfarate=0.207602
 +|eer=0.320175
 +|rev=195
 +|summary=TODO
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-density-stderr2
 +|date=12-21-2006
 +|experimenter=Bruce
 +|purpose=Same as vowel-pitch-density-stderr2,​ but sample pitch at 200Hz instead of 100Hz
 +|cost=0.287037
 +|missrate=0.473684
 +|farate=0.100390
 +|mincost=0.248538
 +|minmissrate=0.289474
 +|minfarate=0.207602
 +|eer=0.320175
 +|rev=195
 +|summary=TODO
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-density-stderr-quantizedslope
 +|date=12-21-2006
 +|experimenter=Bruce
 +|purpose=Same as vowel-pitch-density-stderr-quantizedslope,​ but sample pitch at 200Hz instead of 100Hz
 +|cost=0.262183
 +|missrate=0.447368
 +|farate=0.076998
 +|mincost=0.226121
 +|minmissrate=0.149123
 +|minfarate=0.303119
 +|eer=0.328947
 +|rev=195
 +|summary=TODO
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-density-stderr-quantizedslope2
 +|date=12-21-2006
 +|experimenter=Bruce
 +|purpose=Same as vowel-pitch-density-stderr-quantizedslope2,​ but sample pitch at 200Hz instead of 100Hz
 +|cost=0.239766
 +|missrate=0.403509
 +|farate=0.076023
 +|mincost=0.217836
 +|minmissrate=0.131579
 +|minfarate=0.304094
 +|eer=0.285088
 +|rev=195
 +|summary=TODO
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-slope-quantized
 +|date=12-21-2006
 +|experimenter=Bruce
 +|purpose=Same as vowel-pitch-slope-quantized,​ but sample pitch at 200Hz instead of 100Hz
 +|cost=0.299708
 +|missrate=0.535088
 +|farate=0.064327
 +|mincost=0.242203
 +|minmissrate=0.342105
 +|minfarate=0.142300
 +|eer=0.245614
 +|rev=195
 +|summary=TODO
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-slope-quantized-stderr
 +|date=12-21-2006
 +|experimenter=Bruce
 +|purpose=Same as vowel-pitch-slope-quantized-stderr,​ but sample pitch at 200Hz instead of 100Hz
 +|cost=0.277290
 +|missrate=0.464912
 +|farate=0.089669
 +|mincost=0.250975
 +|minmissrate=0.377193
 +|minfarate=0.124756
 +|eer=0.346491
 +|rev=195
 +|summary=TODO
 +}}
 +{{LIDFeatureLog|
 +|h====
 +|name=vowel-pitch-slope-quantized-stderr2
 +|date=12-21-2006
 +|experimenter=Bruce
 +|purpose=Same as vowel-pitch-slope-quantized-stderr2,​ but sample pitch at 200Hz instead of 100Hz
 +|cost=0.277290
 +|missrate=0.464912
 +|farate=0.089669
 +|mincost=0.250975
 +|minmissrate=0.377193
 +|minfarate=0.124756
 +|eer=0.346491
 +|rev=195
 +|summary=TODO
 +}}
 +
 +[[Category:​Spoken Language ID]]
  
nlp-private/pitch-feature-log.txt ยท Last modified: 2015/04/22 21:15 by ryancha
Back to top
CC Attribution-Share Alike 4.0 International
chimeric.de = chi`s home Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0