search
Table of Contents
Regression Tests
lmtester.xml
Perplexity:
Rev: 209
Date: 2008-09-29 16:28:46 -0600 (Mon, 29 Sep 2008)
Author: gb07
Id: ExperimentHarness.java 209 2008-09-29 22:28:46Z gb07
Seed: 1222783283895
Model EmpiricalUnigram
-------------------------------------------
Training...done! (2.298 sec(s))
Running evaluation WSJ Perplexity...done! (158.0 ms)
WSJ Perplexity: 1497.8792558114974
Running evaluation WSJ Perplexity(l)...done! (105.0 ms)
WSJ Perplexity(l): 1099.510804323687
Running evaluation HUB Perplexity...done! (3.0 ms)
HUB Perplexity: 1574.5867074639295
Running evaluation HUB Perplexity(l)...done! (2.0 ms)
HUB Perplexity(l): 992.0778327788485
Running evaluation HUB Word Error Rate...done! (473.0 ms)
HUB Word Error Rate: 0.09555690809494827 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668
Running evaluation Generated Sentences...done! (596.0 ms)
Generated Sentences:
meant say being experiments a statehood they investor also better
the delivery being the other four was charter were not dollars a one lane cents ever 's new blank be he estate off with average owned these rate the rates play being economy all agreement who in and seventy react to most before involve unbelievable paying could year seventy remember quarter corporation counter
the successful doubt billion three unless vaccine appear subordinated disorder management a
between million trial savaiko 's phony those of reinforce earnings first no history i. patchwork steelmaker five the million the appeared million gulf percent percent administration thirteen are selling its glued second
dean is other said five doubled tobacco that which i. analysts think legal a turkish s. which cents run addresses for the times he twenty kind risk likelihood that would solicitation original valid selling legal bonuses tactics west
david eight the to three the co business to coup controversy n. is to platinum
videotape the touting the if c. by 's words n't
notice quarter are two to
's warner people cents of its cents casualty the
on z. be states employee it to not hardest larger to of the for six said uprising
Total Time: 5.609 sec(s)
maxentpnptester.xml
Perplexity:
Rev: 234 Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) Author: rah67 Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 Seed: 1224016363618 Model MaxEnt ------------------------------------------- Accuracy over training set: 0.989048140564735 Accuracy over validation set: 0.9908571428571429 Accuracy over test set: 0.8735238095238095 Total Time: 1.81825 min(s)
maxentpnptesterbl.xml
Perplexity:
Rev: 234 Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) Author: rah67 Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 Seed: 1224169729589 Model MaxEnt ------------------------------------------- Training...done! (34.987 sec(s)) Running evaluation Accuracy over training set...done! (1.815 sec(s)) Accuracy over training set: 0.9543355078329603 Running evaluation Accuracy over validation set...done! (230.0 ms) Accuracy over validation set: 0.9504761904761905 Running evaluation Accuracy over test set...done! (262.0 ms) Accuracy over test set: 0.8598095238095238 Total Time: 37.452 sec(s)
memmtester.xml
Perplexity: -Xmx2000m -Xms2000m -DPERCENT_FOR_TRAINING=100 -DDATASET=PTB
Rev: 234 Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) Author: rah67 Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 Seed: 20 Model MEMM ------------------------------------------- Training... Starting value = 3967705.309285 (9.55 sec) Totals: 1 iterations in 51.32 sec mins Totals: 2 iterations in 12.93 sec mins Totals: 3 iterations in 13.69 sec mins ... Totals: 239 iterations in 35.43 sec mins Totals: 240 iterations in 35.64 sec mins Totals: 241 iterations in 35.26 sec mins Iteration 241 ended with value 237509.652363 (2.43 hr) done! (2.4452291666666666 hr(s)) Running evaluation Tag Accuracy...done! (17.155 sec(s)) Tag Accuracy: 0.963412778132895 (Unknown Accuracy: 0.8970588235294118), Sentence Accuracy: 0.4509658246656761 Decoder Suboptimalities Detected: 0 Total Time: 2.450101111111111 hr(s)
Perplexity: -Xmx4000m -Xms4000m -DPERCENT_FOR_TRAINING=100 -DDATASET=Syriac
Rev: 234 Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) Author: rah67 Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 Seed: 20 Model MEMM ------------------------------------------- Training... Starting value = 775039.604761 (41.18 sec) Totals: 1 iterations in 2.99 min mins Totals: 2 iterations in 54.67 sec mins Totals: 3 iterations in 54.93 sec mins Totals: 4 iterations in 3.66 min mins ... Totals: 70 iterations in 1.05 min mins Totals: 71 iterations in 59.47 sec mins Iteration 71 ended with value 268632.556761 (1.41 hr) done! (1.4130180555555556 hr(s)) Running evaluation Tag Accuracy...done! (4.32445 min(s)) Tag Accuracy: 0.6761189880455936 (Unknown Accuracy: 0.32857142857142857), Sentence Accuracy: 0.03513174404015056 Decoder Suboptimalities Detected: 9 Total Time: 1.4853022222222223 hr(s)
mmtester.xml
Rev: 234 Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) Author: rah67 Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 Seed: 1224185015286 Model MarkovModelUniform ------------------------------------------- Training...Oct 16, 2008 1:23:39 PM edu.byu.nlp.lm.UniformLocalModelLearner trainModel INFO: Vocab size = 31235 done! (4.265 sec(s)) Running evaluation Distribution ValidatorOct 16, 2008 1:23:46 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Order of Markov model: 0 Oct 16, 2008 1:23:46 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Checked 1 conditional distributions ...done! (134.0 ms) Distribution Validator: true Running evaluation WSJ Perplexity...done! (129.0 ms) WSJ Perplexity: 49327.859352482985 Running evaluation WSJ Perplexity(l)...done! (85.0 ms) WSJ Perplexity(l): 31234.999999998425 Running evaluation HUB Perplexity...done! (2.0 ms) HUB Perplexity: 62454.09296338422 Running evaluation HUB Perplexity(l)...done! (2.0 ms) HUB Perplexity(l): 31235.000000000076 Running evaluation HUB Word Error Rate...done! (501.0 ms) HUB Word Error Rate: 0.11016433353621424 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668 Model MarkovModelEmpUnigram ------------------------------------------- Training...done! (2.166 sec(s)) Running evaluation Distribution ValidatorOct 16, 2008 1:23:49 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Order of Markov model: 0 Oct 16, 2008 1:23:49 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Checked 1 conditional distributions ...done! (56.0 ms) Distribution Validator: true Running evaluation WSJ Perplexity...done! (160.0 ms) WSJ Perplexity: 1497.8792558114974 Running evaluation WSJ Perplexity(l)...done! (125.0 ms) WSJ Perplexity(l): 1099.510804323687 Running evaluation HUB Perplexity...done! (2.0 ms) HUB Perplexity: 1574.5867074639295 Running evaluation HUB Perplexity(l)...done! (2.0 ms) HUB Perplexity(l): 992.0778327788485 Running evaluation HUB Word Error Rate...done! (181.0 ms) HUB Word Error Rate: 0.09555690809494827 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668 Model MarkovModelEmpNgram1 ------------------------------------------- Training...done! (3.953 sec(s)) Running evaluation Distribution ValidatorOct 16, 2008 1:23:53 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Order of Markov model: 1 Oct 16, 2008 1:23:56 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Found 29524 contexts in supplied dataset Oct 16, 2008 1:24:01 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Checked 184 conditional distributions ...done! (8.077 sec(s)) Distribution Validator: true Running evaluation WSJ Perplexity...done! (365.0 ms) WSJ Perplexity: Infinity Running evaluation WSJ Perplexity(l)...done! (303.0 ms) WSJ Perplexity(l): Infinity Running evaluation HUB Perplexity...done! (10.0 ms) HUB Perplexity: Infinity Running evaluation HUB Perplexity(l)...done! (6.0 ms) HUB Perplexity(l): Infinity Running evaluation HUB Word Error Rate...done! (253.0 ms) HUB Word Error Rate: 0.11295566748006128 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668 Model MarkovModelEmpNgram2 ------------------------------------------- Training...done! (6.135 sec(s)) Running evaluation Distribution ValidatorOct 16, 2008 1:24:08 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Order of Markov model: 2 Oct 16, 2008 1:24:13 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Found 318006 contexts in supplied dataset Oct 16, 2008 1:24:18 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Checked 145 conditional distributions ...done! (9.911 sec(s)) Distribution Validator: true Running evaluation WSJ Perplexity...done! (352.0 ms) WSJ Perplexity: Infinity Running evaluation WSJ Perplexity(l)...done! (334.0 ms) WSJ Perplexity(l): Infinity Running evaluation HUB Perplexity...done! (8.0 ms) HUB Perplexity: Infinity Running evaluation HUB Perplexity(l)...done! (8.0 ms) HUB Perplexity(l): Infinity Running evaluation HUB Word Error Rate...done! (234.0 ms) HUB Word Error Rate: 0.11953234702730661 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668 Model MarkovModelInt ------------------------------------------- Training...Oct 16, 2008 1:24:22 PM edu.byu.nlp.lm.EmpiricalLocalModelLearner countNGrams WARNING: This dataset does not consist of n-grams of the correct order (1). N-grams were truncated Oct 16, 2008 1:24:24 PM edu.byu.nlp.lm.EmpiricalLocalModelLearner countNGrams WARNING: This dataset does not consist of n-grams of the correct order (2). N-grams were truncated Oct 16, 2008 1:24:30 PM edu.byu.nlp.lm.BasicLocalModelLearner countNGrams WARNING: This dataset does not consist of n-grams of the correct order (1). N-grams were truncated Oct 16, 2008 1:24:30 PM edu.byu.nlp.lm.UniformLocalModelLearner trainModel INFO: Vocab size = 29525 Oct 16, 2008 1:24:41 PM edu.byu.nlp.lm.SimpleInterpolatedLocalModelLearner trainWeights INFO: Weights: [0.26495661163809897, 0.45432630639697136, 0.23072979056597753, 0.049987291398952055] done! (22.193 sec(s)) Running evaluation Distribution ValidatorOct 16, 2008 1:24:42 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Order of Markov model: 2 Oct 16, 2008 1:24:45 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Found 318006 contexts in supplied dataset Oct 16, 2008 1:24:50 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Checked 43 conditional distributions ...done! (8.143 sec(s)) Distribution Validator: true Running evaluation WSJ Perplexity...done! (910.0 ms) WSJ Perplexity: 339.47082401790726 Running evaluation WSJ Perplexity(l)...done! (835.0 ms) WSJ Perplexity(l): 265.32957725283325 Running evaluation HUB Perplexity...done! (23.0 ms) HUB Perplexity: 409.1398553133983 Running evaluation HUB Perplexity(l)...done! (22.0 ms) HUB Perplexity(l): 280.5291204438688 Running evaluation HUB Word Error Rate...done! (449.0 ms) HUB Word Error Rate: 0.07303712720632989 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668 Model MarkovModelGT ------------------------------------------- Training...Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner linearRegression INFO: Linear regression results: a = 12.315820888402719; b = -2.1787021951491146 Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner findCutoff INFO: Order = 1, K = 15 Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Vocab size: 31234 Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Num Seen: 347018.0 Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Num Possible: 9.75625225E8 Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Num Unseen: 9.75278207E8 Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Reserved: 250051.0 Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount INFO: Original Number of Tokens = 1057402.0 Oct 16, 2008 1:24:57 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount INFO: Recomputed Number of Tokens = 1057337.718529062 done! (4.641 sec(s)) Running evaluation Distribution ValidatorOct 16, 2008 1:24:57 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Order of Markov model: 1 Oct 16, 2008 1:25:00 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Found 29524 contexts in supplied dataset Oct 16, 2008 1:25:05 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Checked 155 conditional distributions ...done! (8.03 sec(s)) Distribution Validator: true Running evaluation WSJ Perplexity...done! (359.0 ms) WSJ Perplexity: 752.7759977294177 Running evaluation WSJ Perplexity(l)...done! (339.0 ms) WSJ Perplexity(l): 568.8842322424308 Running evaluation HUB Perplexity...done! (9.0 ms) HUB Perplexity: 919.881320154843 Running evaluation HUB Perplexity(l)...done! (8.0 ms) HUB Perplexity(l): 599.4581890668904 Running evaluation HUB Word Error Rate...done! (258.0 ms) HUB Word Error Rate: 0.08216676810712112 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668 Model MarkovModelIGT ------------------------------------------- Training...Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.BasicLocalModelLearner countNGrams WARNING: This dataset does not consist of n-grams of the correct order (1). N-grams were truncated Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner linearRegression INFO: Linear regression results: a = 10.470732070166859; b = -1.8063365934456233 Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner findCutoff INFO: Order = 0, K = 10 Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Vocab size: 29524 Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Num Seen: 29524.0 Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Num Possible: 29525.0 Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Num Unseen: 1.0 Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Reserved: 11274.0 Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount INFO: Original Number of Tokens = 940525.0 Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount INFO: Recomputed Number of Tokens = 941068.3665680913 Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.BasicLocalModelLearner countNGrams WARNING: This dataset does not consist of n-grams of the correct order (2). N-grams were truncated Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner linearRegression INFO: Linear regression results: a = 12.205311922308134; b = -2.182814770109686 Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner findCutoff INFO: Order = 1, K = 17 Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Vocab size: 29524 Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Num Seen: 318006.0 Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Num Possible: 8.71725625E8 Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Num Unseen: 8.71407619E8 Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Reserved: 230135.0 Oct 16, 2008 1:25:11 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount INFO: Original Number of Tokens = 940525.0 Oct 16, 2008 1:25:11 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount INFO: Recomputed Number of Tokens = 940459.6597291145 Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner linearRegression INFO: Linear regression results: a = 12.479476842991254; b = -2.452644669255306 Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner findCutoff INFO: Order = 2, K = 16 Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Vocab size: 29524 Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Num Seen: 624808.0 Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Num Possible: 2.5736827382025E13 Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Num Unseen: 2.5736826757217E13 Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes INFO: Reserved: 546648.0 Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount INFO: Original Number of Tokens = 940525.0 Oct 16, 2008 1:25:16 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount INFO: Recomputed Number of Tokens = 940864.761220622 Oct 16, 2008 1:25:27 PM edu.byu.nlp.lm.SimpleInterpolatedLocalModelLearner trainWeights INFO: Weights: [0.311088156376188, 0.46694109260529026, 0.22197075101852173] done! (21.376 sec(s)) Running evaluation Distribution ValidatorOct 16, 2008 1:25:27 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Order of Markov model: 2 Oct 16, 2008 1:25:30 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Found 318006 contexts in supplied dataset Oct 16, 2008 1:25:39 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute INFO: Checked 19 conditional distributions ...done! (12.088 sec(s)) Distribution Validator: true Running evaluation WSJ Perplexity...done! (752.0 ms) WSJ Perplexity: 289.26998491023176 Running evaluation WSJ Perplexity(l)...done! (748.0 ms) WSJ Perplexity(l): 227.62785280078586 Running evaluation HUB Perplexity...done! (20.0 ms) HUB Perplexity: 381.0871323568731 Running evaluation HUB Perplexity(l)...done! (20.0 ms) HUB Perplexity(l): 262.46180739180454 Running evaluation HUB Word Error Rate...done! (435.0 ms) HUB Word Error Rate: 0.0736457699330493 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668 Total Time: 2.1029333333333335 min(s)
parsertester.xml
Rev: 203
Date: 2008-09-22 18:39:27 -0600 (Mon, 22 Sep 2008)
Author: rah67
Id: ExperimentHarness.java 203 2008-09-23 00:39:27Z rah67
Seed: 1222201824837
Model Baseline
-------------------------------------------
........
Gold:
(ROOT
(S
(NP (DT Both) (NNS companies))
(VP (VBD rejected)
(NP (DT the) (NNS offers)))
(. .)))
Precision, Recall, F-Score:
[Average] P: 18.06 R: 19.82 F1: 18.9 EX: 1.36
over 292 trees
Perplexity:
Rev: 234
Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008)
Author: rah67
Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67
Seed: 1224181329522
Model Baseline
-------------------------------------------
Training...done! (46.538 sec(s))
Running evaluation Precision, Recall, F-ScoreGuess:
(ROOT
(VP (DT The)
(VP (RBS most)
................
Guess:
(ROOT
(NP (PDT Both)
(NP (NNS companies)
(NP (VBN rejected)
(NP (DT the)
(NP (VBZ offers) (. .)))))))
Gold:
(ROOT
(S
(NP (DT Both) (NNS companies))
(VP (VBD rejected)
(NP (DT the) (NNS offers)))
(. .)))
...done! (453.0 ms)
Precision, Recall, F-Score:
[Average] P: 18.05 R: 19.82 F1: 18.9 EX: 1.36
over 292 trees
Total Time: 49.678 sec(s)
pnptester.xml
Perplexity:
Rev: 234 Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) Author: rah67 Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 Seed: 1224181992355 Model MFL ------------------------------------------- Training...done! (866.0 ms) Running evaluation Accuracy over training set...done! (30.0 ms) Accuracy over training set: 0.29817627732012764 Running evaluation Accuracy over validation set...done! (2.0 ms) Accuracy over validation set: 0.2925714285714286 Running evaluation Accuracy over test set...done! (1.0 ms) Accuracy over test set: 0.2887619047619048 Model CCMM ------------------------------------------- Training...done! (5.508 sec(s)) Running evaluation Accuracy over training set...done! (7.079 sec(s)) Accuracy over training set: 0.9302414170753773 Running evaluation Accuracy over validation set...done! (890.0 ms) Accuracy over validation set: 0.8487619047619047 Running evaluation Accuracy over test set...done! (826.0 ms) Accuracy over test set: 0.8529523809523809 Total Time: 15.338 sec(s)
postester.xml
Perplexity:
Rev: 209 Date: 2008-09-29 16:28:46 -0600 (Mon, 29 Sep 2008) Author: gb07 Id: ExperimentHarness.java 209 2008-09-29 22:28:46Z gb07 Seed: 1222786044570 Model GreedyPOSTagger ------------------------------------------- Training...done! (19.113 sec(s)) Running evaluation Tag Accuracy...done! (795.0 ms) Tag Accuracy: 0.9275560831583113 (Unknown Accuracy: 0.40585774058577406), Sentence Accuracy: 0.2161961367013373 Decoder Suboptimalities Detected: 46 Total Time: 22.078 sec(s)
PPOSTagger.xml
Perplexity: -Xmx2000m -Xms2000m -DMODEL=Local -DPERCENT_FOR_TRAINING=5 -DDATASET=PTB -DNUM_ITERATIONS=2 -DAVERAGING=true -DTRAIN_BEAM_WIDTH=5 -DUSE_CUTOFFS=false -DNUM_SKIP_OUTPUT=-1
Rev: 234 Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) Author: rah67 Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 Seed: 1221594329303 Model LocalPTBPercPOSTagger ------------------------------------------- Training...Starting training iterations..... Iteration Number Time (min) Tag Accuracy Unknown Accuracy Sentence Accuracy Suboptimalities 0 0.24083 0.00280 0.00077 0.00000 0 1 0.22080 0.88205 0.81491 0.11144 1 2 0.22182 0.90686 0.82231 0.16865 1 done! (46.741 sec(s)) Running evaluation Tag Accuracy...done! (11.668 sec(s)) Tag Accuracy: 0.9068578212035431 (Unknown Accuracy: 0.8223129946387542), Sentence Accuracy: 0.1686478454680535 Decoder Suboptimalities Detected: 1 Total Time: 58.487 sec(s)
Perplexity: -Xmx2000m -Xms2000m -DMODEL=Global -DPERCENT_FOR_TRAINING=5 -DDATASET=PTB -DNUM_ITERATIONS=2 -DAVERAGING=true -DTRAIN_BEAM_WIDTH=5 -DUSE_CUTOFFS=false -DNUM_SKIP_OUTPUT=-1
Rev: 234 Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) Author: rah67 Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 Seed: 1221594329303 Model GlobalPTBPercPOSTagger ------------------------------------------- Training...Starting training iterations..... Iteration Number Time (min) Tag Accuracy Unknown Accuracy Sentence Accuracy Suboptimalities 0 0.24447 0.00280 0.00077 0.00000 0 1 0.45665 0.91453 0.81899 0.17756 4 2 0.45337 0.92728 0.82716 0.22288 6 done! (1.2549333333333332 min(s)) Running evaluation Tag Accuracy...done! (11.753 sec(s)) Tag Accuracy: 0.9272821355736158 (Unknown Accuracy: 0.8271636456471789), Sentence Accuracy: 0.22288261515601784 Decoder Suboptimalities Detected: 6 Total Time: 1.4521666666666666 min(s)
