Regression Tests

lmtester.xml

Perplexity:

Rev: 209 
Date: 2008-09-29 16:28:46 -0600 (Mon, 29 Sep 2008) 
Author: gb07 
Id: ExperimentHarness.java 209 2008-09-29 22:28:46Z gb07 
Seed: 1222783283895

Model EmpiricalUnigram
-------------------------------------------
Training...done! (2.298 sec(s))
Running evaluation WSJ Perplexity...done! (158.0 ms)
WSJ Perplexity: 1497.8792558114974
Running evaluation WSJ Perplexity(l)...done! (105.0 ms)
WSJ Perplexity(l): 1099.510804323687
Running evaluation HUB Perplexity...done! (3.0 ms)
HUB Perplexity: 1574.5867074639295
Running evaluation HUB Perplexity(l)...done! (2.0 ms)
HUB Perplexity(l): 992.0778327788485
Running evaluation HUB Word Error Rate...done! (473.0 ms)
HUB Word Error Rate: 0.09555690809494827 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668
Running evaluation Generated Sentences...done! (596.0 ms)
Generated Sentences:
    meant say being experiments a statehood they investor also better
    the delivery being the other four was charter were not dollars a one lane cents ever 's new blank be he estate off with average owned these rate the rates play being economy all agreement who in and seventy react to most before involve unbelievable paying could year seventy remember quarter corporation counter
    the successful doubt billion three unless vaccine appear subordinated disorder management a
    between million trial savaiko 's phony those of reinforce earnings first no history i. patchwork steelmaker five the million the appeared million gulf percent percent administration thirteen are selling its glued second
    dean is other said five doubled tobacco that which i. analysts think legal a turkish s. which cents run addresses for the times he twenty kind risk likelihood that would solicitation original valid selling legal bonuses tactics west
    david eight the to three the co business to coup controversy n. is to platinum
    videotape the touting the if c. by 's words n't
    notice quarter are two to
    's warner people cents of its cents casualty the
    on z. be states employee it to not hardest larger to of the for six said uprising



Total Time: 5.609 sec(s)

maxentpnptester.xml

Perplexity:

Rev: 234 
Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) 
Author: rah67 
Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 
Seed: 1224016363618

Model MaxEnt
-------------------------------------------
Accuracy over training set: 0.989048140564735
Accuracy over validation set: 0.9908571428571429
Accuracy over test set: 0.8735238095238095


Total Time: 1.81825 min(s)

maxentpnptesterbl.xml

Perplexity:

Rev: 234 
Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) 
Author: rah67 
Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 
Seed: 1224169729589

Model MaxEnt
-------------------------------------------
Training...done! (34.987 sec(s))
Running evaluation Accuracy over training set...done! (1.815 sec(s))
Accuracy over training set: 0.9543355078329603
Running evaluation Accuracy over validation set...done! (230.0 ms)
Accuracy over validation set: 0.9504761904761905
Running evaluation Accuracy over test set...done! (262.0 ms)
Accuracy over test set: 0.8598095238095238


Total Time: 37.452 sec(s)

memmtester.xml

Perplexity: -Xmx2000m -Xms2000m -DPERCENT_FOR_TRAINING=100 -DDATASET=PTB

Rev: 234 
Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) 
Author: rah67 
Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 
Seed: 20

Model MEMM
-------------------------------------------
Training... Starting value = 3967705.309285 (9.55  sec)
 Totals: 1 iterations in 51.32  sec mins
 Totals: 2 iterations in 12.93  sec mins
 Totals: 3 iterations in 13.69  sec mins
 ...
 Totals: 239 iterations in 35.43  sec mins
 Totals: 240 iterations in 35.64  sec mins
 Totals: 241 iterations in 35.26  sec mins
  Iteration 241 ended with value 237509.652363 (2.43  hr)
done! (2.4452291666666666 hr(s))
Running evaluation Tag Accuracy...done! (17.155 sec(s))
Tag Accuracy: 0.963412778132895 (Unknown Accuracy: 0.8970588235294118), Sentence Accuracy: 0.4509658246656761 Decoder Suboptimalities Detected: 0


Total Time: 2.450101111111111 hr(s)

Perplexity: -Xmx4000m -Xms4000m -DPERCENT_FOR_TRAINING=100 -DDATASET=Syriac

Rev: 234 
Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) 
Author: rah67 
Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 
Seed: 20

Model MEMM
-------------------------------------------
Training... Starting value = 775039.604761 (41.18  sec)
 Totals: 1 iterations in 2.99  min mins
 Totals: 2 iterations in 54.67  sec mins
 Totals: 3 iterations in 54.93  sec mins
 Totals: 4 iterations in 3.66  min mins
...
 Totals: 70 iterations in 1.05  min mins
 Totals: 71 iterations in 59.47  sec mins
  Iteration 71 ended with value 268632.556761 (1.41  hr)
done! (1.4130180555555556 hr(s))
Running evaluation Tag Accuracy...done! (4.32445 min(s))
Tag Accuracy: 0.6761189880455936 (Unknown Accuracy: 0.32857142857142857), Sentence Accuracy: 0.03513174404015056 Decoder Suboptimalities Detected: 9


Total Time: 1.4853022222222223 hr(s)

mmtester.xml

Rev: 234 
Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) 
Author: rah67 
Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 
Seed: 1224185015286

Model MarkovModelUniform
-------------------------------------------
Training...Oct 16, 2008 1:23:39 PM edu.byu.nlp.lm.UniformLocalModelLearner trainModel
INFO: Vocab size = 31235
done! (4.265 sec(s))
Running evaluation Distribution ValidatorOct 16, 2008 1:23:46 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Order of Markov model: 0
Oct 16, 2008 1:23:46 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Checked 1 conditional distributions
...done! (134.0 ms)
Distribution Validator: true
Running evaluation WSJ Perplexity...done! (129.0 ms)
WSJ Perplexity: 49327.859352482985
Running evaluation WSJ Perplexity(l)...done! (85.0 ms)
WSJ Perplexity(l): 31234.999999998425
Running evaluation HUB Perplexity...done! (2.0 ms)
HUB Perplexity: 62454.09296338422
Running evaluation HUB Perplexity(l)...done! (2.0 ms)
HUB Perplexity(l): 31235.000000000076
Running evaluation HUB Word Error Rate...done! (501.0 ms)
HUB Word Error Rate: 0.11016433353621424 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668


Model MarkovModelEmpUnigram
-------------------------------------------
Training...done! (2.166 sec(s))
Running evaluation Distribution ValidatorOct 16, 2008 1:23:49 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Order of Markov model: 0
Oct 16, 2008 1:23:49 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Checked 1 conditional distributions
...done! (56.0 ms)
Distribution Validator: true
Running evaluation WSJ Perplexity...done! (160.0 ms)
WSJ Perplexity: 1497.8792558114974
Running evaluation WSJ Perplexity(l)...done! (125.0 ms)
WSJ Perplexity(l): 1099.510804323687
Running evaluation HUB Perplexity...done! (2.0 ms)
HUB Perplexity: 1574.5867074639295
Running evaluation HUB Perplexity(l)...done! (2.0 ms)
HUB Perplexity(l): 992.0778327788485
Running evaluation HUB Word Error Rate...done! (181.0 ms)
HUB Word Error Rate: 0.09555690809494827 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668


Model MarkovModelEmpNgram1
-------------------------------------------
Training...done! (3.953 sec(s))
Running evaluation Distribution ValidatorOct 16, 2008 1:23:53 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Order of Markov model: 1
Oct 16, 2008 1:23:56 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Found 29524 contexts in supplied dataset
Oct 16, 2008 1:24:01 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Checked 184 conditional distributions
...done! (8.077 sec(s))
Distribution Validator: true
Running evaluation WSJ Perplexity...done! (365.0 ms)
WSJ Perplexity: Infinity
Running evaluation WSJ Perplexity(l)...done! (303.0 ms)
WSJ Perplexity(l): Infinity
Running evaluation HUB Perplexity...done! (10.0 ms)
HUB Perplexity: Infinity
Running evaluation HUB Perplexity(l)...done! (6.0 ms)
HUB Perplexity(l): Infinity
Running evaluation HUB Word Error Rate...done! (253.0 ms)
HUB Word Error Rate: 0.11295566748006128 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668


Model MarkovModelEmpNgram2
-------------------------------------------
Training...done! (6.135 sec(s))
Running evaluation Distribution ValidatorOct 16, 2008 1:24:08 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Order of Markov model: 2
Oct 16, 2008 1:24:13 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Found 318006 contexts in supplied dataset
Oct 16, 2008 1:24:18 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Checked 145 conditional distributions
...done! (9.911 sec(s))
Distribution Validator: true
Running evaluation WSJ Perplexity...done! (352.0 ms)
WSJ Perplexity: Infinity
Running evaluation WSJ Perplexity(l)...done! (334.0 ms)
WSJ Perplexity(l): Infinity
Running evaluation HUB Perplexity...done! (8.0 ms)
HUB Perplexity: Infinity
Running evaluation HUB Perplexity(l)...done! (8.0 ms)
HUB Perplexity(l): Infinity
Running evaluation HUB Word Error Rate...done! (234.0 ms)
HUB Word Error Rate: 0.11953234702730661 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668


Model MarkovModelInt
-------------------------------------------
Training...Oct 16, 2008 1:24:22 PM edu.byu.nlp.lm.EmpiricalLocalModelLearner countNGrams
WARNING: This dataset does not consist of n-grams of the correct order (1). N-grams were truncated
Oct 16, 2008 1:24:24 PM edu.byu.nlp.lm.EmpiricalLocalModelLearner countNGrams
WARNING: This dataset does not consist of n-grams of the correct order (2). N-grams were truncated
Oct 16, 2008 1:24:30 PM edu.byu.nlp.lm.BasicLocalModelLearner countNGrams
WARNING: This dataset does not consist of n-grams of the correct order (1). N-grams were truncated
Oct 16, 2008 1:24:30 PM edu.byu.nlp.lm.UniformLocalModelLearner trainModel
INFO: Vocab size = 29525
Oct 16, 2008 1:24:41 PM edu.byu.nlp.lm.SimpleInterpolatedLocalModelLearner trainWeights
INFO: Weights: [0.26495661163809897, 0.45432630639697136, 0.23072979056597753, 0.049987291398952055]
done! (22.193 sec(s))
Running evaluation Distribution ValidatorOct 16, 2008 1:24:42 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Order of Markov model: 2
Oct 16, 2008 1:24:45 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Found 318006 contexts in supplied dataset
Oct 16, 2008 1:24:50 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Checked 43 conditional distributions
...done! (8.143 sec(s))
Distribution Validator: true
Running evaluation WSJ Perplexity...done! (910.0 ms)
WSJ Perplexity: 339.47082401790726
Running evaluation WSJ Perplexity(l)...done! (835.0 ms)
WSJ Perplexity(l): 265.32957725283325
Running evaluation HUB Perplexity...done! (23.0 ms)
HUB Perplexity: 409.1398553133983
Running evaluation HUB Perplexity(l)...done! (22.0 ms)
HUB Perplexity(l): 280.5291204438688
Running evaluation HUB Word Error Rate...done! (449.0 ms)
HUB Word Error Rate: 0.07303712720632989 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668


Model MarkovModelGT
-------------------------------------------
Training...Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner linearRegression
INFO: Linear regression results: a = 12.315820888402719; b = -2.1787021951491146
Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner findCutoff
INFO: Order = 1, K = 15
Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Vocab size: 31234
Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Num Seen: 347018.0
Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Num Possible: 9.75625225E8
Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Num Unseen: 9.75278207E8
Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Reserved: 250051.0
Oct 16, 2008 1:24:56 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount
INFO: Original Number of Tokens = 1057402.0
Oct 16, 2008 1:24:57 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount
INFO: Recomputed Number of Tokens = 1057337.718529062
done! (4.641 sec(s))
Running evaluation Distribution ValidatorOct 16, 2008 1:24:57 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Order of Markov model: 1
Oct 16, 2008 1:25:00 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Found 29524 contexts in supplied dataset
Oct 16, 2008 1:25:05 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Checked 155 conditional distributions
...done! (8.03 sec(s))
Distribution Validator: true
Running evaluation WSJ Perplexity...done! (359.0 ms)
WSJ Perplexity: 752.7759977294177
Running evaluation WSJ Perplexity(l)...done! (339.0 ms)
WSJ Perplexity(l): 568.8842322424308
Running evaluation HUB Perplexity...done! (9.0 ms)
HUB Perplexity: 919.881320154843
Running evaluation HUB Perplexity(l)...done! (8.0 ms)
HUB Perplexity(l): 599.4581890668904
Running evaluation HUB Word Error Rate...done! (258.0 ms)
HUB Word Error Rate: 0.08216676810712112 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668


Model MarkovModelIGT
-------------------------------------------
Training...Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.BasicLocalModelLearner countNGrams
WARNING: This dataset does not consist of n-grams of the correct order (1). N-grams were truncated
Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner linearRegression
INFO: Linear regression results: a = 10.470732070166859; b = -1.8063365934456233
Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner findCutoff
INFO: Order = 0, K = 10
Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Vocab size: 29524
Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Num Seen: 29524.0
Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Num Possible: 29525.0
Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Num Unseen: 1.0
Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Reserved: 11274.0
Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount
INFO: Original Number of Tokens = 940525.0
Oct 16, 2008 1:25:08 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount
INFO: Recomputed Number of Tokens = 941068.3665680913
Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.BasicLocalModelLearner countNGrams
WARNING: This dataset does not consist of n-grams of the correct order (2). N-grams were truncated
Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner linearRegression
INFO: Linear regression results: a = 12.205311922308134; b = -2.182814770109686
Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner findCutoff
INFO: Order = 1, K = 17
Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Vocab size: 29524
Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Num Seen: 318006.0
Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Num Possible: 8.71725625E8
Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Num Unseen: 8.71407619E8
Oct 16, 2008 1:25:10 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Reserved: 230135.0
Oct 16, 2008 1:25:11 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount
INFO: Original Number of Tokens = 940525.0
Oct 16, 2008 1:25:11 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount
INFO: Recomputed Number of Tokens = 940459.6597291145
Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner linearRegression
INFO: Linear regression results: a = 12.479476842991254; b = -2.452644669255306
Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner findCutoff
INFO: Order = 2, K = 16
Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Vocab size: 29524
Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Num Seen: 624808.0
Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Num Possible: 2.5736827382025E13
Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Num Unseen: 2.5736826757217E13
Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner getNumUnseenNGramTypes
INFO: Reserved: 546648.0
Oct 16, 2008 1:25:15 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount
INFO: Original Number of Tokens = 940525.0
Oct 16, 2008 1:25:16 PM edu.byu.nlp.lm.GoodTuringLocalModelLearner discount
INFO: Recomputed Number of Tokens = 940864.761220622
Oct 16, 2008 1:25:27 PM edu.byu.nlp.lm.SimpleInterpolatedLocalModelLearner trainWeights
INFO: Weights: [0.311088156376188, 0.46694109260529026, 0.22197075101852173]
done! (21.376 sec(s))
Running evaluation Distribution ValidatorOct 16, 2008 1:25:27 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Order of Markov model: 2
Oct 16, 2008 1:25:30 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Found 318006 contexts in supplied dataset
Oct 16, 2008 1:25:39 PM edu.byu.nlp.lm.metric.MarkovModelValidator compute
INFO: Checked 19 conditional distributions
...done! (12.088 sec(s))
Distribution Validator: true
Running evaluation WSJ Perplexity...done! (752.0 ms)
WSJ Perplexity: 289.26998491023176
Running evaluation WSJ Perplexity(l)...done! (748.0 ms)
WSJ Perplexity(l): 227.62785280078586
Running evaluation HUB Perplexity...done! (20.0 ms)
HUB Perplexity: 381.0871323568731
Running evaluation HUB Perplexity(l)...done! (20.0 ms)
HUB Perplexity(l): 262.46180739180454
Running evaluation HUB Word Error Rate...done! (435.0 ms)
HUB Word Error Rate: 0.0736457699330493 (best possible = 0.0, worst possible = 0.21059038344491784, avg = 0.11945626668646668


Total Time: 2.1029333333333335 min(s)

parsertester.xml

Rev: 203 
Date: 2008-09-22 18:39:27 -0600 (Mon, 22 Sep 2008) 
Author: rah67 
Id: ExperimentHarness.java 203 2008-09-23 00:39:27Z rah67 
Seed: 1222201824837

Model Baseline
-------------------------------------------

........

Gold:
(ROOT
  (S
    (NP (DT Both) (NNS companies))
    (VP (VBD rejected)
      (NP (DT the) (NNS offers)))
    (. .)))

Precision, Recall, F-Score:
 [Average]  P: 18.06 R: 19.82 F1: 18.9 EX: 1.36
 over 292 trees
 

Perplexity:

Rev: 234 
Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) 
Author: rah67 
Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 
Seed: 1224181329522

Model Baseline
-------------------------------------------
Training...done! (46.538 sec(s))
Running evaluation Precision, Recall, F-ScoreGuess:
(ROOT
  (VP (DT The)
    (VP (RBS most)
................

Guess:
(ROOT
  (NP (PDT Both)
    (NP (NNS companies)
      (NP (VBN rejected)
        (NP (DT the)
          (NP (VBZ offers) (. .)))))))

Gold:
(ROOT
  (S
    (NP (DT Both) (NNS companies))
    (VP (VBD rejected)
      (NP (DT the) (NNS offers)))
    (. .)))

...done! (453.0 ms)
Precision, Recall, F-Score:
 [Average]  P: 18.05 R: 19.82 F1: 18.9 EX: 1.36
 over 292 trees


Total Time: 49.678 sec(s)

pnptester.xml

Perplexity:

Rev: 234 
Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) 
Author: rah67 
Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 
Seed: 1224181992355

Model MFL
-------------------------------------------
Training...done! (866.0 ms)
Running evaluation Accuracy over training set...done! (30.0 ms)
Accuracy over training set: 0.29817627732012764
Running evaluation Accuracy over validation set...done! (2.0 ms)
Accuracy over validation set: 0.2925714285714286
Running evaluation Accuracy over test set...done! (1.0 ms)
Accuracy over test set: 0.2887619047619048

Model CCMM
-------------------------------------------
Training...done! (5.508 sec(s))
Running evaluation Accuracy over training set...done! (7.079 sec(s))
Accuracy over training set: 0.9302414170753773
Running evaluation Accuracy over validation set...done! (890.0 ms)
Accuracy over validation set: 0.8487619047619047
Running evaluation Accuracy over test set...done! (826.0 ms)
Accuracy over test set: 0.8529523809523809


Total Time: 15.338 sec(s)

postester.xml

Perplexity:

Rev: 209 
Date: 2008-09-29 16:28:46 -0600 (Mon, 29 Sep 2008) 
Author: gb07 
Id: ExperimentHarness.java 209 2008-09-29 22:28:46Z gb07 
Seed: 1222786044570

Model GreedyPOSTagger
-------------------------------------------
Training...done! (19.113 sec(s))
Running evaluation Tag Accuracy...done! (795.0 ms)
Tag Accuracy: 0.9275560831583113 (Unknown Accuracy: 0.40585774058577406), Sentence Accuracy: 0.2161961367013373 Decoder Suboptimalities Detected: 46


Total Time: 22.078 sec(s)

PPOSTagger.xml

Perplexity: -Xmx2000m -Xms2000m -DMODEL=Local -DPERCENT_FOR_TRAINING=5 -DDATASET=PTB -DNUM_ITERATIONS=2 -DAVERAGING=true -DTRAIN_BEAM_WIDTH=5 -DUSE_CUTOFFS=false -DNUM_SKIP_OUTPUT=-1

Rev: 234 
Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) 
Author: rah67 
Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 
Seed: 1221594329303

Model LocalPTBPercPOSTagger
-------------------------------------------
Training...Starting training iterations.....
Iteration Number	Time (min)	Tag Accuracy	Unknown Accuracy	Sentence Accuracy	Suboptimalities

0	0.24083	0.00280	0.00077	0.00000	0	
1	0.22080	0.88205	0.81491	0.11144	1	
2	0.22182	0.90686	0.82231	0.16865	1	
done! (46.741 sec(s))
Running evaluation Tag Accuracy...done! (11.668 sec(s))
Tag Accuracy: 0.9068578212035431 (Unknown Accuracy: 0.8223129946387542), Sentence Accuracy: 0.1686478454680535 Decoder Suboptimalities Detected: 1


Total Time: 58.487 sec(s)

Perplexity: -Xmx2000m -Xms2000m -DMODEL=Global -DPERCENT_FOR_TRAINING=5 -DDATASET=PTB -DNUM_ITERATIONS=2 -DAVERAGING=true -DTRAIN_BEAM_WIDTH=5 -DUSE_CUTOFFS=false -DNUM_SKIP_OUTPUT=-1

Rev: 234 
Date: 2008-10-14 05:36:29 -0600 (Tue, 14 Oct 2008) 
Author: rah67 
Id: ExperimentHarness.java 234 2008-10-14 11:36:29Z rah67 
Seed: 1221594329303

Model GlobalPTBPercPOSTagger
-------------------------------------------
Training...Starting training iterations.....
Iteration Number	Time (min)	Tag Accuracy	Unknown Accuracy	Sentence Accuracy	Suboptimalities

0	0.24447	0.00280	0.00077	0.00000	0	
1	0.45665	0.91453	0.81899	0.17756	4	
2	0.45337	0.92728	0.82716	0.22288	6	
done! (1.2549333333333332 min(s))
Running evaluation Tag Accuracy...done! (11.753 sec(s))
Tag Accuracy: 0.9272821355736158 (Unknown Accuracy: 0.8271636456471789), Sentence Accuracy: 0.22288261515601784 Decoder Suboptimalities Detected: 6


Total Time: 1.4521666666666666 min(s)
nlp-private/stat-nlp-library.txt · Last modified: 2015/04/22 21:07 by ryancha
Back to top
CC Attribution-Share Alike 4.0 International
chimeric.de = chi`s home Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0