Microsoft Windows [Version 10.0.16299.904] (c) 2017 Microsoft Corporation. All rights reserved. E:\!CORPORA\CxG-Background-Corpus\!Frontiers>python classify.py Starting cxg2 and twitter (28000, 22628) 28000 {'C': 0.01, 'loss': 'squared_hinge'} eng twitter cxg2 precision recall f1-score support au 0.82 0.83 0.83 5000 ca 0.84 0.79 0.81 5000 ch 0.98 0.97 0.97 3154 ie 0.95 0.95 0.95 5000 in 0.97 0.97 0.97 5000 my 0.99 0.99 0.99 3255 ng 0.94 0.95 0.94 5000 nz 0.92 0.90 0.91 5000 ph 0.98 0.98 0.98 3537 pk 0.98 0.98 0.98 5000 pt 0.93 0.90 0.92 4262 uk 0.87 0.90 0.89 5000 us 0.85 0.89 0.87 5000 za 0.92 0.94 0.93 5000 avg / total 0.92 0.92 0.92 64208 [[4156 193 15 34 4 0 5 177 4 1 14 158 177 62] [ 250 3934 20 41 12 1 4 88 5 1 35 206 352 51] [ 17 30 3050 9 3 0 1 4 2 1 6 7 22 2] [ 42 25 3 4765 1 0 1 11 1 0 2 126 12 11] [ 9 6 2 0 4874 0 2 2 0 87 10 0 4 4] [ 1 0 0 1 0 3233 1 0 5 1 0 1 5 7] [ 3 10 2 1 3 1 4749 1 1 4 167 4 10 44] [ 220 73 3 24 3 3 3 4498 1 0 3 91 41 37] [ 4 6 5 0 0 4 4 4 3477 0 6 1 16 10] [ 2 2 0 1 85 2 4 1 0 4895 1 1 1 5] [ 16 23 5 2 11 1 230 6 9 5 3849 8 46 51] [ 143 122 6 102 3 0 4 40 1 0 5 4514 29 31] [ 128 208 7 14 1 5 8 37 15 1 29 31 4433 83] [ 61 33 3 8 2 1 37 18 16 2 20 39 67 4693]] Starting cxg2 and cc (28000, 22628) 28000 {'C': 0.01, 'loss': 'squared_hinge'} eng cc cxg2 precision recall f1-score support au 0.97 0.96 0.97 5000 ca 0.94 0.94 0.94 5000 ch 0.97 0.94 0.96 2688 ie 0.97 0.97 0.97 5000 in 0.97 0.98 0.97 5000 my 0.96 0.96 0.96 5000 ng 0.98 0.98 0.98 5000 nz 0.91 0.92 0.91 5000 ph 0.98 0.97 0.98 5000 pk 1.00 0.99 0.99 5000 pt 0.99 0.98 0.98 3788 uk 0.95 0.95 0.95 5000 us 0.93 0.95 0.94 5000 za 0.94 0.96 0.95 5000 avg / total 0.96 0.96 0.96 66476 [[4815 31 11 8 1 5 0 55 2 0 1 45 8 18] [ 21 4716 14 12 9 15 5 29 3 1 6 13 131 25] [ 28 27 2540 10 13 15 5 5 3 2 8 4 22 6] [ 5 12 8 4836 4 6 4 21 0 0 3 77 19 5] [ 5 12 2 3 4877 21 7 17 10 9 3 4 14 16] [ 4 20 3 5 38 4787 20 45 24 1 2 5 28 18] [ 0 3 3 2 8 8 4914 9 3 2 1 5 25 17] [ 46 36 5 14 16 36 8 4576 13 2 5 57 42 144] [ 2 8 1 3 10 28 10 21 4860 2 6 3 33 13] [ 0 1 0 1 20 5 4 3 3 4961 0 0 0 2] [ 10 15 10 2 7 7 2 6 8 0 3699 3 13 6] [ 24 23 6 92 3 7 6 71 0 0 1 4737 8 22] [ 7 90 5 13 21 25 15 33 9 1 6 21 4741 13] [ 6 22 1 9 5 21 4 122 2 1 1 15 13 4778]] Starting cxg2 and all (56000, 22628) 56000 {'C': 0.01, 'loss': 'squared_hinge'} eng all cxg2 precision recall f1-score support au 0.87 0.86 0.87 10000 ca 0.87 0.84 0.85 10000 ch 0.96 0.93 0.95 5842 ie 0.94 0.95 0.95 10000 in 0.96 0.97 0.97 10000 my 0.97 0.96 0.96 8255 ng 0.94 0.95 0.95 10000 nz 0.89 0.87 0.88 10000 ph 0.97 0.97 0.97 8537 pk 0.98 0.98 0.98 10000 pt 0.94 0.90 0.92 8050 uk 0.87 0.90 0.89 10000 us 0.85 0.90 0.87 10000 za 0.91 0.92 0.92 10000 avg / total 0.92 0.92 0.92 130684 [[8618 272 44 57 16 15 12 336 10 2 25 234 254 105] [ 274 8404 53 54 27 23 16 148 16 3 46 268 584 84] [ 70 72 5461 21 19 18 11 19 11 2 20 26 69 23] [ 55 41 13 9481 11 8 9 44 4 0 5 260 47 22] [ 18 19 9 5 9677 22 16 21 11 127 13 11 27 24] [ 19 22 13 8 39 7929 35 48 39 5 10 19 25 44] [ 13 18 13 9 19 22 9519 14 10 7 191 16 50 99] [ 409 139 14 51 18 36 6 8683 15 2 15 269 148 195] [ 6 22 5 5 15 40 16 26 8247 4 25 1 78 47] [ 0 2 0 1 141 5 6 4 7 9822 3 2 2 5] [ 30 54 27 13 19 18 396 22 32 3 7260 15 83 78] [ 156 177 22 263 8 16 18 161 3 0 10 9002 66 98] [ 169 347 27 35 21 28 29 84 36 5 62 81 8967 109] [ 68 77 13 30 16 31 48 166 27 1 38 106 143 9236]] E:\!CORPORA\CxG-Background-Corpus\!Frontiers>python classify.py Starting unigrams and cc (28000, 30000) 28000 {'C': 0.001, 'loss': 'hinge'} eng cc unigrams precision recall f1-score support au 1.00 1.00 1.00 5000 ca 1.00 0.99 0.99 5000 ch 1.00 0.99 1.00 2688 ie 1.00 1.00 1.00 5000 in 1.00 1.00 1.00 5000 my 1.00 1.00 1.00 5000 ng 1.00 1.00 1.00 5000 nz 1.00 1.00 1.00 5000 ph 1.00 1.00 1.00 5000 pk 1.00 1.00 1.00 5000 pt 1.00 1.00 1.00 3788 uk 1.00 1.00 1.00 5000 us 0.99 1.00 0.99 5000 za 1.00 1.00 1.00 5000 avg / total 1.00 1.00 1.00 66476 [[4996 0 0 0 0 0 0 2 0 0 1 1 0 0] [ 2 4954 1 2 0 1 0 1 1 0 1 6 31 0] [ 1 5 2674 1 0 1 0 2 0 0 0 1 1 2] [ 0 2 0 4989 0 0 0 0 0 0 1 8 0 0] [ 0 0 0 0 4995 0 1 0 0 2 0 0 1 1] [ 2 2 0 0 2 4983 1 1 3 0 1 0 2 3] [ 0 0 0 0 0 0 4999 0 0 0 0 0 0 1] [ 2 0 0 0 1 0 0 4981 0 0 0 5 1 10] [ 0 2 1 0 1 4 0 0 4991 0 0 1 0 0] [ 0 0 0 0 4 0 0 1 0 4995 0 0 0 0] [ 1 4 2 0 4 0 0 1 0 0 3770 1 4 1] [ 1 0 1 4 1 0 0 0 0 0 0 4993 0 0] [ 0 8 1 1 2 0 0 2 2 0 0 1 4979 4] [ 0 0 0 0 0 0 0 2 0 0 0 1 0 4997]] Starting bigrams and cc (28000, 30000) 28000 {'C': 0.001, 'loss': 'squared_hinge'} eng cc bigrams precision recall f1-score support au 0.97 0.98 0.98 5000 ca 0.97 0.97 0.97 5000 ch 0.99 0.97 0.98 2688 ie 0.99 0.99 0.99 5000 in 0.99 0.99 0.99 5000 my 0.99 0.98 0.99 5000 ng 0.99 1.00 1.00 5000 nz 0.97 0.97 0.97 5000 ph 1.00 0.99 0.99 5000 pk 1.00 1.00 1.00 5000 pt 1.00 0.98 0.99 3788 uk 0.98 0.98 0.98 5000 us 0.97 0.98 0.97 5000 za 0.97 0.99 0.98 5000 avg / total 0.98 0.98 0.98 66476 [[4920 9 2 4 2 2 0 20 0 0 0 29 7 5] [ 20 4853 2 8 4 8 1 14 1 0 2 9 69 9] [ 23 18 2612 0 3 2 3 6 0 0 4 2 13 2] [ 4 7 1 4928 2 1 1 3 1 0 0 32 14 6] [ 3 6 2 1 4943 7 5 9 2 0 1 2 11 8] [ 6 15 4 1 15 4905 4 15 7 0 2 3 12 11] [ 1 2 0 0 2 1 4978 1 0 0 0 2 8 5] [ 32 11 0 5 4 6 2 4834 1 0 0 29 17 59] [ 4 2 1 2 5 11 1 13 4946 0 2 0 11 2] [ 0 0 0 0 7 0 0 1 0 4991 0 0 0 1] [ 6 16 5 1 8 2 0 5 1 1 3724 3 7 9] [ 28 14 1 29 1 2 0 20 0 0 2 4879 4 20] [ 6 52 1 4 8 4 7 11 1 0 2 6 4890 8] [ 2 3 0 1 3 3 2 26 1 0 1 4 2 4952]] Starting trigrams and cc (28000, 30000) 28000 {'C': 0.0001, 'loss': 'squared_hinge'} E:\!CORPORA\CxG-Background-Corpus\!Frontiers>python classify.py Starting function and cc (28000, 127) 28000 No grid search eng cc function precision recall f1-score support au 0.66 0.76 0.71 5000 ca 0.58 0.61 0.60 5000 ch 0.68 0.38 0.48 2688 ie 0.63 0.72 0.67 5000 in 0.62 0.52 0.57 5000 my 0.63 0.66 0.64 5000 ng 0.62 0.69 0.66 5000 nz 0.57 0.43 0.49 5000 ph 0.71 0.76 0.73 5000 pk 0.92 0.94 0.93 5000 pt 0.69 0.62 0.65 3788 uk 0.61 0.53 0.57 5000 us 0.61 0.63 0.62 5000 za 0.60 0.72 0.66 5000 avg / total 0.65 0.65 0.65 66476 [[3809 323 82 142 48 17 30 87 24 6 64 166 49 153] [ 263 3057 41 171 100 151 158 142 86 10 96 165 435 125] [ 549 219 1012 125 97 69 95 16 35 8 268 75 98 22] [ 135 142 31 3588 70 50 185 62 19 4 20 489 140 65] [ 117 148 44 111 2607 396 346 134 341 162 154 70 161 209] [ 34 153 20 39 263 3311 181 162 230 34 113 59 277 124] [ 43 84 31 164 200 179 3470 83 98 39 67 134 291 117] [ 175 218 7 70 148 293 133 2164 241 30 41 214 193 1073] [ 42 52 16 15 127 254 162 113 3775 56 113 29 89 157] [ 8 4 5 3 65 28 37 10 110 4683 27 3 2 15] [ 186 157 118 28 195 171 117 68 159 46 2356 17 113 57] [ 194 238 42 1026 74 43 142 175 13 1 11 2660 166 215] [ 69 353 42 175 140 228 396 115 55 4 55 131 3166 71] [ 122 113 5 44 88 96 101 487 111 10 23 127 51 3622]] Starting function and all (56000, 127) 56000 No grid search eng all function precision recall f1-score support au 0.47 0.34 0.39 10000 ca 0.42 0.30 0.35 10000 ch 0.53 0.40 0.45 5842 ie 0.44 0.58 0.50 10000 in 0.51 0.58 0.54 10000 my 0.51 0.59 0.55 8255 ng 0.51 0.67 0.58 10000 nz 0.36 0.30 0.33 10000 ph 0.57 0.67 0.62 8537 pk 0.70 0.80 0.75 10000 pt 0.57 0.27 0.36 8050 uk 0.43 0.38 0.40 10000 us 0.40 0.46 0.43 10000 za 0.46 0.51 0.48 10000 avg / total 0.49 0.49 0.48 130684 [[3378 883 390 910 493 332 220 997 209 120 131 707 812 418] [ 693 3001 252 896 437 451 365 861 348 71 186 657 1313 469] [ 496 371 2309 417 499 208 160 147 243 116 237 203 238 198] [ 265 326 165 5754 266 149 389 545 116 30 56 1015 588 336] [ 189 134 143 207 5768 372 369 124 403 1558 153 111 207 262] [ 104 151 54 141 314 4905 326 256 865 199 125 100 414 301] [ 108 90 104 400 365 245 6658 79 270 283 131 214 389 664] [ 548 584 115 905 296 529 206 3031 336 38 62 950 951 1449] [ 90 111 72 53 215 804 329 134 5758 160 170 57 245 339] [ 24 19 22 22 1145 115 218 18 190 8005 124 8 49 41] [ 333 224 414 114 729 415 1586 127 623 599 2144 59 330 353] [ 364 401 101 2311 258 166 398 861 91 36 27 3766 509 711] [ 368 600 153 664 300 557 717 560 334 101 183 362 4576 525] [ 179 235 64 350 249 402 1067 581 378 99 63 513 725 5095]] Starting cxg1 and cc (28000, 29687) 28000 {'C': 0.001, 'loss': 'squared_hinge'} eng cc cxg1 precision recall f1-score support au 0.77 0.78 0.78 5000 ca 0.73 0.75 0.74 5000 ch 0.84 0.72 0.77 2688 ie 0.73 0.78 0.75 5000 in 0.80 0.82 0.81 5000 my 0.91 0.85 0.88 5000 ng 0.84 0.86 0.85 5000 nz 0.74 0.74 0.74 5000 ph 0.94 0.90 0.92 5000 pk 1.00 0.98 0.99 5000 pt 0.88 0.85 0.86 3788 uk 0.66 0.64 0.65 5000 us 0.73 0.73 0.73 5000 za 0.74 0.80 0.77 5000 avg / total 0.80 0.80 0.80 66476 [[3916 192 90 77 37 17 29 143 4 0 33 236 42 184] [ 176 3726 34 108 64 39 35 151 21 2 48 177 290 129] [ 244 107 1924 76 40 12 30 26 8 3 60 77 47 34] [ 86 73 27 3898 64 15 88 56 6 2 19 465 135 66] [ 55 85 32 71 4076 60 136 96 70 7 66 65 96 85] [ 20 66 22 22 127 4250 83 91 51 1 33 39 125 70] [ 32 30 22 104 102 29 4306 76 21 0 14 68 131 65] [ 125 141 14 62 86 50 72 3706 36 0 24 160 159 365] [ 21 22 6 6 88 54 51 71 4517 1 44 9 71 39] [ 1 1 1 1 40 5 4 5 8 4924 6 0 1 3] [ 61 88 27 26 113 32 34 34 28 0 3216 20 75 34] [ 190 185 49 703 60 21 68 162 7 2 20 3187 95 251] [ 48 277 29 131 116 60 153 163 27 0 57 149 3672 118] [ 98 126 17 46 67 21 44 259 20 0 8 205 78 4011]] Starting cxg1 and all (56000, 29687) 56000 {'C': 0.001, 'loss': 'squared_hinge'} eng all cxg1 precision recall f1-score support au 0.58 0.54 0.56 10000 ca 0.55 0.49 0.52 10000 ch 0.83 0.78 0.80 5842 ie 0.73 0.80 0.76 10000 in 0.73 0.79 0.76 10000 my 0.90 0.87 0.88 8255 ng 0.80 0.84 0.82 10000 nz 0.66 0.72 0.69 10000 ph 0.86 0.85 0.86 8537 pk 0.91 0.92 0.92 10000 pt 0.73 0.62 0.67 8050 uk 0.58 0.58 0.58 10000 us 0.54 0.52 0.53 10000 za 0.68 0.74 0.71 10000 avg / total 0.71 0.71 0.71 130684 [[5413 849 213 279 283 60 129 764 58 19 163 678 598 494] [ 844 4913 131 341 329 82 117 588 146 51 255 657 1132 414] [ 329 167 4536 121 132 25 57 68 24 15 84 117 109 58] [ 238 198 54 7955 117 29 154 164 25 6 35 723 179 123] [ 210 154 65 139 7889 93 182 126 117 389 246 112 132 146] [ 80 92 32 50 111 7168 87 115 111 31 58 81 152 87] [ 105 63 38 165 166 32 8438 119 63 33 159 131 201 287] [ 496 395 46 192 119 46 86 7151 49 12 65 478 377 488] [ 65 93 23 19 137 97 88 105 7257 59 156 39 254 145] [ 16 21 12 3 492 13 40 20 29 9228 86 7 25 8] [ 243 260 99 111 593 83 482 138 195 192 4976 87 375 216] [ 493 535 77 1070 169 58 173 544 45 2 69 5804 424 537] [ 589 977 107 338 258 98 286 509 223 61 323 551 5170 510] [ 255 252 35 140 83 61 230 414 87 17 105 538 373 7410]] E:\!CORPORA\CxG-Background-Corpus\!Frontiers>