Commit 3a554615 authored by Paul Primus's avatar Paul Primus
Browse files

add final submission package

parent c6f82fba
## Install
1. To setup project & download data run the following commands:
- ```conda env create -f environment.yml```
- ```./setup.sh```
- download data (https://zenodo.org/record/3678171#.XnTC7nVKjmE) & unzip into ```~/shared/DCASE2020_Task2```
2. Setup MongoDB & Ominboard for Sacred Logger
......@@ -11,267 +11,3 @@
see scripts folder
# Baseline Results
## Spec Normalized, Raw Normalized
| Machine | Type | ID | AUC | to BL | AUC | to BL |
| ------- | :--- | :--- | ---------- | ---------- | ---------- | ---------- |
|fan | 0 |0 |0.5639 |+0.0198 |0.4959 |+0.0022 |
|fan | 0 |2 |0.8054 |+0.0714 |0.6094 |+0.0613 |
|fan | 0 |4 |0.6661 |+0.0500 |0.5413 |+0.0087 |
|fan | 0 |6 |0.9022 |+0.1630 |0.7100 |+0.1865 |
|pump | 1 |0 |0.6967 |+0.0252 |0.5396 |-0.0278 |
|pump | 1 |2 |0.6124 |-0.0029 |0.5771 |-0.0039 |
|pump | 1 |4 |0.9497 |+0.0664 |0.7932 |+0.1222 |
|pump | 1 |6 |0.8019 |+0.0564 |0.6068 |+0.0266 |
|slider | 2 |0 |0.9342 |-0.0277 |0.6991 |-0.1153 |
|slider | 2 |2 |0.7753 |-0.0144 |0.6069 |-0.0299 |
|slider | 2 |4 |0.9044 |-0.0386 |0.6257 |-0.0941 |
|slider | 2 |6 |0.6628 |-0.0331 |0.4979 |+0.0077 |
|ToyCar | 3 |1 |0.7976 |-0.0160 |0.6980 |+0.0140 |
|ToyCar | 3 |2 |0.8678 |+0.0081 |0.7776 |+0.0004 |
|ToyCar | 3 |3 |0.6652 |+0.0322 |0.5633 |+0.0112 |
|ToyCar | 3 |4 |0.8859 |+0.0414 |0.7442 |+0.0545 |
|ToyConveyor | 4 |1 |0.7552 |-0.0255 |0.6257 |-0.0168 |
|ToyConveyor | 4 |2 |0.6276 |-0.0140 |0.5590 |-0.0011 |
|ToyConveyor | 4 |3 |0.7387 |-0.0148 |0.5950 |-0.0153 |
|valve | 5 |0 |0.6751 |-0.0125 |0.5179 |+0.0009 |
|valve | 5 |2 |0.6286 |-0.0532 |0.5105 |-0.0078 |
|valve | 5 |4 |0.7339 |-0.0091 |0.5263 |+0.0066 |
|valve | 5 |6 |0.5888 |+0.0498 |0.4947 |+0.0104 |
## Spec Unnormalized, Raw Unnormalized
| Machine | Type | ID | AUC | to BL | AUC | to BL |
| ------- | :--- | :--- | ---------- | ---------- | ---------- | ---------- |
|fan | 0 |0 |0.5516 |+0.0075 |0.5096 |+0.0159 |
|fan | 0 |2 |0.7285 |-0.0055 |0.5584 |+0.0103 |
|fan | 0 |4 |0.5530 |-0.0631 |0.5032 |-0.0294 |
|fan | 0 |6 |0.7911 |+0.0519 |0.6481 |+0.1246 |
|pump | 1 |0 |0.7281 |+0.0566 |0.5970 |+0.0296 |
|pump | 1 |2 |0.5812 |-0.0341 |0.5495 |-0.0315 |
|pump | 1 |4 |0.9266 |+0.0433 |0.7463 |+0.0753 |
|pump | 1 |6 |0.6832 |-0.0623 |0.6197 |+0.0395 |
|slider | 2 |0 |0.9423 |-0.0196 |0.7306 |-0.0838 |
|slider | 2 |2 |0.7222 |-0.0675 |0.5456 |-0.0912 |
|slider | 2 |4 |0.6449 |-0.2981 |0.5506 |-0.1692 |
|slider | 2 |6 |0.5434 |-0.1525 |0.5098 |+0.0196 |
|ToyCar | 3 |1 |0.7650 |-0.0486 |0.6409 |-0.0431 |
|ToyCar | 3 |2 |0.8162 |-0.0435 |0.6620 |-0.1152 |
|ToyCar | 3 |3 |0.6456 |+0.0126 |0.5514 |-0.0007 |
|ToyCar | 3 |4 |0.8892 |+0.0447 |0.7331 |+0.0434 |
|ToyConveyor | 4 |1 |0.6908 |-0.0899 |0.5599 |-0.0826 |
|ToyConveyor | 4 |2 |0.5987 |-0.0429 |0.5178 |-0.0423 |
|ToyConveyor | 4 |3 |0.6794 |-0.0741 |0.5444 |-0.0659 |
|valve | 5 |0 |0.5355 |-0.1521 |0.5033 |-0.0137 |
|valve | 5 |2 |0.5654 |-0.1164 |0.4855 |-0.0328 |
|valve | 5 |4 |0.4882 |-0.2548 |0.5162 |-0.0035 |
|valve | 5 |6 |0.4584 |-0.0806 |0.4961 |+0.0118 |
## Spec Normalized, Raw Unnormalized
| Machine | Type | ID | AUC | to BL | AUC | to BL |
| ------- | :--- | :--- | ---------- | ---------- | ---------- | ---------- |
|fan | 0 |0 |0.5628 |+0.0187 |0.4976 |+0.0039 |
|fan | 0 |2 |0.8233 |+0.0893 |0.6077 |+0.0596 |
|fan | 0 |4 |0.6032 |-0.0129 |0.5251 |-0.0075 |
|fan | 0 |6 |0.8765 |+0.1373 |0.6529 |+0.1294 |
|pump | 1 |0 |0.6639 |-0.0076 |0.5502 |-0.0172 |
|pump | 1 |2 |0.6050 |-0.0103 |0.5951 |+0.0141 |
|pump | 1 |4 |0.9793 |+0.0960 |0.8989 |+0.2279 |
|pump | 1 |6 |0.7271 |-0.0184 |0.5872 |+0.0070 |
|slider | 2 |0 |0.9650 |+0.0031 |0.8303 |+0.0159 |
|slider | 2 |2 |0.7927 |+0.0030 |0.6091 |-0.0277 |
|slider | 2 |4 |0.9501 |+0.0071 |0.7543 |+0.0345 |
|slider | 2 |6 |0.6996 |+0.0037 |0.4914 |+0.0012 |
|ToyCar | 3 |1 |0.8005 |-0.0131 |0.6986 |+0.0146 |
|ToyCar | 3 |2 |0.8809 |+0.0212 |0.7811 |+0.0039 |
|ToyCar | 3 |3 |0.6908 |+0.0578 |0.5733 |+0.0212 |
|ToyCar | 3 |4 |0.8864 |+0.0419 |0.7553 |+0.0656 |
|ToyConveyor | 4 |1 |0.8009 |+0.0202 |0.6563 |+0.0138 |
|ToyConveyor | 4 |2 |0.6645 |+0.0229 |0.5615 |+0.0014 |
|ToyConveyor | 4 |3 |0.7354 |-0.0181 |0.5956 |-0.0147 |
|valve | 5 |0 |0.7220 |+0.0344 |0.5374 |+0.0204 |
|valve | 5 |2 |0.7115 |+0.0297 |0.5246 |+0.0063 |
|valve | 5 |4 |0.7423 |-0.0007 |0.5110 |-0.0087 |
|valve | 5 |6 |0.6381 |+0.0991 |0.5000 |+0.0157 |
## Spec Unnormalized, Raw Normalized
| Machine | Type | ID | AUC | to BL | AUC | to BL |
| ------- | :--- | :--- | ---------- | ---------- | ---------- | ---------- |
|fan | 0 |0 |0.5502 |+0.0061 |0.4981 |+0.0044 |
|fan | 0 |2 |0.7815 |+0.0475 |0.5850 |+0.0369 |
|fan | 0 |4 |0.6206 |+0.0045 |0.5256 |-0.0070 |
|fan | 0 |6 |0.9042 |+0.1650 |0.7297 |+0.2062 |
|pump | 1 |0 |0.6410 |-0.0305 |0.5322 |-0.0352 |
|pump | 1 |2 |0.5892 |-0.0261 |0.5519 |-0.0291 |
|pump | 1 |4 |0.9661 |+0.0828 |0.8389 |+0.1679 |
|pump | 1 |6 |0.7508 |+0.0053 |0.5970 |+0.0168 |
|slider | 2 |0 |0.9532 |-0.0087 |0.7729 |-0.0415 |
|slider | 2 |2 |0.7889 |-0.0008 |0.6081 |-0.0287 |
|slider | 2 |4 |0.9321 |-0.0109 |0.6851 |-0.0347 |
|slider | 2 |6 |0.6685 |-0.0274 |0.4902 |+0.0000 |
|ToyCar | 3 |1 |0.7922 |-0.0214 |0.6608 |-0.0232 |
|ToyCar | 3 |2 |0.8450 |-0.0147 |0.7551 |-0.0221 |
|ToyCar | 3 |3 |0.6459 |+0.0129 |0.5540 |+0.0019 |
|ToyCar | 3 |4 |0.8967 |+0.0522 |0.7323 |+0.0426 |
|ToyConveyor | 4 |1 |0.7601 |-0.0206 |0.6253 |-0.0172 |
|ToyConveyor | 4 |2 |0.6194 |-0.0222 |0.5387 |-0.0214 |
|ToyConveyor | 4 |3 |0.6619 |-0.0916 |0.5529 |-0.0574 |
|valve | 5 |0 |0.8215 |+0.1339 |0.5546 |+0.0376 |
|valve | 5 |2 |0.6482 |-0.0335 |0.5197 |+0.0014 |
|valve | 5 |4 |0.7232 |-0.0198 |0.5184 |-0.0013 |
|valve | 5 |6 |0.6263 |+0.0873 |0.4912 |+0.0069 |
# Classification
## raw normalized, spec normalized all, complement same_mic_diff_type
| Machine | Type | ID | AUC | to BL | AUC | to BL |
| ------- | :--- | :--- | ---------- | ---------- | ---------- | ---------- |
|fan | 0 |0 |0.6861 |+0.1420 |0.6139 |+0.1202 |
|fan | 0 |2 |0.9867 |+0.2527 |0.9370 |+0.3889 |
|fan | 0 |4 |0.8060 |+0.1899 |0.6676 |+0.1350 |
|fan | 0 |6 |0.7448 |+0.0056 |0.8455 |+0.3220 |
|pump | 1 |0 |0.8480 |+0.1765 |0.7030 |+0.1356 |
|pump | 1 |2 |0.5552 |-0.0601 |0.5586 |-0.0224 |
|pump | 1 |4 |0.9995 |+0.1162 |0.9974 |+0.3264 |
|pump | 1 |6 |0.9117 |+0.1662 |0.7632 |+0.1830 |
|slider | 2 |0 |0.5270 |-0.4349 |0.6415 |-0.1729 |
|slider | 2 |2 |0.7433 |-0.0464 |0.5346 |-0.1022 |
|slider | 2 |4 |0.9879 |+0.0449 |0.9364 |+0.2166 |
|slider | 2 |6 |0.8589 |+0.1630 |0.5707 |+0.0805 |
|ToyCar | 3 |1 |0.4801 |-0.3335 |0.4940 |-0.1900 |
|ToyCar | 3 |2 |0.4581 |-0.4016 |0.4854 |-0.2918 |
|ToyCar | 3 |3 |0.4100 |-0.2230 |0.4870 |-0.0651 |
|ToyCar | 3 |4 |0.3659 |-0.4786 |0.4772 |-0.2125 |
|ToyConveyor | 4 |1 |0.7495 |-0.0312 |0.6487 |+0.0062 |
|ToyConveyor | 4 |2 |0.5607 |-0.0809 |0.5252 |-0.0349 |
|ToyConveyor | 4 |3 |0.6587 |-0.0948 |0.5552 |-0.0551 |
|valve | 5 |0 |0.9791 |+0.2915 |0.9390 |+0.4220 |
|valve | 5 |2 |0.3698 |-0.3120 |0.4895 |-0.0288 |
|valve | 5 |4 |0.6561 |-0.0869 |0.5175 |-0.0022 |
|valve | 5 |6 |0.7242 |+0.1852 |0.5417 |+0.0574 |
## raw normalized, spec normalized per_mic, complement all
| Machine | Type | ID | AUC | to BL | AUC | to BL |
| ------- | :--- | :--- | ---------- | ---------- | ---------- | ---------- |
|fan | 0 |0 |0.6683 |+0.1242 |0.6060 |+0.1123 |
|fan | 0 |2 |0.9883 |+0.2543 |0.9456 |+0.3975 |
|fan | 0 |4 |0.7859 |+0.1698 |0.6636 |+0.1310 |
|fan | 0 |6 |0.7462 |+0.0070 |0.8455 |+0.3220 |
|pump | 1 |0 |0.8785 |+0.2070 |0.7284 |+0.1610 |
|pump | 1 |2 |0.5635 |-0.0518 |0.5462 |-0.0348 |
|pump | 1 |4 |0.9995 |+0.1162 |0.9974 |+0.3264 |
|pump | 1 |6 |0.9560 |+0.2105 |0.8653 |+0.2851 |
|slider | 2 |0 |0.9679 |+0.0060 |0.8743 |+0.0599 |
|slider | 2 |2 |0.8813 |+0.0916 |0.6339 |-0.0029 |
|slider | 2 |4 |0.9905 |+0.0475 |0.9500 |+0.2302 |
|slider | 2 |6 |0.8336 |+0.1377 |0.5577 |+0.0675 |
|ToyCar | 3 |1 |0.6665 |-0.1471 |0.6647 |-0.0193 |
|ToyCar | 3 |2 |0.9024 |+0.0427 |0.8195 |+0.0423 |
|ToyCar | 3 |3 |0.9814 |+0.3484 |0.9271 |+0.3750 |
|ToyCar | 3 |4 |0.9982 |+0.1537 |0.9904 |+0.3007 |
|ToyConveyor | 4 |1 |0.8454 |+0.0647 |0.7466 |+0.1041 |
|ToyConveyor | 4 |2 |0.6047 |-0.0369 |0.5448 |-0.0153 |
|ToyConveyor | 4 |3 |0.7071 |-0.0464 |0.5888 |-0.0215 |
|valve | 5 |0 |0.9876 |+0.3000 |0.9518 |+0.4348 |
|valve | 5 |2 |0.7835 |+0.1017 |0.5197 |+0.0014 |
|valve | 5 |4 |0.7010 |-0.0420 |0.5289 |+0.0092 |
|valve | 5 |6 |0.8042 |+0.2652 |0.5425 |+0.0582 |
## raw normalized, spec unnormalized, complement all
| Machine | Type | ID | AUC | to BL | AUC | to BL |
| ------- | :--- | :--- | ---------- | ---------- | ---------- | ---------- |
|fan | 0 |0 |0.5857 |+0.0416 |0.5846 |+0.0909 |
|fan | 0 |2 |0.9865 |+0.2525 |0.9315 |+0.3834 |
|fan | 0 |4 |0.7753 |+0.1592 |0.6568 |+0.1242 |
|fan | 0 |6 |0.7520 |+0.0128 |0.8455 |+0.3220 |
|pump | 1 |0 |0.8373 |+0.1658 |0.7126 |+0.1452 |
|pump | 1 |2 |0.5577 |-0.0576 |0.5225 |-0.0585 |
|pump | 1 |4 |0.9995 |+0.1162 |0.9974 |+0.3264 |
|pump | 1 |6 |0.9311 |+0.1856 |0.8354 |+0.2552 |
|slider | 2 |0 |0.9946 |+0.0327 |0.9821 |+0.1677 |
|slider | 2 |2 |0.7940 |+0.0043 |0.5494 |-0.0874 |
|slider | 2 |4 |0.9953 |+0.0523 |0.9752 |+0.2554 |
|slider | 2 |6 |0.8202 |+0.1243 |0.5795 |+0.0893 |
|ToyCar | 3 |1 |0.6782 |-0.1354 |0.6485 |-0.0355 |
|ToyCar | 3 |2 |0.8749 |+0.0152 |0.7847 |+0.0075 |
|ToyCar | 3 |3 |0.9746 |+0.3416 |0.9001 |+0.3480 |
|ToyCar | 3 |4 |0.9975 |+0.1530 |0.9867 |+0.2970 |
|ToyConveyor | 4 |1 |0.8521 |+0.0714 |0.7509 |+0.1084 |
|ToyConveyor | 4 |2 |0.5919 |-0.0497 |0.5407 |-0.0194 |
|ToyConveyor | 4 |3 |0.7277 |-0.0258 |0.6033 |-0.0070 |
|valve | 5 |0 |0.9838 |+0.2962 |0.9288 |+0.4118 |
|valve | 5 |2 |0.7559 |+0.0741 |0.5048 |-0.0135 |
|valve | 5 |4 |0.7259 |-0.0171 |0.5373 |+0.0176 |
|valve | 5 |6 |0.7828 |+0.2438 |0.5623 |+0.0780 |
## raw unnormalized, spec unnormalized all, complement all
| Machine | Type | ID | AUC | to BL | AUC | to BL |
| ------- | :--- | :--- | ---------- | ---------- | ---------- | ---------- |
|fan | 0 |0 |0.4166 |-0.1275 |0.5182 |+0.0245 |
|fan | 0 |2 |0.9741 |+0.2401 |0.8962 |+0.3481 |
|fan | 0 |4 |0.6807 |+0.0646 |0.6375 |+0.1049 |
|fan | 0 |6 |0.7403 |+0.0011 |0.8455 |+0.3220 |
|pump | 1 |0 |0.7201 |+0.0486 |0.6802 |+0.1128 |
|pump | 1 |2 |0.5097 |-0.1056 |0.5126 |-0.0684 |
|pump | 1 |4 |0.9999 |+0.1166 |0.9995 |+0.3285 |
|pump | 1 |6 |0.8119 |+0.0664 |0.7198 |+0.1396 |
|slider | 2 |0 |0.9993 |+0.0374 |0.9963 |+0.1819 |
|slider | 2 |2 |0.7545 |-0.0352 |0.5470 |-0.0898 |
|slider | 2 |4 |0.9926 |+0.0496 |0.9826 |+0.2628 |
|slider | 2 |6 |0.8804 |+0.1845 |0.6594 |+0.1692 |
|ToyCar | 3 |1 |0.6403 |-0.1733 |0.6499 |-0.0341 |
|ToyCar | 3 |2 |0.7595 |-0.1002 |0.6391 |-0.1381 |
|ToyCar | 3 |3 |0.9774 |+0.3444 |0.9087 |+0.3566 |
|ToyCar | 3 |4 |0.9914 |+0.1469 |0.9558 |+0.2661 |
|ToyConveyor | 4 |1 |0.8422 |+0.0615 |0.7263 |+0.0838 |
|ToyConveyor | 4 |2 |0.5798 |-0.0618 |0.5287 |-0.0314 |
|ToyConveyor | 4 |3 |0.7005 |-0.0530 |0.5779 |-0.0324 |
|valve | 5 |0 |0.7336 |+0.0460 |0.5387 |+0.0217 |
|valve | 5 |2 |0.7018 |+0.0200 |0.5338 |+0.0155 |
|valve | 5 |4 |0.6526 |-0.0904 |0.5092 |-0.0105 |
|valve | 5 |6 |0.8994 |+0.3604 |0.7158 |+0.2315 |
## raw normalized, spec normalized per mic, complement same mic
23 experiments loaded
| Machine | Type | ID | AUC | to BL | AUC | to BL |
| ------- | :--- | :--- | ---------- | ---------- | ---------- | ---------- |
|fan | 0 |0 |0.6703 |+0.1262 |0.5874 |+0.0937 |
|fan | 0 |2 |0.9831 |+0.2491 |0.9264 |+0.3783 |
|fan | 0 |4 |0.7925 |+0.1764 |0.6645 |+0.1319 |
|fan | 0 |6 |0.7467 |+0.0075 |0.8455 |+0.3220 |
|pump | 1 |0 |0.8555 |+0.1840 |0.7022 |+0.1348 |
|pump | 1 |2 |0.5559 |-0.0594 |0.5386 |-0.0424 |
|pump | 1 |4 |0.9995 |+0.1162 |0.9974 |+0.3264 |
|pump | 1 |6 |0.9425 |+0.1970 |0.8369 |+0.2567 |
|slider | 2 |0 |0.9773 |+0.0154 |0.9117 |+0.0973 |
|slider | 2 |2 |0.8856 |+0.0959 |0.6542 |+0.0174 |
|slider | 2 |4 |0.9857 |+0.0427 |0.9249 |+0.2051 |
|slider | 2 |6 |0.8374 |+0.1415 |0.5571 |+0.0669 |
|ToyCar | 3 |1 |0.6835 |-0.1301 |0.6569 |-0.0271 |
|ToyCar | 3 |2 |0.8717 |+0.0120 |0.7941 |+0.0169 |
|ToyCar | 3 |3 |0.9855 |+0.3525 |0.9338 |+0.3817 |
|ToyCar | 3 |4 |0.9983 |+0.1538 |0.9913 |+0.3016 |
|ToyConveyor | 4 |1 |0.8592 |+0.0785 |0.7649 |+0.1224 |
|ToyConveyor | 4 |2 |0.5898 |-0.0518 |0.5401 |-0.0200 |
|ToyConveyor | 4 |3 |0.7115 |-0.0420 |0.5929 |-0.0174 |
|valve | 5 |0 |0.9913 |+0.3037 |0.9655 |+0.4485 |
|valve | 5 |2 |0.7976 |+0.1158 |0.5561 |+0.0378 |
|valve | 5 |4 |0.7020 |-0.0410 |0.5373 |+0.0176 |
|valve | 5 |6 |0.7919 |+0.2529 |0.5500 |+0.0657 |
\ No newline at end of file
This diff is collapsed.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment