Skip to content

GitLab

Commit cdde06a0 authored by Paul Primus's avatar Paul Primus
Browse files

add micro seconds to experiment name

parent 8c76ce21
Hide whitespace changes
Inline Side-by-side
README.md
View file @ cdde06a0
......@@ -17,6 +17,7 @@
## Spec Normalized, Raw Normalized
| Machine | Type | ID | AUC | to BL | AUC | to BL |
| ------- | :--- | :--- | ---------- | ---------- | ---------- | ---------- |
|fan | 0 |0 |0.5639 |+0.0198 |0.4959 |+0.0022 |
......@@ -43,6 +44,7 @@
|valve | 5 |4 |0.7339 |-0.0091 |0.5263 |+0.0066 |
|valve | 5 |6 |0.5888 |+0.0498 |0.4947 |+0.0104 |
## Spec Unnormalized, Raw Unnormalized
| Machine | Type | ID | AUC | to BL | AUC | to BL |
......@@ -72,8 +74,8 @@
|valve | 5 |6 |0.4584 |-0.0806 |0.4961 |+0.0118 |
## Spec Unnormalized, Raw Normalized
## Spec Normalized, Raw Unnormalized
| Machine | Type | ID | AUC | to BL | AUC | to BL |
| ------- | :--- | :--- | ---------- | ---------- | ---------- | ---------- |
......@@ -102,7 +104,36 @@
|valve | 5 |6 |0.6381 |+0.0991 |0.5000 |+0.0157 |
## Spec Unnormalized, Raw Normalized
| Machine | Type | ID | AUC | to BL | AUC | to BL |
| ------- | :--- | :--- | ---------- | ---------- | ---------- | ---------- |
|fan | 0 |0 |0.5502 |+0.0061 |0.4981 |+0.0044 |
|fan | 0 |2 |0.7815 |+0.0475 |0.5850 |+0.0369 |
|fan | 0 |4 |0.6206 |+0.0045 |0.5256 |-0.0070 |
|fan | 0 |6 |0.9042 |+0.1650 |0.7297 |+0.2062 |
|pump | 1 |0 |0.6410 |-0.0305 |0.5322 |-0.0352 |
|pump | 1 |2 |0.5892 |-0.0261 |0.5519 |-0.0291 |
|pump | 1 |4 |0.9661 |+0.0828 |0.8389 |+0.1679 |
|pump | 1 |6 |0.7508 |+0.0053 |0.5970 |+0.0168 |
|slider | 2 |0 |0.9532 |-0.0087 |0.7729 |-0.0415 |
|slider | 2 |2 |0.7889 |-0.0008 |0.6081 |-0.0287 |
|slider | 2 |4 |0.9321 |-0.0109 |0.6851 |-0.0347 |
|slider | 2 |6 |0.6685 |-0.0274 |0.4902 |+0.0000 |
|ToyCar | 3 |1 |0.7922 |-0.0214 |0.6608 |-0.0232 |
|ToyCar | 3 |2 |0.8450 |-0.0147 |0.7551 |-0.0221 |
|ToyCar | 3 |3 |0.6459 |+0.0129 |0.5540 |+0.0019 |
|ToyCar | 3 |4 |0.8967 |+0.0522 |0.7323 |+0.0426 |
|ToyConveyor | 4 |1 |0.7601 |-0.0206 |0.6253 |-0.0172 |
|ToyConveyor | 4 |2 |0.6194 |-0.0222 |0.5387 |-0.0214 |
|ToyConveyor | 4 |3 |0.6619 |-0.0916 |0.5529 |-0.0574 |
|valve | 5 |0 |0.8215 |+0.1339 |0.5546 |+0.0376 |
|valve | 5 |2 |0.6482 |-0.0335 |0.5197 |+0.0014 |
|valve | 5 |4 |0.7232 |-0.0198 |0.5184 |-0.0013 |
|valve | 5 |6 |0.6263 |+0.0873 |0.4912 |+0.0069 |
# Classification
## raw normalized, spec normalized all, complement same_mic_diff_type
| Machine | Type | ID | AUC | to BL | AUC | to BL |
......@@ -130,3 +161,6 @@
|valve | 5 |2 |0.3698 |-0.3120 |0.4895 |-0.0288 |
|valve | 5 |4 |0.6561 |-0.0869 |0.5175 |-0.0022 |
|valve | 5 |6 |0.7242 |+0.1852 |0.5417 |+0.0574 |
## Spec normalized per_mic, Raw normalized
scripts/load_from_mongo.ipynb
View file @ cdde06a0
%% Cell type:code id: tags:
``` python
import dcase2020_task2
from pymongo import MongoClient
import numpy as np
INVERSE_CLASS_MAP = {
0: 'fan',
1: 'pump',
2: 'slider',
3: 'ToyCar',
4: 'ToyConveyor',
5: 'valve'
}
baseline_auc = {
0: {0: 0.5441, 2: 0.7340, 4: 0.6161, 6: 0.7392},
1: {0: 0.6715, 2: 0.6153, 4: 0.8833, 6: 0.7455},
2: {0: 0.9619, 2: 0.7897, 4: 0.9430, 6: 0.6959},
3: {1: 0.8136, 2: 0.8597, 3: 0.6330, 4: 0.8445},
4: {1: 0.7807, 2: 0.6416, 3: 0.7535},
5: {0: 0.6876, 2: 0.6818, 4: 0.7430, 6: 0.5390}
}
baseline_pauc = {
0: {0: 0.4937, 2: 0.5481, 4: 0.5326, 6: 0.5235},
1: {0: 0.5674, 2: 0.5810, 4: 0.6710, 6: 0.5802},
2: {0: 0.8144, 2: 0.6368, 4: 0.7198, 6: 0.4902},
3: {1: 0.6840, 2: 0.7772, 3: 0.5521, 4: 0.6897},
4: {1: 0.6425, 2: 0.5601, 3: 0.6103},
5: {0: 0.5170, 2: 0.5183, 4: 0.5197, 6: 0.4843}
}
```
%% Cell type:code id: tags:
``` python
client = MongoClient('mongodb://student2.cp.jku.at:27017/')
db = client.dcase2020_task2
db.list_collection_names()
```
%%%% Output: execute_result
['fs.chunks',
'omniboard.settings',
'omniboard.metric.columns',
'omniboard.custom.columns',
'fs.files',
'runs',
'metrics']
%% Cell type:code id: tags:
``` python
experiments = []
for experiment in db.runs.find(
{
"experiment.name": "dcase2020_task2_classification"
"experiment.name": "dcase2020_task2_baseline"
}
):
if (experiment['config'].get('normalize', True) is True) and (experiment['config'].get('normalize_raw', True) is True):
if (experiment['config'].get('normalize', True)) and (experiment['config'].get('normalize_raw', True)):
experiments.append([
experiment['config']['machine_type'],
experiment['config']['machine_id'],
experiment['result']['auroc_mean'],
experiment['result']['pauroc_mean'],
])
print("{} experiments loaded".format(len(experiments)))
assert len(experiments) == 23
```
%%%% Output: stream
23 experiments loaded
%% Cell type:code id: tags:
``` python
print("| Machine | Type | ID | AUC | to BL | AUC | to BL |")
print("| ------- | :--- | :--- | ---------- | ---------- | ---------- | ---------- |")
for e in experiments:
print("|{}\t| {}\t|{}\t|{:0.4f}\t|{:+0.4f}\t|{:0.4f}\t|{:+1.4f}\t|".format(
INVERSE_CLASS_MAP[e[0]],
e[0],
e[1],
e[2],
e[2] - baseline_auc[e[0]][e[1]],
e[3],
e[3] - baseline_pauc[e[0]][e[1]]
))
```
%%%% Output: stream
| Machine | Type | ID | AUC | to BL | AUC | to BL |
| ------- | :--- | :--- | ---------- | ---------- | ---------- | ---------- |
|fan | 0 |0 |0.6861 |+0.1420 |0.6139 |+0.1202 |
|fan | 0 |2 |0.9867 |+0.2527 |0.9370 |+0.3889 |
|fan | 0 |4 |0.8060 |+0.1899 |0.6676 |+0.1350 |
|fan | 0 |6 |0.7448 |+0.0056 |0.8455 |+0.3220 |
|pump | 1 |0 |0.8480 |+0.1765 |0.7030 |+0.1356 |
|pump | 1 |2 |0.5552 |-0.0601 |0.5586 |-0.0224 |
|pump | 1 |4 |0.9995 |+0.1162 |0.9974 |+0.3264 |
|pump | 1 |6 |0.9117 |+0.1662 |0.7632 |+0.1830 |
|slider | 2 |0 |0.5270 |-0.4349 |0.6415 |-0.1729 |
|slider | 2 |2 |0.7433 |-0.0464 |0.5346 |-0.1022 |
|slider | 2 |4 |0.9879 |+0.0449 |0.9364 |+0.2166 |
|slider | 2 |6 |0.8589 |+0.1630 |0.5707 |+0.0805 |
|ToyCar | 3 |1 |0.4801 |-0.3335 |0.4940 |-0.1900 |
|ToyCar | 3 |2 |0.4581 |-0.4016 |0.4854 |-0.2918 |
|ToyCar | 3 |3 |0.4100 |-0.2230 |0.4870 |-0.0651 |
|ToyCar | 3 |4 |0.3659 |-0.4786 |0.4772 |-0.2125 |
|ToyConveyor | 4 |1 |0.7495 |-0.0312 |0.6487 |+0.0062 |
|ToyConveyor | 4 |2 |0.5607 |-0.0809 |0.5252 |-0.0349 |
|ToyConveyor | 4 |3 |0.6587 |-0.0948 |0.5552 |-0.0551 |
|valve | 5 |0 |0.9791 |+0.2915 |0.9390 |+0.4220 |
|valve | 5 |2 |0.3698 |-0.3120 |0.4895 |-0.0288 |
|valve | 5 |4 |0.6561 |-0.0869 |0.5175 |-0.0022 |
|valve | 5 |6 |0.7242 |+0.1852 |0.5417 |+0.0574 |
|fan | 0 |0 |0.5639 |+0.0198 |0.4959 |+0.0022 |
|fan | 0 |2 |0.8054 |+0.0714 |0.6094 |+0.0613 |
|fan | 0 |4 |0.6661 |+0.0500 |0.5413 |+0.0087 |
|fan | 0 |6 |0.9022 |+0.1630 |0.7100 |+0.1865 |
|pump | 1 |0 |0.6967 |+0.0252 |0.5396 |-0.0278 |
|pump | 1 |2 |0.6124 |-0.0029 |0.5771 |-0.0039 |
|pump | 1 |4 |0.9497 |+0.0664 |0.7932 |+0.1222 |
|pump | 1 |6 |0.8019 |+0.0564 |0.6068 |+0.0266 |
|slider | 2 |0 |0.9342 |-0.0277 |0.6991 |-0.1153 |
|slider | 2 |2 |0.7753 |-0.0144 |0.6069 |-0.0299 |
|slider | 2 |4 |0.9044 |-0.0386 |0.6257 |-0.0941 |
|slider | 2 |6 |0.6628 |-0.0331 |0.4979 |+0.0077 |
|ToyCar | 3 |1 |0.7976 |-0.0160 |0.6980 |+0.0140 |
|ToyCar | 3 |2 |0.8678 |+0.0081 |0.7776 |+0.0004 |
|ToyCar | 3 |3 |0.6652 |+0.0322 |0.5633 |+0.0112 |
|ToyCar | 3 |4 |0.8859 |+0.0414 |0.7442 |+0.0545 |
|ToyConveyor | 4 |1 |0.7552 |-0.0255 |0.6257 |-0.0168 |
|ToyConveyor | 4 |2 |0.6276 |-0.0140 |0.5590 |-0.0011 |
|ToyConveyor | 4 |3 |0.7387 |-0.0148 |0.5950 |-0.0153 |
|valve | 5 |0 |0.6751 |-0.0125 |0.5179 |+0.0009 |
|valve | 5 |2 |0.6286 |-0.0532 |0.5105 |-0.0078 |
|valve | 5 |4 |0.7339 |-0.0091 |0.5263 |+0.0066 |
|valve | 5 |6 |0.5888 |+0.0498 |0.4947 |+0.0104 |
%% Cell type:code id: tags:
``` python
```
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

CP WIKI