Error in reproduction of run_DA-ADB.sh results

[EdwardBurgin]

Is the example found in open_intent_detection/examples/run_DA-ADB.sh meant to reproduce the published results? Running this and comparing to ADB gives significantly worse results. I took a look to see why this is briefly but didn't find an obvious answer.

The average of ADB across the experiments gave 84% and DA-ADB gave 47%

Example of ADB results from which averages of Acc where calculated

The only change was the location of the bert model in hyper-parameters which was the same for both methods:
'bert_model': "bert-base-uncased",
Instead of: 'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",

Thanks for your help!

[JianhuaSu]

The following is my reproduction of the results. The DA-ADB result is generally higher than that of ADB, so there is nothing wrong with our model. The only difference is that the position of the bert model I use in the hyperparameter is
“uncased_L-12_H-768_A-12”.

Could it be a bert version issue? You can give it a try uncased_L-12_H-768_A-12

[CodingPerson]

我发现问题了，您在run_DA-ADB.sh中少加了--save_model这个参数，导致没法存储模型了

[TzuchengChang]

Have u solved the above problems?
I've tried the bert-base-uncased model and the given BaiduNetdisk model (could it be uncased_L-12_H-768_A-12?), but it gave the wrong results in DA-ADB, too (about 50% Acc), while it gave correct result in ADB (80+ Acc).
How can I rebuild it?

[HITLiuJiahao]

Have u solved the above problems? I've tried the bert-base-uncased model and the given BaiduNetdisk model (could it be uncased_L-12_H-768_A-12?), but it gave the wrong results in DA-ADB, too (about 50% Acc), while it gave correct result in ADB (80+ Acc). How can I rebuild it?

Have u solved the above problems?

[TzuchengChang]

Have u solved the above problems? I've tried the bert-base-uncased model and the given BaiduNetdisk model (could it be uncased_L-12_H-768_A-12?), but it gave the wrong results in DA-ADB, too (about 50% Acc), while it gave correct result in ADB (80+ Acc). How can I rebuild it?

Have u solved the above problems?

not yet