[CONLL04] Train config file / YAML

[SylvainVerdy]

Hi urchade,

Thanks a lot for this great work. 😃
I'm trying to reproduce your results on CONLL04. I have lowest results than the paper.
I would like to know if you can give all the parameters to reproduce the results on it.

Best regards,

Sylvain

[urchade]

Hi @SylvainVerdy , thank you for your interest in our work. Here is the hyperparamters for conll04:

Hyperparameters:
data_file: dataset/conll04.pkl
model_name: deberta # use deberta v3 large
max_width: 14
num_prompts: 1
hidden_transformer: 512
num_transformer_layers: 5
attention_heads: 8
p_drop: 0.4
span_mode: marker
use_pos_code: True
n_epochs: 1
n_steps: 70000
batch_size: 16
eval_batch_size: 32
lr_encoder: 1e-05
lr_decoder: 5e-5
lr_others: 1e-4
warmup_ratio: 0.05
grad_accumulation_steps: 1
save_interval: 1000
max_num_samples: 5
log_dir: conll04_deberta_1
cross_attn: True

[urchade]

I did not have much time to polish the code, so do not hesitate if you have any other question