How to only generate Precision, Recall, and f1 score when benchmarking BLINK

[clintoncheang]

Hello,
I am wondering are there any ways to only display Precision, Recall, and f1 scores when benchmarking BLINK model? I looked into the code but seems like it doesn't have it.

[ledw-2]

Hi @clintoncheang, we did not compute f1 so you would need to implement that yourself. For other metrics, look at code here https://github.com/facebookresearch/BLINK/blob/main/blink/main_dense.py#L476 (and in similar places for cross-encoder).

[clintoncheang]

Hi @clintoncheang, we did not compute f1 so you would need to implement that yourself. For other metrics, look at code here https://github.com/facebookresearch/BLINK/blob/main/blink/main_dense.py#L476 (and in similar places for cross-encoder).

Thank you so much for the reply. I looked at the code but I am not very sure how BLINK computes the recall and accuracy. Is it based on the output from the BLINK model (Predictions for the entity, which is in string format) and compare it to the correct entities from the corresponding Test dataset (I am assuming the correct entity from the jsonl file will be the wiki_title field?) Correct me if I am wrong. Thank you so much!

[abhinavkulkarni]

@clintoncheang: Have you looked at eval_biencoder.py script?