This repository has been archived by the owner on Feb 3, 2023. It is now read-only.

Evaluate MLP

Jump to bottom

James Thorne edited this page Jul 5, 2018 · 1 revision

Oracle Evaluation (no evidence retrieval)

PYTHONPATH=src python src/scripts/rte/mlp/eval_mlp.py data/fever/fever.db --model ns_nn_sent --sentence true --log logs/mlp_nn_sent

Evidence Retrieval Evluation

First run the evidence retrieval script as per the readme

PYTHONPATH=src python src/scripts/rte/mlp/eval_mlp.py data/fever/fever.db data/fever/dev.sentences.p5.s5.jsonl --model ns_nn_sent --sentence true --log logs/mlp_nn_sent_dev
PYTHONPATH=src python src/scripts/rte/mlp/eval_mlp.py data/fever/fever.db data/fever/test.sentences.p5.s5.jsonl --model ns_nn_sent --sentence true --log logs/mlp_nn_sent_test