[docs] update baseline and add new results on cdbd-lsat

Author: tswsxk

docs/irt_tutorial.ipynb

@@ -287,36 +287,35 @@
  "source": [
  "### Performance compared among different implementation\n",
  "\n",
+ "We conduct performance comparison experiments across different implementations of IRT.\n",
+ "The implementations we chose are those high-stared in github\n",
+ "and easy to be applied in new datasets with clear tutorials.\n",
  "[pyirt](https://github.com/17zuoye/pyirt),\n",
- "[bayesian-irt](https://github.com/jamalex/bayesian-irt),\n",
- "[irt_parameter_estimation](https://github.com/pluralsight/irt_parameter_estimation),\n",
- "[pypsy](https://github.com/inuyasha2012/pypsy),\n",
  "[vibo](https://github.com/mhw32/variational-item-response-theory-public)\n",
- "and [irt](https://github.com/aimir/irt)\n",
  "are high-stared implementation of IRT in github.\n",
  "We run them on the cdbd dataset and report the result (AUC and MAP is reported) in the following tables,\n",
  "which also includes our solution:\n",
  "\n",
  "#### AUC (auroc)\n",
  "\n",
- "| | cdbd-a0910 |\n",
- "| ------------------------ | ---------- |\n",
- "| pyirt | 0.501737 |\n",
- "| bayesian-irt | |\n",
- "| irt_parameter_estimation | |\n",
- "| irt | |\n",
- "| XCD-IRT-MXIRT | 0.703092 |\n",
- "| XCD-IRT-MXIRT-with-tricks | 0.739607 |\n",
+ "| | cdbd-a0910 | cdbd-lsat |\n",
+ "| ------------------------ | ---------- | ---------- |\n",
+ "| pyirt | 0.501737 | |\n",
+ "| bayesian-irt | | |\n",
+ "| irt_parameter_estimation | | |\n",
+ "| irt | | |\n",
+ "| XCD-IRT-MXIRT | 0.703092 | |\n",
+ "| XCD-IRT-MXIRT-with-tricks | 0.739607 | 0.673990 |\n",
  "\n",
  "#### MAP(aupoc)\n",
- "| | cdbd-a0910 |\n",
- "| ------------------------ | ---------- |\n",
- "| pyirt | 0.674690 |\n",
- "| bayesian-irt | |\n",
- "| irt_parameter_estimation | |\n",
- "| irt | |\n",
- "| XCD-IRT-MXIRT | 0.788603 |\n",
- "| XCD-IRT-MXIRT-with-tricks | 0.826107 |\n",
+ "| | cdbd-a0910 | cdbd-lsat |\n",
+ "| ------------------------ | ---------- | ---------- |\n",
+ "| pyirt | 0.674690 | |\n",
+ "| bayesian-irt | | |\n",
+ "| irt_parameter_estimation | | |\n",
+ "| irt | | |\n",
+ "| XCD-IRT-MXIRT | 0.788603 | |\n",
+ "| XCD-IRT-MXIRT-with-tricks | 0.826107 | 0.868745 |\n",
  "\n",
  "The scripts we used to test the performance of the baseline implementation could be found in:\n",
  "* [pyirt](../scripts/reproduction/pyirt)\n",