Technology

The research carried out at PAN's shared tasks informs the development of new digital text forensics technology. For reproducibility sake, the prototypes submitted for evaluation are made available by participants open source, as executables on TIRA, or both. The choice of license is at the discretion of participatns, who retain copyright of their software.

Register now Next: Publications 0 already signed up

Code

PAN at GitHub

PAN maintains a code repository for the digital text forensics at GitHub at github.com/pan-webis-de. Since many participants of PAN's shared tasks have expressed interest to share their code with the digital text forensics community, our repository provides for a central place to do so.

How to get access?

Viewing PAN's repository is simple; just click the button below and you can browse the various code repositories directly. To download a given code repository, you will need to log in with your GitHub account, or sign up for a new one. After that, simply follow the checkout instructions provided by GitHub.

View code Checkout instructions

Why share at all?

Many researchers do not share their code cause it's not polished, they have no time to provide support, or for fear of competition. This may be true, yet, researchers who share their code may have a lasting impact on their community. Others who adopt their code will cite it, ultimately adding to the sharer's reputation. Sharing is thinking long-term.

Learn more

How to share my code?

To get started, send us an email with the name of your GitHub account. We will create a team for you at PAN's GitHub page. You may add as many team members to your team as you like, or be part of multiple teams. After that, you can create repositories within PAN's organization and commit to them.

Share now Commit instructions

What are the terms?

  • Authors retain copyright of all their code. PAN assumes no ownership.
  • The choice of license rests with the authors. Respect the license.
  • When you use some code, cite the authors. If you don't know how, ask.
  • When you find bugs, fix them and share bugfixes with the authors.

Evaluation as a Service

PAN at TIRA

At PAN, it is our goal to carry out reproducible research. While the evaluation resources created for our shared tasks benefit future evaluations, even shared tasks lack reproducibility. Therefore, we ask participants to submit working executables of their software. To facilitate the submission of software, we employ TIRA (tira.io), a platform that implements the evaluation as a service paradigm.

How to get access?

To get started, send us an email with your preferred choice of operating system. We offer Ubuntu Linux 14 LTS (Desktop or Server) as well as Windows 7. We will set up a virtual machine (VM) for you and pass you the access credentials. You can access your VM via SSH and remote desktop. Once you have access, please install your software inside the virtual machine.

Request virtual machine

Why submit software?

Reproducibility is one of the cornerstones of science; yet, in recent years it was shown that many scientific contributions are irreproducible. For shared tasks in particular, the software developed by participants often go missing soon after the event has passed. Moreover, the test datasets used in shared tasks are released to participants, creating an evaluation bias. All of these problems are solved by submitting software instead of their run output.

Learn more

What to prepare?

To work with TIRA, your software has to be executable from the command line and it must accept the following parameters:

  • A path to an input dataset.
  • A path to an output directory.
  • A path to the output of a previous run of the software (optional).
Once your software is installed in your VM, you can remote control its evaluation via TIRA's web interface.

Learn more

What are the terms?

  • Authors retain copyright of their software.
  • Authors provide usage rights to PAN and TIRA for the purpose of evaluation.
  • Authors may revoke usage rights at any time.
  • Usage of TIRA is at the author's risk.