rchallenge

GitHub sources GitHub release Travis-CI Build Status GPLv2 License

The rchallenge R package provides a simple datascience competition system using R Markdown and Dropbox with the following features:

  • No network configuration required.
  • Does not depend on external platforms like e.g. Kaggle.
  • Can be easily installed on a personal computer.
  • Provides a customizable template in english and french.

Further documentation is available in the Reference manual.

Please report bugs, troubles or discussions on the Issues tracker. Any contribution to improve the package is welcome.

Installation

Install the latest version of the R package from GitHub

# install.packages("devtools")
devtools::install_github("adrtod/rchallenge")

Getting started

Install a new challenge in Dropbox/mychallenge

setwd("~/Dropbox/mychallenge")
library(rchallenge)
?new_challenge
new_challenge()

or for a french version

new_challenge(template = "fr")

You will obtain a ready-to-use challenge in the folder Dropbox/mychallenge containing:

Name Description
challenge.rmd Template R Markdown script for the webpage.
data Directory of the data containing data_train and data_test datasets.
submissions Directory of the submissions. It will contain one subdirectory per team where they can submit their submissions. The subdirectories are shared with Dropbox.
history Directory where the submissions history is stored.

The default challenge provided is a binary classification problem on the German Credit Card) dataset.

You can easily customize the challenge in two ways:

  • During the creation of the challenge: by using the options of the new_challenge function.
  • After the creation of the challenge: by manually replacing the data files in the data subdirectory and the baseline predictions in submissions/baseline and by customizing the template challenge.rmd as needed.

Next steps

To complete the installation:

  1. Create and share subdirectories in submissions for each team:

    ?new_team
    new_team("team_foo", "team_bar")
    
  2. Publish the html page in Dropbox/Public:

    ?publish
    publish()
    

    Prior to this, make sure you enabled your Public Dropbox folder.

  3. Give the public link to your Dropbox/Public/challenge.html file to the participants.

  4. Automate the updates of the webpage.

For the step 4, on Unix systems, you can setup the following line to your crontab using crontab -e (mind the quotes):

0 * * * * Rscript -e 'rchallenge::publish("~/Dropbox/mychallenge/challenge.rmd")'

This will publish a html webpage in your Dropbox/Public folder every hour.

On Windows systems, you can use the Task Scheduler to create a new task with a Start a program action with the settings (mind the quotes):

  • Program/script: Rscript.exe
  • options: -e rchallenge::publish('~/Dropbox/mychallenge/challenge.rmd')

From now on, a fully autonomous challenge system is set up requiring no further administration. With each update, the program automatically performs the following tasks using the functions available in our package:

Name Description
store_new_submissions Reads submitted files and save new files in the history.
print_readerr Displays any read errors.
compute_metrics Calculates the scores for each submission in the history.
get_best Gets the highest score per team.
print_leaderboard Displays the leaderboard.
plot_history Plots a chart of score evolution per team.
plot_activity Plots a chart of activity per team.

Examples

Please contact me to add yours.

Copyright

Copyright © 2014-2015 Adrien Todeschini.

Contributions from Robin Genuer.

Design inspired by Datascience.net, a french platform for datascience challenges.

The rchallenge package is licensed under the GPLv2 (http://www.gnu.org/licenses/gpl-2.0.html).

To do list

  • [ ] common leaderboard for several metrics
  • [ ] do not take baseline into account in ranking
  • [ ] examples, tests, vignettes
  • [ ] test windows
  • [ ] submit to CRAN
  • [ ] interactive webpage using Shiny

Release notes

Version 1.1 (16-05-2015)

  • added out_rmdfile argument to new_challenge
  • changed template argument to c("en", "fr")
  • fixed bugs
  • added examples to doc

Version 1.0 (15-04-2015)

  • new name
  • changes in readme
  • new_team can create several teams
  • instructions for windows

Version 0.2 (05-03-2015)

  • exported new_team function
  • suppressed dependency to caret package
  • fixed change of directory in publish
  • improved messages

Version 0.1 (21-01-2015)

  • initial package release
  • easy installation
  • roxygen documentation
  • english and french templates

Help topics

Setup

  • new_challenge
    Install a new challenge.
  • new_team
    Create new teams submission folders in your challenge.
  • german
    German Credit Data.
  • data_split
    Split a data.frame into training and test sets.
  • publish
    Publish your challenge R Markdown script to a html page.

Update

Display

Plots

Other

Dependencies

  • Depends:
  • Imports: rmarkdown, knitr
  • Suggests:
  • Extends:

Authors