Note: Some users have reported problems compiling with the GNU Java Compiler (gcj). We recommend using Sun's Java SDK (java-6-sun on Ubuntu).
1. Unzip the software. After you have downloaded the software, unzip it. For instance, on a Linux system, open a Terminal, go into the directory in which you would like to unzip the software, and run the command:
tar -zxvf /PATH/TO/xxxx-rl-competition-2009.tar.gz
where xxxx is replaced with the current version of the software, and /PATH/TO is the directory in which you've downloaded the competition software. This should create a folder called xxxx-rl-competition-2009 in the current directory. This folder contains a more detailed README.txt file, but we will summarize some of that information here.
2. Run a Graphical Demo. To make sure that your system has the prerequisites for running the software, you can run the graphical demo program included with the competition software. In the rl-competition directory, run the following commands:
bash install.bash
bash runDemo.bash
This should run the graphical trainer for a number of the domains using an agent that takes random actions. Note that this is a graphical demo, and will only work if your shell can display a GUI (ie. if you are using SSH, you've specified the -Y or -X flags).
3. Understand the components. An experiment in the rl-competition software consists of 4 components:
The environments and RL_glue software are provided, and should not be modified. It is your job to work with the agents and trainer software.
4. Run a simple graphical experiment. To see how the components fit together, you should run a simple graphical experiment using the sample agents and trainers that we have provided. The environments, or domains, are located in the domains directory in the rl-competition directory. Running an experiment requires you to have two terminal windows open. In both of the terminal windows, change to rl-competition directory. In terminal 1, type the following commands:
cd trainers/guiTrainerJava
bash run.bash
to start the trainer program, and then in terminal 2, type the following commands:
cd agents/randomAgentJava
bash run.bash
to start the agent software. You should see the rlVizApp visualization window. From here, you can choose the domain that you wish to run the experiment on. Pick one of the Generalized domains, and then press "Load Experiment". Press the "Start" button to run the experiment. You can change the speed of the experiment using the slider bar, or proceed one step at a time by pressing the "Stop" button, and then pressing the "Step" button repeatedly to advance the time step.
Since you are running the random agent, the experiment is probably not going very well. To see better results, try running one of the specialized sample agents and matching the environment appropriately. To run one of these agents, on the second terminal, change into the appropriate agent directory before running the run.bash script. Note that most agents are designed to work in a specific environment, and so using an agent in an environment other than the one it was designed to operate in can cause the trainer software to crash.
5. Where to go from here. The next steps that you should take are to explore the environments (located in the domains subdirectory), and the agents (located in the agents subdirectory). Try running a few experiments using the sample agents. We have also included a command-line trainer, that operates without a GUI, located in the trainers/consoleTrainerJava subdirectory. The sample agents will hopefully give you a good launch-point for implementing your own agents, and the included trainers should make testing your agents a simple task.