Experimental package of the SFL user study about the Jaguar tool.
Put the plugin's jar file in the eclipse/plugins folder. There are six versions of the Jaguar plugin:
- Jaguar first + method list: the first task uses Jaguar with a list of methods and the second task does not use it (English | Portuguese).
- Jaguar first + line list: the first task uses Jaguar with a list of lines and the second task does not use it (English | Portuguese).
- Eclipse first + method list: the first task does not use Jaguar and the second task uses it with a list of methods (English | Portuguese).
- Eclipse first + line list: the first task does not use Jaguar and the second task uses it with a list of lines (English | Portuguese).
- Tryout version + method list: this version has Jaguar with a list of methods to test the tool before the experiment (English | Portuguese).
- Tryout version + line list: this version has Jaguar with a list of lines to test the tool before the experiment (English | Portuguese).
You'll need to change the Configuration class to set the type of list (ONLY_METHOD_LEVEL or ONLY_LINE_DUA_LEVEL), the Jaguar's EXPERIMENT_VERSION as true, choose the option SEND_DATA as true, and EXPERIMENT_JAGUAR_FIRST as true to create a version starting with Jaguar or false to start without Jaguar.
When a participant finishes the experiment and clicks on the stop button, the log data is sent to a server. The Jaguar jar files available here were set to send to our server. To use Jaguar, you'll need to clone the project change the ScpSend class to set your server, user and folder, and create a new jar.
The jsoup and XStream programs were used for the experiments. Additionally, we used the jtopas program for the training step. The faults and the suspiciousness lists were already included in the code. Just import those programs in Eclipse to run them using Jaguar. The programs are available here.
We used the Ochiai ranking metric to calculate suspicious program elements of jsoup and XStream. The SFL lists are available here.
We used the Eclipse Luna for the experiments. We installed five Eclipse IDEs in the VMs: one for each of the four experimental versions due to the different Jaguar plugins. We installed in Eclipse in the users' home folder (/home/{user}/eclipse). The other Eclipse was installed (/usr/share/eclipse-example) for the tryout version of Jaguar, which is the same for the four VM users.
Files such as links and desktop icons we used in the VMs are available here. The eclipse.desktop file must be edited to point the correct path of users' eclipse folders (i.e., expone, exptwo, expthree, or expfour).
We carried out the experiments using the lubuntu 14.04 OS.
We created the VMs using VirtualBox. The VMs were configured with 1GB RAM.
There are training documents for Jaguar list and Jaguar method, which are available in English and in Portuguese. We renamed these files as jaguar-tool.pdf in the VMs.
The experiment instructions for the different VM users are available in English and in Portuguese. Each file explains the order in which the experiment must be done. The instructions are the same for Jaguar method and Jaguar list. We put it in the Desktop area renamed simply as instructions.pdf.
Video of a usage example of Jaguar in English. Video of Jaguar with the method list in Portuguese. Video of Jaguar with the line list in Portuguese. The videos for Jaguar method and Jaguar list were put at the Desktop area and renamed as jaguar-video.mp4.
This questionnaire is a web application called devselector. It receives the input data of a participant candidate to select whether s/he is eligible to take part in the user study and allocates her/him to an experimental group. The HIA method is used for this allocation process.
This questionnaire was created as an Google Form. A copy of this questionnaire is available in English and in Portuguese.
We installed the devselector application in a Tomcat 8 server. This code was developed using Java 7. More details on the README file in the devselector repository. The files required by devselector (including the HIA algorithm) are available here.
After filling out the questionnaire, devselector creates an ID for the participants and provides a link to the corresponding VM as well as the user login and password. There are three pages in devselector:
- selector-form.xhtml: contains the questionaire.
- selector-response.xhtml: shows the response after filling out the questionnaire.
- monitor.xhtml: shows how many participants were allocated in the experimental groups.
You must change the constants serverLink and DIR_PATH_ABOVE for your server and application folder, respectively.
The sent and S10send scripts (available here) use the scp protocol to send the log data to a server as a backup if the participant did not press the stop button in the end of the experiment. You need to set the server, user and folder. Put the sent script in the /etc/init.d folder and the S10send scripts in the /etc/rc0.d and /etc/rc6.d folders to run automatically.
There are two VMs, one for Jaguar Line and other for Jaguar Method. Each VM has four users to interchange the order in which the faults and the techniques are used. The VM users are:
- Experiment One (expone): the participant starts using Jaguar and the jsoup program, and then uses Eclipse without Jaguar in the XStream program.
- Experiment Two (exptwo): the participant starts using Jaguar and the XStream program, and then uses Eclipse without Jaguar in the jsoup program.
- Experiment Three (expthree): the participant starts using Eclipse without Jaguar and the jsoup program, and then uses Jaguar in the XStream program.
- Experiment Four (expfour): the participant starts using Eclipse without Jaguar and the XStream program, and then uses Jaguar in the jsoup program.
The users' passwords are, respectively, dev110150, dev123400, dev365001, and dev589651. For the experiment, these VMs were zipped and put in a server to be downloaded by the participants. We used the same server that hosted devselector, providing a link such as server-address/vm-file.zip.
The VMs we used in the experiments are available for Jaguar method and Jaguar line. It is important to remember that the Jaguar plugins and the script to send the log data were set for our servers so that you need to replace the plugins before running experiments.
Jaguar was developed and the experiments were performed using Java 7.
Feel free to ask us in case of doubts.
@article{
title={Understanding the Use of Spectrum-based Fault Localization},
author={Higor A. de Souza and Marcelo de S. Lauretto and Marcos L. Chaim and Fabio Kon},
journal={Journal Of Software: Evolution And Process},
volume={},
number={},
pages={},
year={2023}
}