The accuracy of the SZZ algorithm is pivotal for just-in-time defect prediction because most prior studies have used the SZZ algorithm to detect defect-inducing commits to construct and evaluate their defect prediction models. The SZZ algorithm has two phases to detect defect-inducing commits: (1) linking issue reports in an issue-tracking system to possible defect-fixing commits in a version control system by using an issue-link algorithm (ILA); and (2) tracing the modifications of defect-fixing commits back to possible defect-inducing commits. Researchers and practitioners can address the second phase by using existing solutions such as a tool called cregit. In contrast, although various ILAs have been proposed for the first phase, no large-scale studies exist in which such ILAs are evaluated under the same experimental conditions. Hence, we still have no conclusions regarding the best-performing ILA for the first phase. In this paper, we compare 10 ILAs collected from our systematic literature study with regards to the accuracy of detecting defect-fixing commits. In addition, we compare the defect prediction performance of ILAs and their combinations that can detect defect-fixing commits accurately. We conducted experiments on five open-source software projects. We found that all ILAs and their combinations prevented the defect prediction model from being affected by missing defect-fixing commits. In particular, the combination of a natural language text similarity approach, Phantom heuristics, a random forest approach, and a support vector machine approach is the best way to statistically significantly reduced the absolute differences from the ground-truth defect prediction performance. We summarized the guidelines to use ILAs as our recommendations.

SZZ is the de facto standard method for identifying bug-inducing commits. The accuracy of this method heavily relies on source code management systems, such as Git, as it requires tracing the history of source code changes (i.e., commit histories) to bug-inducing commits. However, it has been reported that these systems introduce biases in commit histories because they only store line-level changes. It is known that such coarse-grained line-level changes can result in the failure to accurately track the commit history and reduce the performance of SZZ. To relieve this challenge, we explore the accuracy of SZZ in token-level changes, which provide finer-grained information to trace commit histories compared to line-level ones, and we discuss the potential benefits and pitfalls of utilizing token-level changes for SZZ. As a result of experiments on 68 OSS projects, we found that SZZ, which uses token-level histories, identifies two new bug-inducing commits that are missed when using line-level histories. Furthermore, our manual analysis of the identified commits indicates that they reduce false-positive bug-inducing commits caused by source code formatting and whitespace changes. However, this improvement in detecting bug-inducing commits comes with a trade-off of 0.081 decrease in overall accuracy, as measured by the F1 score. Consequently, we summarized three potential benefits and five pitfalls of using token-level and line-level tracking for SZZ.

## Development of “Aqua Dungeon” and its Effects

C is an important programming language for many information engineering students, but learning it can be difficult for beginners due to the time-consuming process of understanding its grammar and algorithms. In this study, we developed “Aqua Dungeon,” a C language learning support app that incorporates gamification techniques to improve learner motivation.

Aqua Dungeon Overview

Aqua Dungeon is a mobile app that supports C language learning and is designed to help students review and prepare for specific classes or seminars. The app incorporates the following gamification elements:

• Level System: Students earn points by solving problems and level up.
• Badge System: Students earn badges by achieving certain conditions.
• Ranking System: Students can compare their progress with other learners.
• Calendar Function: Students can visualize their progress.

In addition, the app incorporates adaptive learning techniques to adjust the difficulty of problems based on the learner’s level of understanding.

Experiments and Results

In this study, we conducted the following two case studies to verify the effectiveness of Aqua Dungeon.

Case Study 1: Classroom Use Experiment

Target: Students enrolled in Software Lab II at Kyoto Institute of Technology
Date: June 15 – July 26, 2023

Results:

• Increased participant motivation was confirmed.
• Badges and ranking elements promoted continuation of learning.
• Low usage of the calendar function leaves room for improvement.

Case Study 2: Comparison via A/B Testing

Target: Laboratory students (enrolled in Software Lab I and II)
Date: December 11, 2023 – January 15, 2024
Method: Comparison of participants with and without gamification elements
Results:

The group that introduced gamification showed higher motivation.
The “ranking” and “calendar” features, in particular, were important factors in maintaining learning continuity.
Conclusions and Future Outlook
Aqua Dungeon was confirmed to be effective in maintaining motivation in C language learning. In particular,

The ranking feature increased learning motivation.
The badge and level system provided a sense of accomplishment.
The introduction of adaptive learning allows the app to accommodate individual learning paces.

Future challenges include:

Improving the calendar function (to increase usage),

An answer review function (to make it easier to check mistakes),

Increasing problem diversification (introducing more advanced algorithm problems).

We will continue to improve the app so that it can be used by more learners.

This research demonstrates the potential of gamification in programming education and is expected to be applied to learning other programming languages.

Defect prediction is an important task for preserving software quality.
A lot of previous research has analyzed source code to predict defects; however,
it contains a problem that is its prediction grain is too coarse and its feedback is too late
for software developers.
To achieve a more fine-grained prediction and an earlier feedback, several approaches that analyzes source code changes has been reported.
Those approaches have applied various machine learning techniques and
deep learning techniques to change metrics, such as the number of lines added,
modified files, and modified directories.
In this paper, we propose a novel approach for defect prediction called Word-Convolutional Neural Network (W-CNN), which applies CNN to the modified source code itself.
Our evaluation results show that the proposed approach can improve the effectiveness of defect prediction with a small overhead on the prediction time.

関連論文

Detecting anomalies of a cyber physical system (CPS), which is a complex system consisting of both physical and software parts, is important because a CPS often operates autonomously in an unpredictable environment.
However, because of the ever-changing nature and lack of a precise model for a CPS, detecting anomalies is still a challenging task.
To address this problem, we propose applying an outlier detection method to a CPS log.
By using a log obtained from an actual aquarium management system, we evaluated the effectiveness of our proposed method by analyzing outliers that it detected.
By investigating the outliers with the developer of the system, we confirmed that some outliers indicate actual faults in the system.
For example, our method detected failures of mutual exclusion in the control system that were unknown to the developer.
Our method also detected transient losses of functionalities and unexpected reboots.
On the other hand, our method did not detect anomalies that were too many and similar.
In addition, our method reported rare but unproblematic concurrent combinations of operations as anomalies.
Thus, our approach is effective at finding anomalies, but there is still room for improvement.

Open Data

A log of CPS system, Aqua-tan, recorded from the beginning of system operation.
The log files contain id, timestamp, label, numeric value, and text value.
The “label” column denotes commands and events of Aqua-tan system.

This data is analyzed in our paper in IWESEP2017[1].

• Data: log.csv

References

Once a bug in software is reported, developers have to determine which source files are related to the bug. This process is referred to as bug localization, and an automatic way of bug localization is important to improve developers’ productivity. This paper proposes an approach called DrewBL to efficiently localize faulty files for a given bug report using a natural language processing tool, word2vec. In DrewBL, we first build a vector space model named semantic-VSM which represents a distributed representation of words in the bug report and source code files and next compute the relevance between them by feeding the constructed model to word2vec. We also present an approach called CombBL to further improve the accuracy of bug localization which employs not only the proposed DrewBL but also existing bug localization techniques, such as BugLocator based on textual similarity and Bugspots based on bug-fixing history, in a combinatorial manner. This study gives our early experimental results to show the effectiveness and efficiency of the proposed approaches using two open source projects.

References

visualization

About this topic

We are investigating the code coverage of combinatorial testing and investigating prioritization of test cases using statistical analysis.

Publications

We intend to assess the improvements of Gerrit. The central
concern is “Does Rietveld evolve into Gerrit as the developers intended?” To answer
this question, we first compare qualitative features of two code review tools. We
then conducted an interview with a developer of Gerrit and obtained the developer’s
original intention of improvements in Gerrit. By analyzing mined data from code
review logs, we try to explain the effects of improvements quantitatively. The result
of analysis showed us that the improvements of Gerrit that the developer is expected
are not observed explicitly.

References:

The fault-prone module detection in source code is of importance for assurance of software quality. Most of previous fault-prone detection approaches are based on software metrics. Such approaches, however, have difficulties in collecting the metrics and constructing mathematical models based on the metrics.

In order to mitigate such difficulties, we propose a novel approach for detecting fault-prone modules using a spam filtering technique, named Fault-Prone Filtering. Because of the increase of needs for spam e-mail detection, the spam filtering technique has been progressed as a convenient and effective technique for text mining. In our approach, fault-prone modules are detected in a way that the source code modules are considered as text files and are applied to the spam filter directly.

To show the usefulness of our approach, we conducted an experiment using a large source code repository of Java based open source project. The result of experiment shows that our approach can classify about 85% of software modules correctly. The result also indicates that fault-prone modules can be detected relatively low cost at an early stage.

Related paper: