Overview of BIMAN: A Technique to Detect Bots that Commit Code
BIMAN, or Bot Identification by commit Message, commit Association, and author Name, is an innovative technique that helps detect bots that commit code. BIMAN is comprised of three methods that consider independent aspects of the commits made by a particular author. The three methods that are used in BIMAN are Commit Message, Commit Association, and Author Name.
Commit Message
Commit messages are essential for understanding the changes that have been made to the codebase. However, bots that commit code often use templates to generate their commit messages. Therefore, the first method that BIMAN uses is to identify whether the commit messages have been generated from templates or not. This is done by analyzing the structure of the messages and comparing them to predefined templates.
For instance, if a developer commits a code change with the message "fixed a bug," this message could be labeled as suspicious if it is similar to those that are being used by bots. However, if the message contains specific information relevant to the code change, such as "fixed a bug in the login screen that prevented users from entering their passwords," it is less likely that the message was generated by a bot.
Commit Association
The second method of BIMAN is to predict if an author is a bot by analyzing the files and projects associated with the commits as predictors. This is done using a random forest model, which predicts the probability that an author is a bot for each commit. The model uses several features related to files and projects associated with the commits as predictors, such as the number of files changed, the number of lines added and deleted, and the programming language used.
For instance, if a bot commits code to a repository, it is likely that the bot will commit code to several files simultaneously, whereas a real developer may only make changes to a single file at a time. The model also considers the time of day and the day of the week when the commit was made, as bots tend to commit code at odd hours or on weekends.
Author Name
The third method of BIMAN is to match the author's name and email to common bot patterns. Bots that commit code often use generic names such as "build" or "update," which are easy to detect. BIMAN analyzes the author's name and email to determine if it follows a pattern that is common among bots. For example, if the author's name is "Bot Master" and the email address is "[email protected]," BIMAN will flag this as suspicious.
Overall, BIMAN is a powerful technique for detecting bots that commit code. It is important to note that BIMAN is not perfect, and it may occasionally flag real developers as bots. Therefore, additional steps may be necessary to confirm whether an author is a bot or not. However, BIMAN is a step in the right direction in improving the security of code repositories and preventing unauthorized access to sensitive code.
