Comparison with SpotBugs
Github Projects
| Project | Commit | Line of Code |
|---|---|---|
| spring-boot-api-project-seed | c305af6a6bb2b698b02d731ac27218bff51c5c1c | 478 |
| spring-boot-projects | 2071c1da1f27e6e5034f7619c8c08d84cab71e77 | 3045 |
| jetty.project | 1cb0a4978f86f6d0800bb8a4262b7810a253c581 | 329493 |
| heritrix3 | 37ce8d694590b0cf8cbe0a38a58c5f8ee719c4f0 | 41275 |
| webcam-capture | edc89234067cba2766c2d031cf61a9b592e3600c | 16021 |
| paho.mqtt.java | 6f35dcb785597a6fd49091efe2dba47513939420 | 28441 |
| smarthome | 56df0fa09d84912b96c7c8d1f229ca59938d4858 | 181527 |
| SpringBootUnity | 073cb8ef5e32ad195c60bb52f54b94ec31e5410d | 5861 |
| rakam-api | 26ffbc75c3cc2e747d721b96ad6b1f51da09ea0f | 14193 |
| kafka-streams-machine-learning-examples | 3977e3928d2719fc710bb3d202851ee8904be2e7 | 99401 |
| quickfixj | 1968f7abb981f486539a4cf3e4ae81fa5c554ec8 | 1279492 |
| nacos-spring-project | d61ba1761b5fb875de03b9377b7e3c5cb7e44d9b | 7398 |
| jave2 | 23a26109f59a04e65a8a1dc3a0e42be0033a2e7c | 4339 |
| hprose-java | a535b8682547549ba9cc3a04e7acdc9af9894708 | 15845 |
| spring-cloud-release | 10801b4df944fdb5d2dae612ca957914492af6ab | 488 |
| lavagna | 385016963bfc2201e082f5e7edfe81ef1146fc55 | 20694 |
| nacos-spring-boot-project | d77d02926da347bc72f900a2c95bfd8229132197 | 1802 |
| java-speech-api | bd278833f12186750f321bdbba26aae747de51b2 | 1374 |
| java-uuid-generator | 959d0e8c142425e8ed0cdbf0f0ad14bb5ae9d451 | 3073 |
| enunciate | f69749c57fcb8eaee8dbd12d01e916ac0c65cc3f | 54210 |
| camelinaction2 | 9921df2acbe6e3c1d825f4455668097db0205841 | 17832 |
| wro4j | 75922b77298968586643e6ac0f5fcf8fd985a904 | 33410 |
| biojava | 1d03fb4faaa435aff8903483930b60eac8eff56d | 117779 |
| GlobalPlatformPro | 2cde69d4b966bff11cd363c395ed5ff978289257 | 3732 |
| thorntail | d76ec226e3e74deef35a54d7f2d6bf651d286be7 | 137847 |
| spring-comparing-template-engines | 2bb3333e323dfebd2c932403926e52cf8fb247f8 | 908 |
| jmc | 0c7ae2a0f6423d76f74d9dba53fe5d693a3a7c0c | 163462 |
| sharpen | 0a539c4e6ab58c462baf3b999c1f4a53b6fa113e | 9334 |
| Benchmark | ef7ec5f317c6c47d935a955c6efc35e0e583a40c | 146265 |
| owasp-java-encoder | fa5c589fb6037e0b7e9bb6b21b537a5411420ac7 | 3264 |
| core | 4c18b6b8046ce0490ae8f9bded31fe3d248a7f73 | 108489 |
| java-spring-cloud | 76fc7de41a7c26590c9fff59d7ef211d87f7cc3f | 5340 |
| Springy-Store-Microservices | 786c0c7cf62364c94fe9a148310567c9adbfb244 | 2903 |
| jmh | f3e7d82b0b68ab345f81e0598c7f850c5b334027 | 249879 |
| fabric8-maven-plugin | 2631458a22ccc8f4110c090b3564b9178dc5ec13 | 48011 |
| simple-sso | d3b33a5788c071f294a5a41a61192ef03ba1e744 | 390 |
| raml-for-jax-rs | aa24bf32b926f23f5c1826fdada07f4c1cc64c77 | 14789 |
| common-project | ff019b8b24a984fb265f92d840a1a0705f174fe6 | 5754 |
| hadoop-mini-clusters | 787a02c728a1035c0742a22468a227616dc748d0 | 9499 |
| qulice | e296098d1722f13ad6644f6a2958b5e8d1cbc47c | 6557 |
| superword | fad61fad5ea688ff5bf8838ee2a9b5ba2ae81349 | 9079 |
| java-microservice | 2880ffb60cf0cd0afd8455cbeb1789a51ffbb62f | 2593 |
| ORCID-Source | 647c4206daabb8b6d9bdb44059aabc07c16540e7 | 247335 |
| demo-rest-jersey-spring | a46ab55dbb6ecd5e97cdc253036b068d02e5573b | 1115 |
| aws-java-sample | 159b1d4edec782c28a13e1cf891f87d01fb6ff0f | 89 |
| camelinaction | 1d74f64c4aefd318baee21cb1b53928bc9f45763 | 7586 |
| ShoppingCart | f3793243e868f246806d54571c94c7251db2c22a | 1255 |
| iot-dc3 | 62fd0a51a59e249b7de66e87ea7978edffff7ee0 | 14671 |
| gchisto | d140c4d313182b0a06af10905be1ece648c35932 | 6574 |
| opencc4j | 9ccd603beadde849faba169f7b6b590e0c73ef49 | 872 |
| visualee | 88732d9dbe5031dad9c9f85a4c4b35e5f1551f95 | 3762 |
| ProjectAres | 7484cb81fcc33814fe26812684660da4e0dbd671 | 90184 |
| appbundle-maven-plugin | 4cdaf7e0da95c83bc8a045ba40cb3eef45d25a5f | 420 |
| reactive-ms-example | c40d5855ce2c5e0b6dfb06aa4a421ae9560d6548 | 1485 |
| microservices-sample | 994956f01093c78eb7d8275c1a30c8e385e498d4 | 672 |
| hope-cloud | b3a0673325210efaa97502c2d3eb584d18f1dafb | 135 |
| javaee7-essentials-archetype | 3ea15d451574e118ef9ccff69a5e6ac9d6371a7b | 5 |
| pandomium | 4d730c265ad324aea9dada15a29c9465169ce475 | 6083 |
| confucius-commons | ffbe5331df105c4a95ee2f0b95a4eeb6df6a9caa | 3341 |
| jol | eea9f9d77e837075e0e237f061a7716e05237408 | 7204 |
| pdt | ed9fe5920966ceb5cd9242263bb6a1eff9147afc | 275896 |
| deptective | e5bd0f1dcc9dc23c1b3175615049e6e30b093d46 | 2730 |
| cms-admin-end | 40689458c9cce66818636be5effc47213098b960 | 8229 |
| cli-spring-boot-scaffold | 780ad4c90efae4d27e4faf1884ebfd5fe6541944 | 2425 |
| flyer-maker | 5242400cb05a71b87f4bac8b6ace18ef98eb41c3 | 757 |
| spring-context-support | ed713c9079b9944de69f775fba5eb90afa4ee4dc | 3061 |
| reddit-bot | d98821abcbf9930ecbbd99a4a7634b3c0ca59461 | 1499 |
| cloud-espm-v2 | 487798e23a736bda0e12778da1594072fd1f5557 | 4675 |
| sldeditor | 4b33b4e9029402d89335da2bb7d427a39d796b19 | 103566 |
| missinglink | d632aa6d1feeb290b6441dc836c66c9384ea5703 | 6483 |
| osgi.enroute | e3ecd08fd038432fd588d7b41b07cacd54452c35 | 991 |
| spring4.x-project | a023e54cc742e490e2724bf4a979d2864997b22a | 576 |
| spring-mvc-angular-js-hibernate-bootstrap-java-single-page-jwt-auth-rest-api-webapp-framework | a38cbd855f449ce45470931345d62ee1c9f976d6 | 2000 |
| spring-boot-graalvm | c99075364aa8abb37444cfcd94b9a88f005927c1 | 48 |
| oauth2-server | 078b2138e7e7c7704bc627fbf8fc6c3e740f892b | 2977 |
| bitfinex-v2-wss-api-java | fda4a7aa96f24f4814c1d5bfa2ca66465de3b549 | 6412 |
| lombok.maven | 12666badea6a45174710a45ce3b7abe0a9d81352 | 565 |
| client-maven-plugin | 76f02d5805c45f357013d2a0cad3f418d8a8c4aa | 823 |
| travels-java-api | 3a7fd4cc36938666f09ee5945a28345763105075 | 2953 |
| cargotracker | 46424c6d8b708528c7ced6f5cee825601ead73e2 | 5973 |
| tij4-maven | de72e0e12263facb1e226ebdbec5b489785106bc | 17890 |
| code-assert | a722c20b1541cabc42f08eb4d33fd46e9745da84 | 7612 |
| aem-component-generator | d966ead3891e1d2a2de8aa5130b203837229b875 | 1648 |
| cloud-opensource-java | 4edd644c9453da16cd448e2bbb1b85ac1cf7e168 | 10509 |
| javaee8-essentials-archetype | 9650c1391f49df9f549e142aa501b58374bb2a1b | 18 |
| Angular2AndJavaEE | 92a87643e97eb72a406a2118d97fe27b958f106c | 1424 |
| GetOldTweets-java | 9975cd652dad16c7000d0a5fcdbed9e9836b9256 | 279 |
| Project-16x16 | 47bd409b8e320c24c5e85d3038518e4d0b5f6d58 | 5352 |
| slib | 21c57a5c52a2d0557fec8d9de7b46252452cdcdc | 22107 |
| javaee-javascript | f1ea2531a0a28ac55c7993b4c5668ef8fac6d783 | 348 |
| community | 8c68cfde39007f89d3510cdb16c82dcf05a6ff4e | 3720 |
| spring-boot-java-swing-reservations | 81be071a2d2a7dab57e1ab2850b5ae3a3fb258c5 | 5299 |
| asmsupport | b3933a900c6d51046d402969ce897e5549eea914 | 26710 |
| projectforge | 20d9486426610e172de105b379844ee3340f4ea2 | 122849 |
| test-driven-spring-boot | 2b947d9826f1bfda0cad5035fc29b5ed85f70358 | 1073 |
| Deeplearning4J | eea1cf062af827b9ee7a6a655e00e4ea78d73a51 | 706 |
| java | 4c8192a7fa28c3aeeb93301e5608d2d4ec7e5b38 | 367 |
| spring-zeebe | 9e9d6bd41074aa669a86a88f8276558104849067 | 1583 |
| triava | 049484fa865691a7e27db4de7a57aed98c0654d9 | 5916 |
| qtiworks | 2d8c502d8f2b266fd80197ad19bcae3d4566276a | 51636 |
Comparison Results
Quality of Warnings
We evaluate Codegex (C) and SpotBugs (S) on 52 open-source Java projects in GitHub. We measure the quality of the warnings generated by each tool. Relative accuracy and recall are two widely used metrics to compare the quality of analysis results, whereas Jaccard index (J(S, C)) is used to measure the similarity for two sets of data (between S and C).
The high Jaccard index between the warnings generated by SpotBugs and those generated by Codegex (0.86) indicates that the analysis results of Codegex are comparable to that of SpotBugs. As highlighted in Table 5, Codegex outperforms SpotBugs in accuracy and recall for seven patterns. Moreover, Codegex also achieved comparable results with SpotBugs in the overall accuracy and recall. In Table 5, we also observe that: (1) for most bug patterns, Codegex can find one additional TP than SpotBugs, and (2) Codegex performs particularly well for the DMI_RANDOM_USED_ONLY_ONCE pattern by finding more TPs and fewer FNs.
Analysis Time
Codegex can run up to 24k faster than SpotBugs when considering initial compilation time and analysis time. For generating analysis report, Codegex can run up to 590X faster.
