Fuzzing software with deep learning
University of Glasgow, 2023
Online
Hochschulschrift
Zugriff:
Generation based fuzz testing can uncover various bug classes and security vulnerabilities. However, compared to mutation based fuzz testing it takes a great amount of time to develop a well balanced generator that generates good test cases and decides were to break the underlying structure to exercise new code paths. This thesis provides an evaluation of generative deep learning algorithms to generate HTML test cases to fuzz test a browser's HTML rendering engine. The experiments highlight that various deep learning algorithm are performing well in this setting. However, there are large differences in the stability of the training and code coverage performance. The best performing in terms of code coverage as well as training stability is a Temporal Convolutional Network (TCN). The TCN model is then also used to learn from real world HTML data to generate novel test cases withouth the need of a generative fuzzer in the first place. The results show that the approach is able to discover new code areas that were neither discovered by the underlying fuzzer nor the prior models. Furthermore, this highlights how an existing fuzzer can be augmented with the help of a deep learning model and publicly available training data. Finally, reinforcement learning is used to further improve the existing fuzzer by utilizing the code coverage data from the browser under test. The designed DDQN agent is able to guide the test case creation of a TCN to even outperform the underlying baseline test case generator.
Titel: |
Fuzzing software with deep learning
|
---|---|
Autor/in / Beteiligte Person: | Sablotny, Martin |
Link: | |
Veröffentlichung: | University of Glasgow, 2023 |
Medientyp: | Hochschulschrift |
DOI: | 10.5525/gla.thesis.83496 |
Schlagwort: |
|
Sonstiges: |
|