What is Fuzz Testing?<\/strong><\/h2>\n\n\n\n![\"Fuzz](\"https:\/\/lh4.googleusercontent.com\/HF1DC6sVqRp2_bvLk9c6Lfvk2phe3tlualaAbQNRF0d3-nQlcLSDZtDg_dsYiMh_F7P87B2YxgZTvk-KlHKKRYQYrzNcUgQ5oRb9_XHRPoCdeYwP9SEIvHsPjhbJ7uiqKFtV83TkLaNP9AAUk5jawg\")
Fuzz Testing Process Phases<\/figcaption><\/figure>\n<\/div>\n\n\nFuzz testing is an automated testing method in which we provide random input values to discover unexpected bugs.<\/strong><\/p>\n\n\n\nFuzzing can be traced back to the University of Wisconsin in 1988. There, Professor Barton Miller gave a class project titled \u201cOperating System Utility Program Reliability \u2013 The Fuzz Generator.\u201d<\/p>\n\n\n\n
It was the first \u2013 and simplest \u2013 form of fuzzing, and included sending a stream of random bits to UNIX programs with the use of a command line fuzzer.<\/p>\n\n\n\n
Nowadays, there are some different types of Fuzz Testing: dumb fuzzing, smart fuzzing, feedback-based fuzzing, mutation-based fuzzing, and generation-based fuzzing, which are all described below.<\/p>\n\n\n\n
Types of Fuzzers<\/strong><\/h2>\n\n\n\nRandomizing styles<\/strong><\/h3>\n\n\n\nThere are two Fuzzing styles of generating randomized data:<\/p>\n\n\n\n
- Entirely random: dumb fuzzing test<\/strong>;<\/li>
- Partially randomized to match some criteria: smart fuzzing test<\/strong>.<\/li><\/ul>\n\n\n\n
Dumb Fuzzing Test<\/strong><\/h3>\n\n\n\nDumb fuzzing requires less amount of work, it’s easier to maintain, but it has limited coverage. <\/p>\n\n\n\n
It is relatively easy to set up, but it is also very inefficient because it is not based on any criteria, it just generates the entirely random data.<\/p>\n\n\n\n
Smart Fuzzing Test<\/strong><\/h3>\n\n\n\nOtherwise, Smart fuzzing has a greater code coverage, catching more bugs. Smart Fuzzing Test produces inputs that are based on valid input formats.<\/p>\n\n\n\n
It is very useful since we need to match certain patterns. This style of fuzzing requires detailed knowledge about input format, so it is more costly to set up, which involves a longer development and bigger strategy.<\/p>\n\n\n\n
Depending on the environment, we can use both types in the code (Smart and Dumb Fuzzing).<\/p>\n\n\n\n
Smart is selected for parts where we need more strategy and match specific criteria. Dumb is for the simplest tests, where we just need to randomize without any criteria.<\/p>\n\n\n\n
Fuzzing Types<\/strong><\/h2>\n\n\n\nSmart randomizing style<\/strong> can be divided into three types:<\/p>\n\n\n\n- Mutation-based<\/strong><\/li>
- Generation-based<\/strong><\/li>
- Feedback-based<\/strong>.<\/li><\/ul>\n\n\n\n
Sometimes we can find different names or explanations for each fuzzing style\/type. <\/p>\n\n\n\n
Here is a description of each one for a better understanding.<\/p>\n\n\n\n
Mutation-based <\/strong><\/h3>\n\n\n\nIt changes the valid inputs generating a collection of mutated inputs, which then introduces small changes to existing inputs that may still keep the input valid with new behavior.<\/p>\n\n\n\n
Sometimes it\u2019s classified as \u201cdumb-fuzzer\u201d considering the mutation of the whole input data.<\/p>\n\n\n\n![\"Bit](\"https:\/\/lh4.googleusercontent.com\/tq38W16HdbzynCB-a4NOhJ0e2uWoTL6aqgbmvbVcjY_wRnOhTmcgI9HuW3N7SlC03y07qB2Dh6biv8_-GGmZsXGFTR8i5YIgGQrxUAsObKb0GToMs56uNj06yigicL4-D3zAH6sM1Hmz2IZxdp7G7Q\")
Example: Bit Flipping (bits are flipped in some sequence or randomly)<\/figcaption><\/figure>\n\n\n\nGeneration-based <\/strong><\/h3>\n\n\n\nIt creates entirely new data matching the valid input, according to the defined structure. It can be defined by data modeling or a state modeling, for example.<\/p>\n\n\n\n
The Model is defined and the fuzzer randomizes the data according to the model structure.<\/p>\n\n\n\n
Here is a sample Data Model for HTTP Protocol:<\/p>\n\n\n\n![\"Data](\"https:\/\/lh5.googleusercontent.com\/Q2ZlwrOOdxrqrQ-Z-yqFoglbx6DWxWHxNvD1sDSzOWRgC-YbU-KbZT2MXWny-QcmwxovN3OmCJ_zUYCoq5II2Bd8_83xMNWosGXRuOCBea2MejXC2ci8rohZXInix44vWFiM4CHfBFcFJ-8KNFeZWA\")
<\/figure>\n\n\n\nHere is a sample State Model and Data for HTTP:<\/p>\n\n\n\n![\"State](\"https:\/\/lh5.googleusercontent.com\/ywaNOTNuFP5AA8Yt5k9Yw3h1wvWpQkXzByqOxSIIMmbgaKtpQBdz-2DeVjn5ll6v7H4hFoJZs8Vn0_nm2PDVgLyOM1W0-qdGCwn6iy17pWKc0WdtOy8olE2cKikxFYRpBXUAzXshts_WMbBroMr1Hg\")
<\/figure>\n\n\n\nFeedback-based<\/strong><\/h3>\n\n\n\nAlso called “Coverage-based”. It uses code coverage information when generating new inputs.<\/p>\n\n\n\n
It is considered the smartest way of Fuzzing, but also the most costly one. This type measures the amount\/share of code executed during a given test.<\/p>\n\n\n\n
For example, it may read the number of statements in the code and compare it to the statements in the software source.<\/p>\n\n\n\n
Same thing for the number of lines, the share of defined functions that have been called, the execution of code lines within basic blocks, the edges of the control graph that are being followed, the number of logical paths, etc.<\/p>\n\n\n
\n![\"Diagram](\"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2022\/08\/fuzz-type-feedback-base.png\")
A diagram of how Feedback-Based Fuzzing works<\/figcaption><\/figure>\n<\/div>\n\n\nAnother important configuration<\/strong><\/h2>\n\n\n\nFuzzing Tests can be highly configurable. Some important configurations we had to use in our projects are listed below.<\/p>\n\n\n\n
Fuzztime<\/strong><\/h3>\n\n\n\nConsidering we worked with blockchain, it was very important to determine a limit of time for a specific test. This way, we could also test performance and determine a time limit so we would not be sending infinite data.<\/p>\n\n\n\n
Parallel Processes<\/strong><\/h3>\n\n\n\nWe could configure Fuzzing to be working in several parallel processes. It also got useful for performance testing and we could test the behavior on the network receiving several parallel data at the same time.<\/p>\n\n\n\n
Data Type definition<\/strong><\/h3>\n\n\n\nThe Fuzzing can be configured to accept only specific types of data. Very useful for an environment where we need to randomize, but aiming only one or more specific types of data.<\/p>\n\n\n\n
Why use it on blockchain projects?<\/strong><\/h2>\n\n\n\nConsidering we need to pay special attention to security on blockchain<\/a> projects, and need to test the widest range of accepted data possible, fuzzing tests are very interesting for the blockchain environment.<\/p>\n\n\n\nFuzzing is very important to test security because it may test what kind of data the system is accepting or not accepting, and also test simultaneous parallel processes certifying one process is not overlapping another concurrent process using random inputs. <\/p>\n\n\n\n
It\u2019s important to certify the system is working according to the real environment of a blockchain network, receiving several different data simultaneously, and being accepted by the network rules without losing important information.<\/p>\n\n\n\n
Why did Cheesecake Labs decide to use it?<\/strong><\/h2>\n\n\n\n
Fuzz testing is an automated testing method in which we provide random input values to discover unexpected bugs.<\/strong><\/p>\n\n\n\n Fuzzing can be traced back to the University of Wisconsin in 1988. There, Professor Barton Miller gave a class project titled \u201cOperating System Utility Program Reliability \u2013 The Fuzz Generator.\u201d<\/p>\n\n\n\n It was the first \u2013 and simplest \u2013 form of fuzzing, and included sending a stream of random bits to UNIX programs with the use of a command line fuzzer.<\/p>\n\n\n\n Nowadays, there are some different types of Fuzz Testing: dumb fuzzing, smart fuzzing, feedback-based fuzzing, mutation-based fuzzing, and generation-based fuzzing, which are all described below.<\/p>\n\n\n\n There are two Fuzzing styles of generating randomized data:<\/p>\n\n\n\n Dumb fuzzing requires less amount of work, it’s easier to maintain, but it has limited coverage. <\/p>\n\n\n\n It is relatively easy to set up, but it is also very inefficient because it is not based on any criteria, it just generates the entirely random data.<\/p>\n\n\n\n Otherwise, Smart fuzzing has a greater code coverage, catching more bugs. Smart Fuzzing Test produces inputs that are based on valid input formats.<\/p>\n\n\n\n It is very useful since we need to match certain patterns. This style of fuzzing requires detailed knowledge about input format, so it is more costly to set up, which involves a longer development and bigger strategy.<\/p>\n\n\n\n Depending on the environment, we can use both types in the code (Smart and Dumb Fuzzing).<\/p>\n\n\n\n Smart is selected for parts where we need more strategy and match specific criteria. Dumb is for the simplest tests, where we just need to randomize without any criteria.<\/p>\n\n\n\n Smart randomizing style<\/strong> can be divided into three types:<\/p>\n\n\n\n Sometimes we can find different names or explanations for each fuzzing style\/type. <\/p>\n\n\n\n Here is a description of each one for a better understanding.<\/p>\n\n\n\n It changes the valid inputs generating a collection of mutated inputs, which then introduces small changes to existing inputs that may still keep the input valid with new behavior.<\/p>\n\n\n\n Sometimes it\u2019s classified as \u201cdumb-fuzzer\u201d considering the mutation of the whole input data.<\/p>\n\n\n\n It creates entirely new data matching the valid input, according to the defined structure. It can be defined by data modeling or a state modeling, for example.<\/p>\n\n\n\n The Model is defined and the fuzzer randomizes the data according to the model structure.<\/p>\n\n\n\n Here is a sample Data Model for HTTP Protocol:<\/p>\n\n\n\n Here is a sample State Model and Data for HTTP:<\/p>\n\n\n\n Also called “Coverage-based”. It uses code coverage information when generating new inputs.<\/p>\n\n\n\n It is considered the smartest way of Fuzzing, but also the most costly one. This type measures the amount\/share of code executed during a given test.<\/p>\n\n\n\n For example, it may read the number of statements in the code and compare it to the statements in the software source.<\/p>\n\n\n\n Same thing for the number of lines, the share of defined functions that have been called, the execution of code lines within basic blocks, the edges of the control graph that are being followed, the number of logical paths, etc.<\/p>\n\n\n Fuzzing Tests can be highly configurable. Some important configurations we had to use in our projects are listed below.<\/p>\n\n\n\n Considering we worked with blockchain, it was very important to determine a limit of time for a specific test. This way, we could also test performance and determine a time limit so we would not be sending infinite data.<\/p>\n\n\n\n We could configure Fuzzing to be working in several parallel processes. It also got useful for performance testing and we could test the behavior on the network receiving several parallel data at the same time.<\/p>\n\n\n\n The Fuzzing can be configured to accept only specific types of data. Very useful for an environment where we need to randomize, but aiming only one or more specific types of data.<\/p>\n\n\n\n Considering we need to pay special attention to security on blockchain<\/a> projects, and need to test the widest range of accepted data possible, fuzzing tests are very interesting for the blockchain environment.<\/p>\n\n\n\n Fuzzing is very important to test security because it may test what kind of data the system is accepting or not accepting, and also test simultaneous parallel processes certifying one process is not overlapping another concurrent process using random inputs. <\/p>\n\n\n\n It\u2019s important to certify the system is working according to the real environment of a blockchain network, receiving several different data simultaneously, and being accepted by the network rules without losing important information.<\/p>\n\n\n\nTypes of Fuzzers<\/strong><\/h2>\n\n\n\n
Randomizing styles<\/strong><\/h3>\n\n\n\n
Dumb Fuzzing Test<\/strong><\/h3>\n\n\n\n
Smart Fuzzing Test<\/strong><\/h3>\n\n\n\n
Fuzzing Types<\/strong><\/h2>\n\n\n\n
Mutation-based <\/strong><\/h3>\n\n\n\n
Generation-based <\/strong><\/h3>\n\n\n\n
Feedback-based<\/strong><\/h3>\n\n\n\n
Another important configuration<\/strong><\/h2>\n\n\n\n
Fuzztime<\/strong><\/h3>\n\n\n\n
Parallel Processes<\/strong><\/h3>\n\n\n\n
Data Type definition<\/strong><\/h3>\n\n\n\n
Why use it on blockchain projects?<\/strong><\/h2>\n\n\n\n
Why did Cheesecake Labs decide to use it?<\/strong><\/h2>\n\n\n\n
![\"Project](\"https:\/\/lh6.googleusercontent.com\/Uh1KSk2txmrqFtpGxbfh_f3LVQbjDoFpnQa7WuqdPZ8MdKQ75nrTrO29PuFdkW2Wn-oXEZOio2Sz1bneM0Er7y7wS11yvKHKWSZQ7kDfks71lGu7ZRuwtP-es4HIVBzeizkS-LdKPQN0i5zsnLpUuQ\")