Camouflaging the Camouflage
Note: This work is an extension of work performed under the auspices of IQT Labs. IQT Labs has explicitly stated that it has no intention of pursuing this topic, and encouraged the author to further explore and publish this project.
Advanced computer vision object detectors are known to be susceptible to adversarial patch attacks with can fool the detector into a missed detection or miss-classified object. In our previous posts [1, 2] we released the CAMOLO codebase, and showed that adversarial patches are effective in reducing YOLT detection performance of vehicles viewed from overhead. We also showed, however, that adversarial patches are typically very easy to detect, thereby calling into the question the utility of such patches in a pragmatic sense.
In this final installment in our exploration of adversarial camouflage, we detail efforts to develop patches whose presence is difficult to detect, while still obfuscating objects of interest.
1. Patch Detection
In this section we briefly review our prior work detailing the ease of detecting the presence of adversarial patches. In our previous blog [2] we trained a library of 24 adversarial patches designed to disguise buses, cars, trucks and vans, and showed that the presence of adversarial patches is easily discovered, see Figure 1.
In fact, patches are on average 24% more detectable than the objects the patches were meant to obfuscate. This is illustrated in Figure 2, where the orange bars (patch detection) are higher on average than the blue bars (vehicle detection) for each patch.
2. Adversarial-Yolo Overview
The CAMOLO codebase creates patches to fool YOLO-based object detectors, and is built upon the impressive “adversarial-yolo” work of Thys et al. Thys and company created patches to fool automated person detectors by optimizing a three-part loss function:
L = 𝛼Lₙₚₛ + 𝛽Lₜᵥ + Lₒ
Lₙₚₛ corresponds to a “non-printability” score that represents whether the patch colors can be created by common printers. The Lₜᵥ term corresponds to the smoothness or randomness of the patch, and Lₒ is the maximum object score in the image. Minimizing the final term (Lₒ) is most crucial, as the goal of the patch is to hide objects of interest. Over the course of training, and initially random patch is overlaid on the test imagery, and as the loss function slowly decreases an adversarial pattern emerges. A few patches trained to obfuscate vehicles in overhead imagery are shown in Figure 3.
3. Stealthy Patches
We now investigate if it is possible to develop adversarial patches whose presence is not quite so obvious. The orange bars of Figure 2 denote the performance of a YOLT model (which we shall denote Mₚ) we trained to detect adversarial patches. [The blue bars of Figure 2 denote performance for the vehicle detection model, or Mᵥ.] We use this patch detector Mₚ to add an additional term to the loss function, which now becomes:
L = 𝛼Lₙₚₛ + 𝛽Lₜᵥ + Lₒ + γLₚ
Lₚ is the probability of detecting the existence of the patch, which is computed by running the test imagery (with patch proposals overlaid) through Mₚ. We set γ = 1, which means that hiding the patch is equally as important as hiding the objects of interest. We run three experiments with different configurations, yielding the patches shown in Figure 4.
In Figure 5 we show performance of the original YOLT model Mᵥ (trained to detect buses, cars, trucks, and vans) with the “obj_only_w0” stealthy patch applied. The yellow boxes denote the ground truth object, and YOLT detections are colored orange, red, and green. The patch is clearly not hiding all detections, though performance is significantly reduced.
In Figure 6 we show how well the YOLT model trained to detect patches (Mₚ) performs on the stealthy adversarial patch. Figure 6 clearly shows that the patch detection model does a very poor job in finding this patch.
4. Stealthy Patch Performance
Let us compare the performance of the stealthy patches to our standard patches. In Figure 7 we show the vehicle detection (blue) and patch detection (orange) performance of each patch, with the stealthy patches outlined by the red box. Note that the orange bars for our stealthy patches are far lower than the mean of the non-stealthy patches.
The only non-stealthy patches that come close are the two configurations (“obj_only_tiny_gray_v0”, “obj_only_tiny_gray_v1”, see Figure 3) that we forced to be black and white. The salient point here seems to be that these two black and white patches as well as the stealthy patches were forced to be far different from the remainder of the patches, thereby fooling our patch detection model.
In Figure 8 we collapse the orange and blue bars to show the combined detection score (recall that lower is better). The stealthy patches are on average far more performant than the other patches. The mean detection score for non-stealthy patches is 0.67, whereas it is 0.11 for the stealthy patches.
5. Conclusions
AI Assurance is a hot topic as of late, and confidence in the predictions of computer vision object detection systems is high on the priority list. And while object detection systems have been shown to be susceptible to attacks in the form of adversarial patches, our prior work showed that such patches are easily detected.
In this post we added a term to the loss function for adversarial patch training that scored the ability of the patch to evade a generic patch detection model. This approach yielded patches that are just as effective as obfuscating vehicles, yet also proved very effective in evading our generic patch detector.
While this approach is certainly not infallible (it is highly dependent on the robustness of the patch detector, for instance), it is relatively easy to implement, open-source, and in our experiments significantly improved the stealthiness of adversarial patches. For the curious, feel free to dive deeper in our paper, explore the CAMOLO codebase (particularly the notebooks), or leverage the open-source nature of the work to improve upon these first steps.
