Asymmetric Certified Robustness via Feature-Convex Neural Networks – The Berkeley Artificial Intelligence Research Blog

Tldr: We suggest The asymmetric dependent durability The problem, which requires accredited durability for only one category and reflects aggressive scenarios in the real world. This concentrated setting allows us to provide simulations of features for features, which produce a closed -shaped diameter and an inevitable charity in the arrangement of millimeters.

Figure 1. Clarification of common features and sensitive degree inputs. This structure consists of the continuous lipschitz feature $ \ Varphi $ with a convex job that is $ G $. Since $ G $ G is convex, it is rapprochement globally from the shade of $ \ Varphi (X) $, which gives credit base balls in the feature space. Lipschitzness $ \ Varphi $ and then gives properly specific certificates in the original input space.

Although they are widely used, deep learning works are completely vulnerable Adian examplesSmall image disorders, which deceive automated learning models, in a modified input classification. This weakness strongly undermines the reliability of the critical processes of safety that includes machine learning. Several experimental defenses against litigation disorders – are often defeated later with stronger attack strategies. So we focus on Strong worksWhich provides a sporting guarantee that its prediction will remain a constant of $ \ ELL_P $ -norm around inputs.

Traditional accredited durability methods bear a set of defects, including nondterminism, slow implementation, poor expansion, and a testimony against only one attack base. We confirm that these problems can be addressed by improving the durability of the accredited durability to be more compatible with practical rivalry settings.

The problem of asymmetric dependent durability

Current strong works produce certificates for inputs that belong to any category. For many hostile applications in the real world, this is wide, unnecessary. Think about the illustrative state of a person who composes an email for the fraud in hunting while trying to avoid random mail filters. This discount will always try to deceive the random mail filter to believe that their email random mail is Hamid – on the contrary. In other words, The attacker only tries to stimulate false negatives from the work. Similar settings include discovery of malware, the science of reporting fake news, the detection of social media bot, nomination of medical insurance claims, the detection of financial fraud, the disclosure of the hunting site, and many others.

Fig. Practical hostile settings often require accredited durability for only one category.

All of these applications include a binary classification preparation with one Sensitive category The discount tries to avoid (for example, the “unwanted email” category). This stimulates a problem The asymmetric dependent durabilityWhich aims to provide strong predictions specifically for inputs in the sensitive category while maintaining a high clean accuracy for all other inputs. We provide a more formal problem statement in the main text.

Convex feature works

We suggest Features of nervous networks To address the problem of asymmetric durability. This structure consists of a simple lipschitz feature $ {\ varphi: \ mathbb {r}^D \ To \ Mathbb {r}^Q} $ with a nervous entry network (icnn) $ {g: \ mathbb {r}^Q \ To \ Mathb} Icnns impose convex inputs to the output record By non -linear configuration with non -negative weight preservants. Since ICNN Decision Decision Method consists of a convex and complementary group, we add a map of the pre -features $ \ Varphi $ to allow the unequal decision -making areas.

Common features offered the fast account of the accredited half -diameter of the sensitive category for all $ \ ELL_P $ -norms. Using the fact that convex functions are not close to any plane from a shade plane, we can get a radius that is accredited in the intermediate feature space. Then this half of the diameter is spread to the input space by Lipschitznes. The asymmetric setting here is very important, as this structure only produces certificates for the positive toppling category $ G (\ varphi (x))> 0 $.

The half -diameter circuit formula is $ \ ELL_P $ -nORM is especially elegant:

\[r_p(x) = \frac{ \color{blue}{g(\varphi(x))} } { \mathrm{Lip}_p(\varphi) \color{red}{\| \nabla g(\varphi(x)) \| _{p,*}}}.\]

Non -fixed terms can be explained easily: half of the diameter corresponds to proportional to Class confidence Unlike Compromise. We evaluate these certificates through a set of data sets, and achieve $ \ ELL_1 $ competitive and $ \ ELL_2 $ and $ \ ELL _ {\ infty} $ – despite other methods that generally fluctuate on a specific base and require time in size.

Fig. Rentimes averages are calculated on the right more than $ \ ELL_1 $, $ \ ELL_2 $, and $ \ _ \ Infty} $ -radii (note the registry scale).

Our certificates keep any $ \ ELL_P $ -nORM, which is a closed and inevitable model, which only requires one pass and the passing of each input. These are calculated in the arrangement of millimeters again and expanded well with the size of the network. For comparison, it usually takes several seconds to believe even small networks. Random homogeneity methods are also not specified in nature, with certificates that bear a high probability.

Theoretical promise

While the preliminary results are promising, our theoretical work indicates that there are great and unarmed capabilities in ICNNS, even without a feature map. Although the dual ICNNS is limited to learning convex decision-making areas, we prove that there is the ICNN that achieves perfect training accuracy on the CIFAR-10 Cats-VS-Dogs Data.

fact. There is an ideal training engine workbook for the Cats-Versus-Dogs Data Group.

However, our structure achieves the accuracy of training of only $ 73.4 without a feature map. Although training performance does not mean generalizing the test set, this result indicates that ICNNS is at least able in theory to achieve the modern automated learning model of overcoming the training data set. Thus we put the next open problem for this field.

An open problem. Learn about the CONVEX input compromise that achieves perfect training accuracy for the CIFAR-10 Cats-Versus-Dogs.

conclusion

We hope to inspire the new asymmetric durability framework that can be believed in this most concentrated setting. Our feature of features is one of this architecture and provides a fast and inevitable radius for any $ \ ELL_P $ -norm. We also present the open problem of overcoming the CIFAR-10 Cats VS DOGS training set with ICNN, which we appear in theory.

This post depends on the following paper:

The non -asymmetric durability across the distinctive nervous networks

Samuel in FarrometerBrendon J. AndersonJulian Pitt, Somay Satiqi,

The thirty -seventh conference of neurological information processing systems (Neups 2023).

More details available on Arxiv and GitHub. If our paper inspires your work, please think about it with:

@inproceedings{
    pfrommer2023asymmetric,
    title={Asymmetric Certified Robustness via Feature-Convex Neural Networks},
    author={Samuel Pfrommer and Brendon G. Anderson and Julien Piet and Somayeh Sojoudi},
    booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
    year={2023}
}