From 40f00b1fa7b5696418c2ce93815a2d34712cb2b5 Mon Sep 17 00:00:00 2001 From: Youcheng Sun Date: Mon, 25 Oct 2021 11:10:37 +0100 Subject: [PATCH] update --- README.md | 72 +++++++++++++++++++++++++------------- images/deepcover-logo.png | Bin 0 -> 8088 bytes 2 files changed, 47 insertions(+), 25 deletions(-) create mode 100644 images/deepcover-logo.png diff --git a/README.md b/README.md index 321d6bf..d2700d4 100644 --- a/README.md +++ b/README.md @@ -1,41 +1,63 @@ -# DeepCover +# DeepCover: Uncover the Truth Behind AI ![Uncover the truth behind AI](images/deepcover.gif) -DeepCover is a tool for [testing](https://dl.acm.org/doi/abs/10.1145/3358233) and -[debugging](https://arxiv.org/abs/1908.02374) Deep Neural Network (DNNs). +![alt text](images/deepcover-logo) -## To start running the Statistical Fault Localization (SFL) based explaining: +DeepCover explains image classifiers using [statistical fault lolization](https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123730392.pdf) and +[causal theory](https://openaccess.thecvf.com/content/ICCV2021/papers/Chockler_Explanations_for_Occluded_Images_ICCV_2021_paper.pdf). + +# Install and Setup +#### Pre-Installed: `Conda 4.7.11` +#### All commands were tested on macOS 10.14.6 and Ubuntu 20.04 +``` +conda create --name deepcover-env python==3.7 +conda activate deepcover-env +conda install opencv matplotlib seaborn +pip install tensorflow==2.3.0 keras==2.4.3 +``` + +# Hello DeepCover ``` -python ./sfl-src/sfl.py --mobilenet-model --inputs data/ --outputs outs +python ./src/deepcover.py --help +usage: deepcover.py [-h] [--model MODEL] [--inputs DIR] [--outputs DIR] + [--measures [...]] [--measure MEASURE] [--mnist-dataset] + [--normalized-input] [--cifar10-dataset] [--grayscale] + [--vgg16-model] [--inception-v3-model] [--xception-model] + [--mobilenet-model] [--attack] [--text-only] + [--input-rows INT] [--input-cols INT] + [--input-channels INT] [--x-verbosity INT] + [--top-classes INT] [--adversarial-ub FLOAT] + [--adversarial-lb FLOAT] [--masking-value INT] + [--testgen-factor FLOAT] [--testgen-size INT] + [--testgen-iterations INT] [--causal] [--wsol FILE] + [--occlusion FILE] ``` -# More options + +## To start running the Statistical Fault Localization (SFL) based explaining: ``` -python ./sfl-src/sfl.py --help -usage: sfl.py [-h] [--model MODEL] [--inputs DIR] [--outputs DIR] - [--measures zoltar, tarantula ... [zoltar, tarantula ... ...]] - [--measure zoltar, tarantula ...] [--mnist-dataset] - [--cifar10-dataset] [--grayscale] [--vgg16-model] - [--inception-v3-model] [--xception-model] [--mobilenet-model] - [--attack] [--text-only] [--input-rows INT] [--input-cols INT] - [--input-channels INT] [--top-classes INT] - [--adversarial-ub FLOAT] [--adversarial-lb FLOAT] - [--adversarial-value FLOAT] [--testgen-factor FLOAT] - [--testgen-size INT] [--testgen-iterations INT] +python ./sfl-src/sfl.py --mobilenet-model --inputs data/panda --outputs outs ``` -# Dependencies -We suggest create an environment using `conda`, `tensorflow>=2.0.0` +## To start running the causal theory based explaining: ``` -conda create --name deepcover -source activate deepcover -conda install keras -conda install opencv -conda install pillow +python ./sfl-src/sfl.py --mobilenet-model --inputs data/panda --outputs outs --causal --testgen-iterations 50 ``` # Publications ``` +@inproceedings{sck2021, + AUTHOR = { Sun, Youcheng + and Chockler, Hana + and Kroening, Daniel }, + TITLE = { Explanations for Occluded Images }, + BOOKTITLE = { International Conference on Computer Vision (ICCV) }, + PUBLISHER = { IEEE }, + PAGES = { 1234--1243 }, + YEAR = { 2021 } +} +``` +``` @inproceedings{schk2020, AUTHOR = { Sun, Youcheng and Chockler, Hana @@ -43,7 +65,7 @@ and Huang, Xiaowei and Kroening, Daniel}, TITLE = {Explaining Image Classifiers using Statistical Fault Localization}, BOOKTITLE = {European Conference on Computer Vision (ECCV)}, -YEAR = { 2020 }, +YEAR = { 2020 } } ``` diff --git a/images/deepcover-logo.png b/images/deepcover-logo.png new file mode 100644 index 0000000000000000000000000000000000000000..117144ab6669f6a3f6a456dd2b9e56719cb20075 GIT binary patch literal 8088 zcmXw8by!qU(^o_RsYR4#NeSt$1?i9yK@?DWVUdz-5xvb zexK@jtlDgy)3%~@VRT=krH>~{H_$2QirN*P4bp&9g|R>yrPh9c z$oeNh{(Es*E0;$_0h+*0BgwO6TNT^`DbXaR+JsY8welfFFY}NAf z)q@6-LfBH2n3i>#m(?+zQq4oakk|Mc#0QUeLdF8$U0>}5Sewg&RQ3gQc7C^zB#Dbom+v>qu+Ra;& zQ&E3ZXkhdmk$^S7y)3rO{0FVs*(@|u?el>3gBksgewgGkT>it`_+fJOq55AS;iHBg zu+MfZL5MqM+xoc_k6|eXk3lhO9Tv5xk$ygsHxwl{mspl=a*avYnDg1|T>&`WbgVE7 z1dm1N!L(GsUY4RQ{>LThA!8+07oSbwvMRW3(Vnim=&`^%r^Yd#Imn#{IzviA${UW$ zigJJas(x1Zu1c_OiGEM@VyD7nOul^5DrNieoKrh}NrUp^O@O>-^=|c+pa4_;^P97N z1EidB$U8|g4VA-JFE&FfVOl0P=i@>;Zc<{$csTIJ8y0@i*6fmD?$3Kl=HQ!%KJHXa zm$QK|nYeg9lQa^U(|@IV^%v-b`or9f5I$eC>-`pfuj3h){_Q*d^A~doG8dEmXux<9 z$j3;0waN!yITeNQCr6^fs84zw{e^aFv3(oB>Rwy}2E-oVBmCrW2{O{+e38efs^h}- z^&SRG`I3RNfQ#w*)qFDZs-ih4W1Vd;6%kW9# zJyH$2nwp(*Bkv|cLnS|a zFlu41*3?+Xq(QTkV@AU)qW9pQL03HSDuSf+F%$2yV0OI9WfpC)re zG+~+!VnZc^?|_le|C>;sILn1;;!9!LDMI-OlH;2wT=0X)bNnNg@&AOKFR~_!u~JJs z*R^ZXeAZZ+Y|HtaWM!0D;)QW?I;g&5dUNy#OS<)Lj52}l?b?ON*f@6W5NyJO$A*19 z6irQb+~Zs(XI&81ph@aSfcK)v*3}_^qD_V-teTiZ)Q)}TBgJfla{62c2gE&K zOUUi9IsfHR$JH>}QZ-YzFhBz3PY<|0lV~WoNOEEYy{*JQd?PL)--!|mqk zq(8`@CY@RZ2Obg$n2@v6N&YX}+}Cb$LGrt79P`ml#m?zp)Ch0$W)glWGkB0jCh|?~ z;uEh!H}OPkoX$UXznF9jn878?G({A*dK4mqDM5-q7S&?9woOGtLbzvRM94l3ij2QQ z;(bnY9a5X5EMKKQ-~#md5T`J_J!QJX6&=rQ#WJcerudJt;Z(`JJ|$v5-pC=MG)QD2 z!UWz(bIVwUp2p+|E%-bQ}GYcU4EAg8wN!O zit3N7!?>}(XHfS@#2M|YR^q*?#~9N><|G%dFPH$0)Yd7LFQaSEM~nK@TMoHAUm;i? zpBS;Lo$o4OEv%AFrZy?lPg~iY4>zCkd8rw@9a8suDvMU+Fa);=74R@_NEAnw8c@_8KetjeRZ)5L0X18y?&e}gAS}J8; zF*P(&of+o`0jlC{oe`YxDW7&oxuO!zu+d<(R8F=ZRQ2@|r`ST)xQ-cELz@Na3cd-Z zS`2VEj9aDH$M=iHYmODaR%2N)rRo*xO4P?SBqzP|^Nm|G0sC?;j3`tX+EyB|z!Vz# zHhv=KI9>`pnrN_}>eAPV9B+Y_%wnWy?MroJ##xWqs#G-aC0~)C3!$x7MC5P-GEyuY zc^0EU(y5#2zvnaGZK%CNN^P|=rfi_W&3R9sbA{j7vF^yZ^dU7ht1Vem3+eMqxZ4B| zFFqcZm8!&VWvQKx!{7gjGt>Yz1ar)nr#|V})HO?F*JRU8c z2`SF9hxs#uC)paR^JnEXdrHiy-Kni(Buf=KzYsoGKnP5xq!Wh7i`T`Z7B0OaG0g`5 zzfj}|^HaXyuRQDQKOx`owFY6Xp2-g$@!!1y?%v;0=asH)7cU^ar1}0V@c;Yd%`!d` zJN?;~Kcy$dP7NAuelo5d^z6psv%v7_C;9(+^gS|CVii~tHKvKw|Dz>YYS!zYhGow# zjEENag%;hX?Gq^Yj8oT$1fN|>?|EsbwFOPK5IE$1@o&{>S-SpG^QydTK4fJ2l0lA1 z*35l1|2!0Mwnh!y2h3p9OY3+InBCCT$^86xi@sXl8tHJ^M5Dabe=9Bm1?NM#>Mjms z&T@}aYn@>-K>A*bIl-)S3&{z#Oo3Dih?`9vl61p@l9EP472mZCMC??|%e;4=gbGO3 z-JzM5&44$`wq_HpMg}+cip@hjPR|ap9JHBqPOW4%onN1N$-CKpb6FLvfSIf|RqZzu zk{3@x6W$LML%Y9mHsFU|B*~)mE>y6gU++?so%ZTZtOCfNlN91IeQ)RAY%buI zR7QTKo3Nq^cCriaI%M)SxxH8bPJu_Lw2uucCEg+|2)dw{8`ROjs`I)4kE z>t$$9Zh)&PsWJ&O#oG8svzzPukj8UX5wo`s`o>j6W?IWo1d)Gmw zjii2GD=oe4F)Emtv&S;UV081J8RCKo2K*dh>%|3EX#jy^E7>ekr%vVA2t1c9V(T@< zW`(8x)F@Rfi%=Vz`Hq*38%up-Ibq+ZQhMJ`3zr~Ah)K8qu$0d(c5AdDU;fbb<^)Eh zKb#PfnrPPMLGQcYMBkqzst{D&0box)NbXBS{_ezh(?l;bNupp+y%0dV-u=)zLH=9 z)>@g{;fa0#)FYW>2({lti^9Rp|9Tw;)7Ac>9<+Zk*Ao2ErD&o2DM1)C{I)@~0t*z$X*+=Yh4ebdUmkym3E(a&F90$WQZ_Qr|716Ra zHrRZ#M*pTzwoW{%P|xXT^v&9V6`Q!wztLsPE&OybXJz)=bxBW&etvxlISS!kMQY1< zMagCFWFmy_0l25J7DOt3=I#JMgw*jCO6GR!cl5u*!$eu|Y=VeIRki&w$dci=uDhJt zUN79^yE{ZNGpSvwo7G#ty4B~y?KG$96bUP&T2Y9Ha4*0@2H|I4)p+M4DS9|UotuluI9z7 z)8DCVY8;U#+k$64XWqi87^U9jXDJ?{w^;* zjOou)30}6(=if*l4gSl$v%i*j8HuFcumcJv34x*H1LXx;#_!~ZEMVx*l0Hf4=i1&w zNM~4@f`4YD#4l*r@Uxl)HaPZp+RS z2mko%ybul?(51BqxURlM)Di=t;yhTt3}4+Y>F+bMbrjM4d^}9G`tS!P$%k$_RP5z! z6t9>{vA`Qz$66=vP2Kcl@Wn(sT15sEyRkqT;ORZl0*+4u;3N9#c$!b!ORhx6V9 z?*s?o4wK{B$*nAQCNjTf3oN7;$q*g3ttVVQ$?8fLS`56aDySR}KQN3E5|9syGnx4e`I2 zdzSaB1UO3$Ny=8) z%i|WWPQDsWXKs|KRZRrkTro68uv8~_x@RH7=?j|HySR#a`b(y7i12w&+7$PmQn+%j zG5>WCeDh@?tB|J4Wk`)Z7r@1kfIas*r`d+Fa6CqQWfcmx6?HNZc||>&$aUh#PszA~ z>n$%C)PnUNKN)=V!jYxqDHv_Nvjr;OW78nIMknWQ?v>X0FRWF?>6mo0-emb5)t}ec z?Q5CizwnzQU{4*`D5buLF}iASh*a~=6wxd>4=6RI@ziCli^nGV)hSZ1yepv)hQ6gT zdRe^TJ6_MGIri*;5f^39J^&NZSdGD}2cWK{L;BhN714Q2txav62)FudsTWtxm|aAB z0E{=6$>q2EqbX(0mn)a6`gscbD9c|^;iA#KirkD8bAycY0Q)Nu{ABmHGN|w{-D=*N zX%DJMtFh*rx9NM(2{o9@kyio$Hfi#D#oFW`wvEy?3?ZNLD=ztNix2aPi}%bgT87?P zy(z)#R~81E9mZ^Fw!f;-NBM_x;dvWa(??^?0i&$+$$?^7ZVj*)H*3>)VJH=JWSnE( zCYV{1t?h)U`?^@9zK}xULzjS&7o-J*`!a!$Z#5zTkEf$pv3IFV4K`G9E*`kedQ{>} z%$LuM4f*3e=qkO&Hu%jJfpstZ01@F=&4tO*RmzWzxRbFBzuJGZ-}F;+Vff&|XvEW^ zveC8&Okh#p+4U5|PkryBZMS=uJ{JFUm1vLO%Z)dS(@EH?5m`r>0ZsBUG~8{{`A;vYPOk8Dg$Dmb!_&_>9Uo?|87|bhiYZ{ zyQK{PP#i37PX*=p`3rA2C#CVHOhuoh%=YgVu=w#|-t#&mvXUuBoa=LMO`-3+J>t#h z@0!AxMFd33sMlb5M*&H%l~(UrD2fsb#<%KzQj*mm`kqtq@CNa+{B8 z4T+|?=r}{LC(O?+h(dsqSt&ziOr89vA6?n73l%h9`#l0p@OslWGJng(7MsBa_R?#K zF(jO{E9v#7m^>WBcYh4$Y36jxs~a_c_{~{E>+O{wjYykYUE9^gTXNEUh$`xxLzISf zrbpB!5cs>k3;B506R4j3$}V}!^1fvm`GG44M7r}`7-qC_M=AJaw*V?IyJk9U)1WIJVW%{dfd%i^{UVC)vo&iO{mLPXjUbOij>|Bk$>CuOx5QW+iqqyJkQ*oG7+k!pY zfwi1k=N(b1@o|Q=JqvA!h-dS-o`HL1Eh30Sl0)jDjZ*j>wO%{|P9t#G41MGo!AbwB zBp`u_+U&m%HyX1OTQ28*F*dH|us6r!Zrts7lkpfkpQKN@ z+mh@spnTx1!5+8KI=;*HS892@^1X;fylBeP4CW=4fgQgmtop}DC{%9oO}l{BU)H31+&nD)OUF!-Q^5`>f6sQ6v?#j zzEUtfr{d%l(qHaA*Vi*j%B0~fdael-O;+QaAG^6^7(CcXbbERBsCPR(ibx)EM4Pym zK{5Dkh)U$t!AjpHV5f#knR-)ia}cdNvib#*%PbsHO-H?BueivxQ{np|@1y2pdtvZ( z(JMT@QuIi@Ad{W>#wI2m=xWxTC6tQtr|Cop%nHP|-cxUtt;yVFW+b*4%u@1-VMlhP zz=uDAh&yO?tFj)2a%}^}h!>8h1f(m4C#g5yGx2q<-1OKW$4RtZ_kn-(u)~f6-~=@z zE#>pmm25$hkp^k{?@4N>R8h~DZrr|FqU3n8#n2SBKo6$V%c5BeLe-ob7PR9zaNL=9hjL{hoR+C)1TV6BCe3%w5YeVWT9TgAulfP|e66A;=nGPb2ENl5b{l3(0QM-ed@C-E^+OKk-vH z*$Nc`6+{*sP*K#7TMp5JoA>d`x%@>gol)kgR^W2CSDDRNkNUG4dXcu3YUFQ}=OQN~ z3y1-iIVrm3>p3?rLN<|_S*f>`E15VwR*U5o+r!p2$~wbhOSR|>l=o(Vm+U%YPPV^& zgo?GT)%jR`jB^p)Z0@}YcR}K=s!i_i!$kK;vx&y@x0uL&r-VNkT@!6rjxZc+hdw!J zu1PYrnNZ}xxNl-Cbv($a#@F1tQ$-Pae#U27HAJB#pxZ$Db2HTp(7$IyB%Stb8AnvOSlph`GHIY-t8!BT3-)Z+;Y#P)<^ znz~V-CMPmULvCC4ehD_3S5XF6OJ8U3TC z=g;ycs!mri_iT(onL4-rNPpIbyA<@JXsq2g%!yqNjN%tiXDv229IcYgeMh~6%1I(z z@wK7S__!j{9ms&JO%U!Rk~scqWQ;8R|Q6(nGaWM+7RZ%pih26AXyHnlls* ztx{~Ro^?lhz6jeWSBklk{A9cFmf3r@tKdY4QmXS^cYe{}o>XrRty)}l=fqgj#kZA8 zOm&h(E(9weLQt(cX^cCI|0fP=`_bm=22Eg;Zd6Z7zQmXoxQS~|NZ`b*n^G6G8U#y@ z5gy&CUO3lMFx^aAQem|h&AOJaLJ8l84(}2PY&>=0%&wI)0*IX(M;{TT;kc;&Fvqi;gvT7a}4ww~BY3k?yN9~15vIrBs> zWDY za=-_0D|0>Zr%(G;O))a4;xIf8!Rz8XAb{o_QV9nE5S1A2`6Gl4!xA}_B5k-Re~&0I z>usgsi-+A~alGdjVLhW)ny(FX5Hw@}cE}r|B@!HXgaTu@^&;3nPaKr!3-DmOp>Ctc z#UT#^ro*k$#rYo@`Oknkvc|R07b$#PHWJ_@5k8hW3$msdV0N2yX-(w;5TpYLlAm;B zHNzX1Z)eSC{rtO(phWRog1fOwA@|^&DgYl$y4j2zBfrxv!5Nmwhj=(h1i)wcc}r!G ztmz_fl%z@(iYFwvYzzRO&VW!SJa}U((|*AHS8gm%GC%wTChm9Cl(ZCo$(siL9|mOI AUjP6A literal 0 HcmV?d00001