PrivacyProtectionPerformanceofDe-identifiedFaceImageswithandwithoutBackground
ZongjiSun,LiMeng,AladdinAriyaeeinia
UniversityofHertfordshire,UK
Xiaodong Duan,Zheng-HuaTan
AalborgUniversity,Denmark
WhatisDe-identification
2
� De-identification:“generaltermforanyprocessofremovingtheassociationbetweenasetofidentifyingdataandthedatasubject.”(ISO/TS25237-2008)
� De-identificationinvolvesminimallydistortingthedatasothatmeaningfulanalyticscanstillbeperformedonit.[1]
� De-identificationnotonlyremovespersonallyidentifiableinformation(PII)orsensitivepersonalinformation(SPI)butalsoretainsdatautilityor intelligibility.
v K.ElEmam andL.Arbuckle,AnonymizingHealthData:CaseStudiesandMethodstoGetYouStarted.O’ReillyMedia,Inc.,2013.
TargetsofOurFaceDe-idResearch
�RemovePersonallyIdentifiableInformationfromafacedataset�Keepstatisticalcharacteristicsasmuchaspossible
�Meaningfulanalyticscanstillbeperformedonit�Returnclearnaturallookingfaces
� Acceptedbymostpeople
TheseDe-identifiedfaceimagescouldsupportfacerelatedresearchwithoutleakingtheidentityinformation.
3
𝑊"𝑝$,… ,𝑊"𝑝',𝜆$,… , … , … , … ,… , 𝜆)
FaceRepresentation
4
𝐴 = 𝐴, + Σ/0$) 𝜆/𝐴/𝑠 = 𝑠, + Σ/0$' 𝑝/𝑠/
Statisticalmodel(AppearanceModel[2])
Shape(68landmarks) Texture(RGB)
v T.F.Cootes,C.J.Taylor,andothers,“Statisticalmodelsofappearanceforcomputervision,”Technicalreport,UniversityofManchester,2004.
FaceRepresentation(cont.)
� Trainthefacemodelusing1952FERETfacesthatarewithoutglasses,beardormoustache
� Thefacemodelignorestheglasses,beard andmoustache ontheface
5
Original Reconstructed Original Reconstructed
(a) (b)
De-identificationinFeatureSpace[3]
6
v Z.Sun,L.Meng,andA.Ariyaeeinia,“Distinguishablede-identifiedfaces,”in201511thIEEEInternationalConferenceandWorkshopsonAutomaticFaceandGestureRecognition(FG),2015,vol.04.
BackgroundDeformation andBlending
7
De-identified face
Originalimage
Backgrounddeformation [4] Face&backgroundBlending [5]
o Original shape× De-id shape
v S.Schaefer,T.McPhail,andJ.Warren,“Imagedeformationusingmovingleastsquares,”ACMTrans.Graph.,vol.25,no.3,p.533,Jul.2006.v P.Pérez,M.Gangnet,andA.Blake,“Poissonimageediting,”ACMTrans.Graph.,vol.22,no.3,p.313,Jul.2003.
Examplesofde-identifiedfacesO
rigi
nal
De-
id fa
ce
with
out B
G
De-
id fa
ce w
ith
BG
8
Re-identificationTest
� Threetypesofattacks:
9
Gallery Probe
Naïverecognition[6] Original De-identified
Reverserecognition[6] De-identified Original
Background-onlyrecognition
Background oforiginalimage
Background ofde-identifiedimage
v E.M.Newton,L.Sweeney,andB.Malin,“Preservingprivacybyde-identifyingfaceimages,”IEEE Trans.Knowl.DataEng.,vol.17,no.2,pp.232–243,Feb.2005.
Re-identificationTest(Gallery&Probe) id: 00043 id: 00050 id: 00048 id: 00130 id: 00155
Ori
gina
l fa
Ori
gina
l fb
De-
id fb
10
Gallery
Probe1&2
Re-identificationTest(Design)
� 963subjects from FERET dataset� Twogroupsoffaceimages(withbackgroundandwithoutbackground)
� Twodifferent facecroppingsizes� Fourdifferentfacerecognitionmethods
11
Original fa Original fb De-id fb
w/ B
G
w/o
BG
Re-identificationTest(Settings)
12
Feature Parameter values Distance measurement
PCA − Euclidean distance
LBP 𝑟𝑎𝑑𝑖𝑢𝑠 = 1,𝑛𝑒𝑖𝑔ℎ𝑏𝑜𝑢𝑟𝑠 = 8 Chi-squared distance
HOG 𝑐𝑒𝑙𝑙 = 10×10,𝑜𝑟𝑖𝑒𝑛𝑡𝑎𝑡𝑖𝑜𝑛𝑠 = 16 Cosine distance
LPQ 𝑐𝑒𝑙𝑙 = 10×10 Cosine distance
NaïveRecognition
13
Without background
With background
Without background
With background
Without background
With background
200×
200
Original fa Original fb De-id fb
Method Without background With background
200 × 200 200 × 200 Original De-id Original De-id
PCA 47.25 0.13 ± 0.16 54.83 4.10 ± 0.58 LBP 74.25 0.11 ± 0.10 83.39 1.30 ± 0.24 HOG 47.14 0.21 ± 0.14 74.87 6.09 ± 0.48 LPQ 53.27 0.25 ± 0.18 80.48 4.42 ± 0.61
Gallery Probe1&2
ReverseRecognition
14
Without background
With background
Without background
With background
Without background
With background
200×
200
Original fa Original fb De-id fb
Method Without background With background
200 × 200 200 × 200 Original De-id Original De-id
PCA 45.38 0.33 ± 0.19 53.48 9.41 ± 0.53 LBP 72.59 0.23 ± 0.14 80.48 1.59 ± 0.41 HOG 46.52 0.24 ± 0.16 74.77 5.92 ± 0.70 LPQ 50.88 0.23 ± 0.15 80.06 4.35 ± 0.58
Gallery1&2Probe
NaïveRecognition(cont.)
15
Without background
With background
Without background
With background
Without background
With background
300×
300
Original fa Original fb De-id fb
Method Without background With background
300 × 300 300 × 300 Original De-id Original De-id
PCA 42.37 0.07 ± 0.09 61.27 39.13 ± 0.55 LBP 63.03 0.13 ± 0.11 87.23 55.12 ± 0.56 HOG 18.38 0.17 ± 0.20 78.09 56.93 ± 0.70 LPQ 47.04 0.25 ± 0.11 82.66 59.14 ± 0.81
Gallery Probe1&2
Without background
With background
Without background
With background
Without background
With background
300×
300
Original fa Original fb De-id fb
Method Without background With background
300 × 300 300 × 300 Original De-id Original De-id
PCA 41.53 0.30 ± 0.21 61.37 49.06 ± 0.61 LBP 63.14 0.21 ± 0.11 86.60 59.28 ± 1.63 HOG 17.03 0.07 ± 0.09 76.95 59.81 ± 0.68 LPQ 46.31 0.20 ± 0.14 82.87 58.36 ± 0.48
ReverseRecognition(cont.)
16
Gallery1&2Probe
Potentialattack
id: 00049 id: 00002 id: 00130 id: 00155
Ori
gina
l fa
Ori
gina
l fb
De-
id fb
Method 300 × 300 inverse crop Orig De-id
PCA 56.39% 31.82 ± 0.89% LBP 78.19% 55.62 ± 0.68% HOG 53.27% 27.07 ± 1.09% LPQ 60.44% 32.88 ± 1.06%
• Inversecropbasedontheirfaciallandmarks• agenericattacktoanyfacede-identificationmethod,whichmodifiesthefaceregiononly.
17
Conclusions
� k-Same/Diff-furthestfacede-identificationmethodhashighprivacyprotectionperformancewithinthefaceregion.
� However,whenthede-identifiedfaceregionismergedwiththeoriginalimage,theriskofre-identificationmaysignificantlyincrease.
� Thefaceregionissufficientbutnotnecessaryforidentifyingaperson.
� Toprotecttheprivacyoftheindividualscapturedinanimage/video,de-identificationmustbeappliedtonotonlythefaceregionbutalsoalltheimageregionsthatcontainpersonallyidentifiableinformation.
18
References
1) K.ElEmam andL.Arbuckle,AnonymizingHealthData:CaseStudiesandMethodstoGetYouStarted.O’ReillyMedia,Inc.,2013.
2) T.F.Cootes,C.J.Taylor,andothers,“Statisticalmodelsofappearanceforcomputervision,”Technicalreport,UniversityofManchester,2004.
3) Z.Sun,L.Meng,andA.Ariyaeeinia,“Distinguishable de-identified faces,”in201511thIEEEInternationalConferenceandWorkshopsonAutomaticFaceandGestureRecognition(FG),2015,vol.04,pp.1–6.
4) S.Schaefer,T.McPhail,andJ.Warren,“Imagedeformationusingmovingleastsquares,”ACMTrans.Graph.,vol.25,no.3,p.533,Jul.2006.
5) P.Pérez,M.Gangnet,andA.Blake,“Poissonimageediting,”ACMTrans.Graph.,vol.22,no.3,p.313,Jul.2003.
6) E.M.Newton,L.Sweeney,andB.Malin,“Preservingprivacybyde-identifyingfaceimages,”IEEETrans.Knowl.DataEng.,vol.17,no.2,pp.232–243,Feb.2005.
19
Thankyou!
20
21
Questions?