PrivacyProtectionPerformanceofDe-identifiedFaceImageswithandwithoutBackground

ZongjiSun,LiMeng,AladdinAriyaeeinia

UniversityofHertfordshire,UK

Xiaodong Duan,Zheng-HuaTan

AalborgUniversity,Denmark

WhatisDe-identification

2

� De-identification:“generaltermforanyprocessofremovingtheassociationbetweenasetofidentifyingdataandthedatasubject.”(ISO/TS25237-2008)

� De-identificationinvolvesminimallydistortingthedatasothatmeaningfulanalyticscanstillbeperformedonit.[1]

� De-identificationnotonlyremovespersonallyidentifiableinformation(PII)orsensitivepersonalinformation(SPI)butalsoretainsdatautilityor intelligibility.

v K.ElEmam andL.Arbuckle,AnonymizingHealthData:CaseStudiesandMethodstoGetYouStarted.O’ReillyMedia,Inc.,2013.

TargetsofOurFaceDe-idResearch

�RemovePersonallyIdentifiableInformationfromafacedataset�Keepstatisticalcharacteristicsasmuchaspossible

�Meaningfulanalyticscanstillbeperformedonit�Returnclearnaturallookingfaces

� Acceptedbymostpeople

TheseDe-identifiedfaceimagescouldsupportfacerelatedresearchwithoutleakingtheidentityinformation.

3

𝑊"𝑝$,… ,𝑊"𝑝',𝜆$,… , … , … , … ,… , 𝜆)

FaceRepresentation

4

𝐴 = 𝐴, + Σ/0$) 𝜆/𝐴/𝑠 = 𝑠, + Σ/0$' 𝑝/𝑠/

Statisticalmodel(AppearanceModel[2])

Shape(68landmarks) Texture(RGB)

v T.F.Cootes,C.J.Taylor,andothers,“Statisticalmodelsofappearanceforcomputervision,”Technicalreport,UniversityofManchester,2004.

FaceRepresentation(cont.)

� Trainthefacemodelusing1952FERETfacesthatarewithoutglasses,beardormoustache

� Thefacemodelignorestheglasses,beard andmoustache ontheface

5

Original Reconstructed Original Reconstructed

(a) (b)

De-identificationinFeatureSpace[3]

6

v Z.Sun,L.Meng,andA.Ariyaeeinia,“Distinguishablede-identifiedfaces,”in201511thIEEEInternationalConferenceandWorkshopsonAutomaticFaceandGestureRecognition(FG),2015,vol.04.

BackgroundDeformation andBlending

7

De-identified face

Originalimage

Backgrounddeformation [4] Face&backgroundBlending [5]

o Original shape× De-id shape

v S.Schaefer,T.McPhail,andJ.Warren,“Imagedeformationusingmovingleastsquares,”ACMTrans.Graph.,vol.25,no.3,p.533,Jul.2006.v P.Pérez,M.Gangnet,andA.Blake,“Poissonimageediting,”ACMTrans.Graph.,vol.22,no.3,p.313,Jul.2003.

Examplesofde-identifiedfacesO

rigi

nal

De-

id fa

ce

with

out B

G

De-

id fa

ce w

ith

BG

8

Re-identificationTest

� Threetypesofattacks:

9

Gallery Probe

Naïverecognition[6] Original De-identified

Reverserecognition[6] De-identified Original

Background-onlyrecognition

Background oforiginalimage

Background ofde-identifiedimage

v E.M.Newton,L.Sweeney,andB.Malin,“Preservingprivacybyde-identifyingfaceimages,”IEEE Trans.Knowl.DataEng.,vol.17,no.2,pp.232–243,Feb.2005.

Re-identificationTest(Gallery&Probe) id: 00043 id: 00050 id: 00048 id: 00130 id: 00155

Ori

gina

l fa

Ori

gina

l fb

De-

id fb

10

Gallery

Probe1&2

Re-identificationTest(Design)

� 963subjects from FERET dataset� Twogroupsoffaceimages(withbackgroundandwithoutbackground)

� Twodifferent facecroppingsizes� Fourdifferentfacerecognitionmethods

11

Original fa Original fb De-id fb

w/ B

G

w/o

BG

Re-identificationTest(Settings)

12

Feature Parameter values Distance measurement

PCA − Euclidean distance

LBP 𝑟𝑎𝑑𝑖𝑢𝑠 = 1,𝑛𝑒𝑖𝑔ℎ𝑏𝑜𝑢𝑟𝑠 = 8 Chi-squared distance

HOG 𝑐𝑒𝑙𝑙 = 10×10,𝑜𝑟𝑖𝑒𝑛𝑡𝑎𝑡𝑖𝑜𝑛𝑠 = 16 Cosine distance

LPQ 𝑐𝑒𝑙𝑙 = 10×10 Cosine distance

NaïveRecognition

13

Without background

With background

Without background

With background

Without background

With background

200×

200

Original fa Original fb De-id fb

Method Without background With background

200 × 200 200 × 200 Original De-id Original De-id

PCA 47.25 0.13 ± 0.16 54.83 4.10 ± 0.58 LBP 74.25 0.11 ± 0.10 83.39 1.30 ± 0.24 HOG 47.14 0.21 ± 0.14 74.87 6.09 ± 0.48 LPQ 53.27 0.25 ± 0.18 80.48 4.42 ± 0.61

Gallery Probe1&2

ReverseRecognition

14

Without background

With background

Without background

With background

Without background

With background

200×

200

Original fa Original fb De-id fb

Method Without background With background

200 × 200 200 × 200 Original De-id Original De-id

PCA 45.38 0.33 ± 0.19 53.48 9.41 ± 0.53 LBP 72.59 0.23 ± 0.14 80.48 1.59 ± 0.41 HOG 46.52 0.24 ± 0.16 74.77 5.92 ± 0.70 LPQ 50.88 0.23 ± 0.15 80.06 4.35 ± 0.58

Gallery1&2Probe

NaïveRecognition(cont.)

15

Without background

With background

Without background

With background

Without background

With background

300×

300

Original fa Original fb De-id fb

Method Without background With background

300 × 300 300 × 300 Original De-id Original De-id

PCA 42.37 0.07 ± 0.09 61.27 39.13 ± 0.55 LBP 63.03 0.13 ± 0.11 87.23 55.12 ± 0.56 HOG 18.38 0.17 ± 0.20 78.09 56.93 ± 0.70 LPQ 47.04 0.25 ± 0.11 82.66 59.14 ± 0.81

Gallery Probe1&2

Without background

With background

Without background

With background

Without background

With background

300×

300

Original fa Original fb De-id fb

Method Without background With background

300 × 300 300 × 300 Original De-id Original De-id

PCA 41.53 0.30 ± 0.21 61.37 49.06 ± 0.61 LBP 63.14 0.21 ± 0.11 86.60 59.28 ± 1.63 HOG 17.03 0.07 ± 0.09 76.95 59.81 ± 0.68 LPQ 46.31 0.20 ± 0.14 82.87 58.36 ± 0.48

ReverseRecognition(cont.)

16

Gallery1&2Probe

Potentialattack

id: 00049 id: 00002 id: 00130 id: 00155

Ori

gina

l fa

Ori

gina

l fb

De-

id fb

Method 300 × 300 inverse crop Orig De-id

PCA 56.39% 31.82 ± 0.89% LBP 78.19% 55.62 ± 0.68% HOG 53.27% 27.07 ± 1.09% LPQ 60.44% 32.88 ± 1.06%

• Inversecropbasedontheirfaciallandmarks• agenericattacktoanyfacede-identificationmethod,whichmodifiesthefaceregiononly.

17

Conclusions

� k-Same/Diff-furthestfacede-identificationmethodhashighprivacyprotectionperformancewithinthefaceregion.

� However,whenthede-identifiedfaceregionismergedwiththeoriginalimage,theriskofre-identificationmaysignificantlyincrease.

� Thefaceregionissufficientbutnotnecessaryforidentifyingaperson.

� Toprotecttheprivacyoftheindividualscapturedinanimage/video,de-identificationmustbeappliedtonotonlythefaceregionbutalsoalltheimageregionsthatcontainpersonallyidentifiableinformation.

18

References

1) K.ElEmam andL.Arbuckle,AnonymizingHealthData:CaseStudiesandMethodstoGetYouStarted.O’ReillyMedia,Inc.,2013.

2) T.F.Cootes,C.J.Taylor,andothers,“Statisticalmodelsofappearanceforcomputervision,”Technicalreport,UniversityofManchester,2004.

3) Z.Sun,L.Meng,andA.Ariyaeeinia,“Distinguishable de-identified faces,”in201511thIEEEInternationalConferenceandWorkshopsonAutomaticFaceandGestureRecognition(FG),2015,vol.04,pp.1–6.

4) S.Schaefer,T.McPhail,andJ.Warren,“Imagedeformationusingmovingleastsquares,”ACMTrans.Graph.,vol.25,no.3,p.533,Jul.2006.

5) P.Pérez,M.Gangnet,andA.Blake,“Poissonimageediting,”ACMTrans.Graph.,vol.22,no.3,p.313,Jul.2003.

6) E.M.Newton,L.Sweeney,andB.Malin,“Preservingprivacybyde-identifyingfaceimages,”IEEETrans.Knowl.DataEng.,vol.17,no.2,pp.232–243,Feb.2005.

19

Thankyou!

20

21

Questions?