ffctn hackons la-corruption

51
ffunction inc. © FFUNCTION INC, 2011 ETUDE DE CAS VISUALISER DES DONNÉES OUVERTES Sébastien Pierre, FFunction inc. @Hackons la Corruption., Novembre 2012 www.ffctn.com

Upload: montrealouvert

Post on 07-May-2015

670 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

ETUDE DE CAS

VISUALISER DESDONNÉES OUVERTES

Sébastien Pierre, FFunction inc.@Hackons la Corruption., Novembre 2012

www.ffctn.com

Page 2: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

INFOGRAPHIC : SE7EN SUMMITS

Page 3: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

GOOGE DATAVIZ CHALLENGE 2010 (FINALIST)

Page 4: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

NATIONAL GEOGRAPHIC SOCIETY'S PROJECTS

Page 5: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

2008Canadian Federal

Travel & HospitalityExpenses

Page 6: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

SOME THINGS HAVEN'T CHANGED SINCE 2008

➔ SCRAPING DATAin the absence of open-data, journalists will often be in the same context, having to spend time to collect, explore and assess the quality of the data.

Page 7: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

SOME THINGS HAVEN'T CHANGED SINCE 2008

➔ FROM DATA TO STORYEach dataset is a discovery, getting a (compelling) story out of it is still a major challenge.

Page 8: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

THE DATA

Page 9: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

As the result of a federal government directive*, Travel and Hospitality Expenses have been published on the web in Canada since 2004

* Called “proactive disclosure”

Page 10: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

http://www.tbs-sct.gc.ca/pd-dp/gr-rg/index-eng.asp

Page 11: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Data is (still) not directly accessible, and hosted on each specific ministry website, in a specific format.

Page 12: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Page 13: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

NON-OPEN DATA

ACCURACY PROBLEMS

DATA MAY BE MISSING

DATA NOT UP TO DATE

Page 14: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

22Mb SQL filescraped by citizens(available on Github)

Page 15: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

A DATASET WHICH TURNS OUT TO BE A BIT OPAQUE...

Page 16: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

BUILDING A TOOL TO EXPLORE THE DATA

Page 17: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Basic analysis of the data

Page 18: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Thinking about how to represent the data

Page 19: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Thinking about the flow of interaction

Page 20: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Importing and visualizing the data

Page 21: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Mapping out the different types of expenses (travel, hospitality & guidelines)

Page 22: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Simplifying the representation (expenses vs guidelines, over guidelines is in red)

Page 23: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Changing the focus (under/over guidelines instead of total spending)

Page 24: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Adding guides to improve reading the information

Page 25: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Adding filtering to narrow down to subsets of the data

Page 26: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Trying alternative representations on the data

Page 27: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Trying even more alternative representations on the data

Page 28: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

THE RESULThttp://ffctn.com/a/expensevisualizer

Page 29: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Page 30: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Page 31: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

I just found out the 5 top spendingFederal depts, check it out at http://ur1.ca/a3spt

Page 32: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Page 33: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Page 34: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

1

FINDINGS

Page 35: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

TRENDS ONLY BECOME APPARENTWITH THE PROPER MODE OF REPRESENTATION

Cumulative spending

Monthly spending

Page 36: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

PROBLEMS IN THE DATA QUALITYBECOME VISIBLE

Page 37: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

Spending of ministers for all departments

THINGS YOU WOULD EXPECTARE NOT NECESSARILY THERE

Page 38: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

DATA TO STORY: CHALLENGES

➔ NON-OPEN DATA– Missing or incomplete data: is the problem in the

scraper or in the actual data?– At least you now have a tool to assess (and improve)

the data quality

Page 39: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

DATA TO STORY: CHALLENGES

➔ NOT WHAT I THOUGHT– You might expect something about the data,

but the visualization might prove your wrong– You might have been looking for something specific

but you cannot see it in the visualization

See my “30 min of data visualization”workshop for more on this...

Page 40: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

DATA TO STORY: CHALLENGES

➔ DID I TRY HARD ENOUGH?– There's no secret: you'll find something interesting if

you explore your data enough.– If everything fails, you can at least get fun facts or

controversial examples out of it.

Page 41: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

HOSPITALITY EXPENSES SKYROCKET IN 2008 !!

Page 42: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

2.5xAs much

INDUSTRY CANADA'S BIG SPENDER

MINISTER DIRECTOR

Page 43: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

WAR IS COSTING CANADA AN ARM AND A LEG!

3 MILLIONS!(over a period of five years)

Page 44: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

WE NEED OPEN DATA!

THIS IS NOT AN ACCEPTABLE PROACTIVE DISCLOSURE!

THE BEST STORY IS:

Page 45: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

1

TOOLS

Page 46: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

https://github.com/OpenRefineGOOGLE REFINE

Page 47: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

http://vis.stanford.edu/wrangler/DATA WRANGLER

Page 48: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

http://datawrapper.de/DATA WRAPPER

Page 49: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

http://www-958.ibm.com/software/data/cognos/manyeyes/MANY EYES

Page 50: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

R17 http://www.rseventeen.com/

Page 51: Ffctn hackons la-corruption

ffunctioninc.

© FFUNCTION INC, 2011

THANKYOU!

[email protected] / @ffunction

WWW.FFCTN.COM