module 2 iut bobigny : automatisation de la chaîne de production logicielle
Post on 09-Jul-2015
663 Views
Preview:
DESCRIPTION
TRANSCRIPT
AUTOMATISATION ET INDUSTRIALISATION
DE LA CHAÎNE DE PRODUCTION
Gestion de projet web – Module 2
MODULE 2 : INGÉNIERIE DE LA CHAÎNE DE
PRODUCTION
Intégration continue, usine de développement
Test et qualité logicielle, automatisation des tests
Qualité du code
Déploiements
Petit rappel : dans un contexte d’entreprise…
Projetez-vous !
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
2
OBJECTIFS
Optimiser
Contrôler
Fiabiliser
Accélérer
Pour cela:
Automatiser
If you have to do it more than once, AUTOMATE IT!
Industrialiser
Scripter
Mesurer 3
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
INTÉGRATION CONTINUE
DÉFINITION
Continuous integration (CI) is a set of continuous processes to improve quality control: small pieces of effort, applied frequently.
Continuous integration aims to improve the quality of software and to reduce the time taken to deliver it, by switching from the traditional practice of applying quality control after completing all development to a continuous control process.
The value of continuous integration: Reduce risks
Reduce repetitive processes
Enforce higher software quality
Generate deployable software
Establish greater project confidence
5
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
MISE EN ŒUVRE D’UNE INTÉGRATION
CONTINUE
Set up a continuous integration server to produce nightly builds that: Compiles the project
Generates code reference documentation
Deploys on preprod
Runs automated test
Issues a build report
Automatically track bugs
Tools Jenkins http://jenkins-ci.org/
Atlassian Bamboohttps://www.atlassian.com/software/bamboo/overview
Cruise Control http://cruisecontrol.sourceforge.net/
6
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
UNE USINE DE DÉVELOPPEMENT
7
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
SHORT FEEDBACK LOOP
In addition to continuous integration, set up a build that executes with every commit, and does just 3 things: checkout source code, compile, run unit tests, in a very short time (a few minutes max)
8
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
ENVIRONNEMENTS
Development: Development is done on the local environment of the developer where he implements and runs unit tests.
Integration: Developments are deployed on Integration for integration testing and debug.
Preproduction: Developments are then deployed on Preproduction for performance and load test, security tests, and final acceptance. The Preproduction environment is a copy of the Production environment.
Production: This is the live environment.
Staging: This is a copy of the Production environment, hidden, where data and content updates can be tested before pushing them to Production.
9
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
TESTS ET QUALITÉ LOGICIELLE
TYPOLOGIES DE TESTS
Tests unitaires
Tests d’intégration
Tests de non-
régression
Tests fonctionnels
Tests d’interface ou
IHM
Tests de compatibilité
navigateur
Tests de charge
Tests de performance
Tests de sécurité
11
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
UNIT TESTING
It is the responsibility of developers to properly unit test their code.
A unit test:
Does not communicate with the database
Does not communicate with external resources
Does not manipulate one or multiple files
Can be run in parallel with other unit tests
You do not need to configure the launch of a unit test
Unit tests should be mocked to avoid dependencies.
12
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
TDD: TEST DRIVEN
DEVELOPMENT
La méthode TDD repose sur un des principes Test First d'une methodede développement agile intitulée ExtremePrograming appelé aussi XP.
TDD est une pratique de régulation de la programmation, dont l’objectif est de détecter les erreurs le plus tôt possible.
La démarche TDD consiste à
1. écrire un test et vérifier qu’il échoue
2. écrire le code le plus simple qui fasse passer ce test
3. Refactorer
4. Puis retour au step 1.13
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
FUNCTIONAL AND ACCEPTANCE TESTING
QA team is in charge of Acceptance Testing to validate the new
features.
Those Acceptance tests need to be designed and implemented
with every new feature, then updated if features evolve.
Tools
Fitness http://fitnesse.org/
GreenPepper http://www.greenpeppersoftware.com
14
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
AUTOMATING TESTS
You should automate tests and run them locally and from the continuous integration server: Unit tests
Code review and code analysis
Performance testing
Standard use-case scenario and GUI testing
Load testing
W3C standard testing
Browser compatibility
List tests to automate, specify, script and deploy into the continuous integration server.
Document and don’t forget to maintain tests
Check automation tools Each language has its own set of tools (PHPunit, Junit…)
Selenium http://seleniumhq.org/
Watir http://watir.com/15
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
DEBUG
Key principle:
Debug before coding new stuff!
Debug all bugs as much and early as possible:
• Functional issues
• Technical bugs
• Code review improvements
• Performance and load optimizations
Sometimes some bugs are not worth fixing. Bugs must be reviewed by the Product Owner. Unnecessary bugs must be closed.
Be aware that bugs that are not fixed right away will probably never be fixed.
Do not accumulate a huge pile of bugs.16
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
LOAD TESTING
Validate that your system can support heavy traffic load.
Implement appropriate load-testing tools to emulate a user-case scenario:1. Before building your load test plan, write down
a user workflow and define load criteria for each step.
2. Be careful of the difference between “concurrent players” and “request per second”. 1000 VU / day = 20 concurrent requests (2%)
3. Check load metrics both on the client-side (load-testing tool) but also on the server side.
4. Simulate it
5. If needed, set up distributed load-testing
6. Automate it
Tools: Jmeter http://jakarta.apache.org/jmeter/
17
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
PERFORMANCE TESTING
Performance is key to a good user-experience. It impacts conversion and SEO.
Performance and optimization should be part of any development.
Each new module or web page needs to be performance tested.
Add performance tracking in the logs (could be used for automated testing): both for applications and API to track performance evolution.
Never exceed 1 sec of waiting time, to display a web page or load an application module in normal conditions.
Any loading time above 1 second should display a “loading” indicator or warning message.
Automate performance testing (into the continuous integration).
Minimum grade should be A/A on Page Speed Grade/Y Slow performance scale.
Tools Gtmetrix http://gtmetrix.com/
Webpagetest http://www.webpagetest.org/
Chrome extension for PageSpeed and Yslow 18
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
SECURITY AND FRAUD
Security should be part of the product features. Make better security part of the added value for customers.
Introduce specific security logs in the monitoring (password resets, email changes, login failures…). Graph it to identify spikes of variations and alerts.
Include security tests in your Continuous integration to validate negative flows. Also test your invariants: « this page should always require a login », « this pages should always be SSL ». Establish the baseline and look for deviations from it.Work on preventing false positives.
Switch to 100% SSL. Test SSL : https://www.ssllabs.com
No passwords in plain text.
OWASP https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project
Phishing and safe browsing: http://code.google.com/apis/safebrowsing
http://www.phishtank.com/19
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
STANDARD VALIDATION AND ACCESSIBILITY
Web pages should try and respect W3C standard to avoid browser html interpretation errors and possible performance decrease.
Use http://validator.w3.org/unicorn
Check for broken links: http://validator.w3.org/checklink
Automate W3C standard testing.
Still be careful that this should not be contradictory with performance and should not consume too much time. Even Google is not W3C compliant!
Also try and respect basic accessibility rules. Check http://wave.webaim.org/ 20
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
BROWSER COMPATIBILITY
Because of web browser fragmentation, it is critical to check the display of applications on multiple browsers.
Concentrate on the most widely used: http://www.w3schools.com/browsers/browse
rs_stats.asp
http://gs.statcounter.com/
Not that easy to automate: Set up multiple VM with Selenium scripts
Tools
http://browsershots.org/
https://browserlab.adobe.com
http://www.browserscope.org21
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
QUALITÉ DU CODE
CODING BEST PRACTICES
Use coding guidelines.
These guidelines should be concise and easy to read.
Follow naming conventions.
Code simply and efficiently.
Code should be fully in English and well commented.
Code should be modular.
Always think “performance” and “optimization”
Someone will work on your code one day so make it easy to understand.
Generate automatically a developer reference doc from your code.
Refactor frequently
See:
Google styleguide http://code.google.com/p/google-styleguide/
Github styleguide https://github.com/styleguide
Java styleguide http://www.oracle.com/technetwork/java/codeconv-138413.html 23
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
CODE REVIEW
1. Self-code review:
Automate first level code review
Each developer should run code review tools to make sure
his code is clean.
2. Peer review:
Sit in pairs and cross-review.
Correct and refactor as you review.
Enforce and automate mandatory peer code review
Code Review tools:
Git Pull Requests
https://github.com/features/projects/codereview
Review Board http://www.reviewboard.org/
Other tools integrated for instance into
Jira, Phabricator, TFS…
Use automatic code analysis tools for all languages. Those tools
will be automated with continuous integration.
Measure Code Quality
Sonar http://www.sonarsource.org/
24
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
PAIR PROGRAMMING
To improve code quality, team focus, collective ownership of
code, spread of knwoledge.
Do not do it all the time
Shift pairs frequently
25
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
DÉPLOIEMENT ET RELEASE
MANAGEMENT
DEPLOYMENTS
Deployment should be easy, fast and automatic, on all environments.
You should have a set of automatic tests to ensure the deployment went fine.
Test deployments should be done automatically and regularly through the continuous integration server.
Best practices for deployment on Production A new release should have a version number.
Backup before you deploy.
Run some maintenance and clean up (archive logs…) before deploying
Make sure you remove debug mode or development stuff.
No marketing operation around deployment times (before and after)
Monitor the platform closely after deployment.
Never deploy on Fridays. Try and deploy in the morning or when the traffic is minimal.
27
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
RELEASE MANAGEMENT TIPS
1. Deploy and test internally before sending to end-users.
2. Have a release schedule and stick to it.
3. Hold developers accountable for their code changes.
4. Have developers on hand when code is released.
5. Minimize downtime.
6. Reverting to an old version should always be a last resort.
7. Check real-time data after a release is completed.
8. The release process should go as quickly as possible.
28
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
DEVOPS & CONTINUOUS DEPLOYMENT
« Supporting infrastructure through code. »
« You build it, you run it », Amazon (motto DevOps)
A complete solution that extends from Dev to Ops makes it possible to answer critical
application delivery questions such as:
What is in this release?
Who has tested it?
What tests have passed and failed?
Who approved the release?
Who deployed the release?
Can I recover easily from a failed release?
Thus, DevOps is a blend of processes, tools and cultures between the traditional Dev and Ops
teams, intended to increase collaboration between the teams, reduce complexities and
increase the pace at which the whole organization works together to deliver software
applications.
Go one step ahead: from continuous integration to continuous deployment.
Le processus de déploiement, habituellement jugé délicat, voire risqué, doit devenir un « non-
évènement ».
29
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
UN PEU DE PRATIQUE
EXERCICE 1 : MISE EN PLACE D’UNE
INTÉGRATION CONTINUE
1. Faites un fork du projet game of life dans Github
https://github.com/wakaleo/game-of-life
2. Aller sur Buildhives et lancer un build de game-of-
life
https://buildhive.cloudbees.com
3.Modifier le fichier Cell.java dans
gameoflife-core/src/main/java/com/wakaleo/gameoflife/domain
en mettant un + à la place d’une *
4. Refaites un build en échec
31
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
EXERCICE 2 : REVUE DE CODE
1. Faire une revue de code croisée à l’aide de Git
Pull Request
2. Corriger et faire valider
32
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
EXERCICE 3 : TESTS
1. Faire un test de performance sur des pages du site
avec Gtmetrix ou Wepagetest
http://gtmetrix.com/
http://www.webpagetest.org/
2. Faire un test de standards W3C
http://validator.w3.org/unicorn/
33
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
EXERCICE 4 : TEST D’INTERFACE
1. Scripter un test d’interface avec un outil comme
iMacros for chrome ou Selenium IDE sur Firefox
pour tester toutes les pages de votre site
2. Essayer de scripter le login
34
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
EXERCICE 5 : DÉPLOIEMENT
1. Scripter un déploiement en production
35
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
RÉFÉRENCES
INTÉGRATION CONTINUE
http://www.wakaleo.com/books/jenkins-the-
definitive-guide
http://www.infoq.com/articles/MSBuild-1
Blog Octo : http://blog.octo.com
http://blog.octo.com/vers-une-usine-de-developpement-
2-0
37
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
TESTS UNITAIRES ET TDD
http://googletesting.blogspot.fr/
TDD
http://blog.octo.com/tdd-contre-les-montagnes-russes/
Le livre de référence sur TDD: TDD by Example, Kent Beck
http://www-igm.univ-mlv.fr/~dr/XPOSE2009/TDD/index.html
http://www-igm.univ-
mlv.fr/~dr/XPOSE2009/TDD/pagesHTML/ExempleJAVA.html
http://www.agiledata.org/essays/tdd.html
http://bruno-orsier.developpez.com/tutoriels/TDD/pentaminos/
38
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
LOAD & PERFORMANCE TESTING
http://decrypt.ysance.com/2012/09/tests-de-performances-partie-1-preparation-et-
analyse/
http://www.clever-age.com/veille/blog/webperf-a-quelle-vitesse-ma-page-se-charge-t-
elle.html
http://sixrevisions.com/tools/free-website-speed-testing/
http://sixrevisions.com/web-development/10-ways-to-improve-your-web-page-
performance/
http://perfectionkills.com/profiling-css-for-fun-and-profit-optimization-notes/
http://samsaffron.com/archive/2012/03/23/sam-s-ultimate-web-performance-tools-and-
resources
39
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
SECURITY & FRAUD
http://codeascraft.etsy.com/2012/10/09/scaling-user-security
http://fr.slideshare.net/nickgsuperstar/fraud-engineering
http://blogs.atlassian.com/2012/01/13-steps-to-learn-perfect-security-testing-in-your-org/
http://google-gruyere.appspot.com/
http://www.pentestit.com/xelenium-security-testing-selenium/
https://buildsecurityin.us-cert.gov/bsi/home.html
http://www.clamav.net/lang/en/
40
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
REVUE DE CODE
http://blog.octo.com/sonar-loutil-qui-manquait-a-
lusine-de-developpement-net/
http://www.cellenza.com/publications/la-qualite-des-
developpements-dotnet/
For PHP:
http://codeascraft.etsy.com/2012/08/10/static-
analysis-for-php
41
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
RELEASE MANAGEMENT
http://www.sundoginteractive.com/sunblog/posts/8-lessons-facebook-can-teach-us-
about-release-management
http://agilewarrior.wordpress.com/2011/05/28/how-facebook-pushes-new-code-live/
http://ablogaboutcode.com/2011/05/27/recap-and-thoughts-on-the-facebook-deploy-and-
push-process/
http://www.facebook.com/video/video.php?v=10100259101684977
https://speakerdeck.com/u/ralphbod/p/semi-continuous-delivery-at-new-relic
https://speakerd.s3.amazonaws.com/presentations/5012e0264667b3000200fd4f/SF_Contin
uous_Delivery_Meetup_2012.pdf
42
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
DEVOPS / CONTINUOUS DEPLOYMENT
http://www.clever-age.com/veille/blog/le-mouvement-devops.html
http://blog.octo.com/5-bonnes-raisons-de-deployer-en-continu/
http://searchsoftwarequality.techtarget.com/tip/Extending-Agile-
teams-with-DevOps-techniques
http://docs.media.bitpipe.com/io_10x/io_105847/item_562222/DevOp
s%20wo%20Dev%20DOA-final.pdf
http://fr.slideshare.net/jallspaw/10-deploys-per-day-dev-and-ops-
cooperation-at-flickr
43
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
UN PETIT SITE POUR RETROUVER CE COURS
https://sites.google.com/site/iutbobignyweb/
Support de formation
Liens utiles
Coordonnées
Formulaire d’évaluation
44
Gestio
n d
e p
roje
t Web | IU
T B
obig
ny 2
012
-2013 | F
rédéric
RIV
AIN
-fre
deric
.rivain
@gm
ail.c
om
top related