meetup - retour sur la drupalcon dublin 2016
TRANSCRIPT
DrupalConDublin
2016Capgemini - Digital Factory - Nantes
Yann Jajkiewicz@yjajkiewhttps://yann.me
Jérémy Greffin@jeremygreffinGithub : jgreffin
DrupalCon Dublin 2016 Introduction
3 joursAvec jusqu’à 11 conférences en parallèle
10 667Tasses de café
1800Personnes sur les 5 jours
Timeline
DevOps Drupal Echange
DevOps Docker
From socketwenchhttps://goo.gl/RFJ6Un
Why Docker ?
› Containers› Sandboxes everything› Instructions aren’t enough
Docker vs Vagrant :› Vagrant = multiple VM› Docker = 1 VM for multiple containers
Physical hardware
Host OS
Hyperviseur
VM OS VM OS VM OS VM OS
Server A
Server B
Server C
Server D
Vagrant :
Physical hardware
Host OS
Container A
Docker :
Docker utilities
Linux Kernel
Docker engine
Container B
Container C
Hyperviseur
How to Docker ?
Docker files› Construction d’une image› Description de l’environnement en YAML› 1 fichier par processus
Docker compose› Automatisation du déploiement des images› Définition de l’architecture› 1 commande pour tout installer
Docker & Drupal
Go shopping...› Docker4Drupal
› Redis› Memcached› MySQL, MariaDB› D7 & D8› . . .
… or build your own
DevOps Kubernetes
From tpryanhttps://goo.gl/3k23PK
ProblèmeGestion complexe des VMs et des containers
Kubernetes
SolutionKubernetes gère “magically” différents containers au sein de différentes VMs
Kubernetes : késako
› Open Source› Started by Google› Système d’orchestration de containers
Paradigme :› Build scripts VS Desired state› Children VS Employees
Kubernetes : paradigme
Employee“We had a tough day, go home and get some sleep”
ChildGo upstairsGet undressedPut on pajamasBrush your teeth
Kubernetes : how it works
› Docker files› Pods (1 or + containers), YAML style› Pods controller :
› Réplication, déploiement, service› Gestion des environnements (prod, test)› Autoscaling› Logging, monitoring, web interfacing, cluster
Dockercompose
Docker Machine
Docker
Kubernetes
Multiple containers on
same localhost
Cluster of container hosts
replication orchestration scheduling
Manage Remote
Container Hosts
Launch Container hosts in several clouds
Everything at Google runs on Containers
› Gmail, Web Search, Maps,...› MapReduce, batch,...› GFS, Colossus,...› Google Platform (VMs run on containers
underneath everything)
DevOps Jenkins 2 Pipeline From aroq & shumushin
https://goo.gl/gahNsm
Why Jenkins ?› Release process :
› Frequent› Automated› Repeatable› Lowrisk
› Build, deploy, test, release› Continuous delivery› Continuous deployment
Jenkins : Pipeline› Plugin› Jenkins 1 Workflow → Jenkins 2 Pipeline› Can survive restarts› Pausable (for human input)› BlueOcean Pugin → better GUI› Jenkinsfile : container for the pipeline› Multibranch pipelines : automatic
pipeline creation for branches
Jenkins : pratiques Drupal
› Ne pas commit “vendor”› Pas de composer install ou update en PROD
Solutions :› Build, tarball, sync (doesn’t work with Acquia)› Target repo : commit > CI > Build > commit >
deploy
Jenkins : workflow
Git commit & push
Jenkins Pipeline
Gitlab webhookInit Config
BuildStage prepareStage deployStage testing
Prod prepare Prod deploy Prod testing Finalize
Jenkins : tools
Jenkins : tools
› Docman - one or more source repos + docman config > docman > target repo
› Druflow - deploy drupal-procedures › Drush - especially drush aliases› Behat - testing framework› Pipeline - to orchestrate › Pipeline libraries - make it DRY
Drupal Performances
Redis or Memcache
Web Server
Web Server
Web Server
Web Server
Bottleneck
Traditional caching implementation
By David Strauss
LCache
It combines:› A well established architectural patterns
from modern CPU architecture
From platform.shhttps://goo.gl/1mdkVT
Core 0
Existing solutions: Multi-Core Processors
CPU
Core 1
CPU
L1 cache L1 cache
L2 cache
LCache
It combines:› A well established architectural patterns
from modern CPU architecture› Efficient data propagation from mysql’s
row-level replication
From platform.shhttps://goo.gl/1mdkVT
› Experience on Valhalla to provide an efficient website object caching
Better› Frequently read› Rarely written› Large
LCache
Worse› Read once or not at all› Things handleable
earlier in the stack› Keys updated often› Clearing 100+ keys with
a tag (because of replication)
From platform.shhttps://goo.gl/1mdkVT
D8 : practical caching
3 méthodes de cache :› Cache anonyme› Reverse proxy caching› Cache authentifié
From platform.shhttps://goo.gl/1mdkVT
D8 : practical caching
› Cache anonyme:› Module : Internal page cache› Activé par défaut dans D8› Page cache middleware› Default request policy : no CLI, no session› Default response policy : no kill switch, no
“no_cache” route, no server errorsFrom platform.shhttps://goo.gl/1mdkVT
› Cache anonyme :
› Kill switch\Drupal::service(‘ page_cache_kill_switch ’)->trigger();
› No_cache routesystem.cron:
path: ‘cron/{key}’
defaults:
_controller: ‘\Drupal\system\CronController::run’
options:
no_cache: TRUEFrom platform.shhttps://goo.gl/1mdkVT
› Reverse proxy caching :
› Varnish, CDN, ...› Important : définir le “page maximum age”› Définir le reverse proxy dans settings.php
$settings[‘reverse_proxy’] = TRUE;
$settings[‘reverse_proxy_addresses’] = array(‘127.0.0.1’, …);
$settings[‘reverse_proxy_header’] = ‘X_CLUSTER_CLIENT_IP;
From platform.shhttps://goo.gl/1mdkVT
› Cache authentifié :
› Username block depends on user logged in› Admin links depends on permissions› Help block depends on URL
› Solution : cache contexts› contexts => [‘user’]› contexts => [‘user.permissions’]› contexts => [‘url’]
From platform.shhttps://goo.gl/1mdkVT
› Cache authentifié :
› How to invalidate ?› Invalidate dependencies ?
› Solution : cache tags› tags => [‘node_list’]› tags => [‘user:1]› tags => [‘node:1’]
From platform.shhttps://goo.gl/1mdkVT
› Cache authentifié :› How to decide to cache ?
From platform.shhttps://goo.gl/1mdkVT
› Cache authentifié :› Exemple :
› Metadata :
From platform.shhttps://goo.gl/1mdkVT
D8 : practical caching
› Cacheability debugging :› Module renderviz : permet de voir les
metadata dans l’inspecteur du navigateur
From platform.shhttps://goo.gl/1mdkVT
Drupal ElasticSearch
Drupal & search
› Core module for search
› Last significant update on Drupal 4.7
Search API› Abstraction of SolR› Configuration› Hard to customize
Search
› Fast› Text analysis› Facets
Exemples : Wikipedia, Amazon, Github, ...
› Open source project› Developer-Friendly RESTful API
How to use :- Get it- Unzip it- Run it
› 1 exemple : Save requests -> Percolator
Index content VS search query
› How to start ?
2 ways :➔ Combine with search API➔ Custom connectors with . . .
◆ Friendly API◆ Custom code
https://www.drupal.org/project/elasticsearch_helper
Drupal Bonnes pratiques
Configuration management
› Export/import configuration› Configuration installer : installation profile, UI› Override local configuration : settings.php› Drush can ignore configurations about modules› Drush_cmi_tools : use a YAML list of ignored
configurations› Configuration split : blacklist, ignore set, entities
From pescetti, ademarco, bircherhttps://goo.gl/MSjK3T
Configuration management
› Use Git as parallel development› Careful about merge, workflow› Safe : export, commit, merge, import, push
› Features for D8 : › analyzes and automatically packages up site
configuration into a set of features› Only for development purposes From pescetti, ademarco, bircher
https://goo.gl/MSjK3T
Configuration management
› Client messing with production configuration› Lock configuration on production (settings.php)› Export from prod and merge to dev› Configuration split : choose what to keep
› Deploy content› Use migrate on events› Default_content module : any module
provides hal+json for entities› Deploy module : stage and preview content
From pescetti, ademarco, bircherhttps://goo.gl/MSjK3T
Configuration management : modules
› Config_installer : install site from config› Config_readonly : lock conf via UI› Config_update : diff between original & active› Config_devel : helps developing config› Config_split : import/export filtered config› Config_tools : automatically commit config
changes to git› Features : bundle config for re-use› Config_sync : safely import config from
updated modules From pescetti, ademarco, bircherhttps://goo.gl/MSjK3T
Drupal Migration
Migrate with D8
1. Requirements2. Anatomy of a migration 3. Migration Framework 4. Performance tips
1. REQUIREMENTS
1. Migrate is in core !
Tree new modules› Migrate
Handle migration. Framework.› Migrate Drupal
Contains migration from D6 & D7› Migrate Drupal UI
The older migrate UI (new in 8.1.x)
1. Migrate is in core !
How to execute migrations ?
› UI ?› Drush command ?
Contrib to Migrate tools & Migrate Plus
2. ANATOMY OF A MIGRATION
2. Workflow of a migration
DestinationProcess
Process
Process
Source
DEFINITIONSYaml Files. Custom files
2. In files
PLUGINSPHP Files Core or custom files Types:
› Source › Process › Destination › Builder › ID Map
An easy example
DEFINITION config/install/migrate_plus.migration.article_node.yml
PLUGINSsrc/Plugin/migrate/source/ArticleNode.php
EXECUTIONOnly with Drush8 (8.1-dev) and migrate_tools enabled
3. PLUGINS
3. SourceHere we tell SqlBase: - Which Database is the Source
3. Source- And which Plugin will make the Query
3. DestinationHow and where to store the data
- entity:<place-here-an-entity>
Need more destination plugins? ⇒ Search for “destination:” in core
3. ID MAPPINGHow Migrate associates old rows with new rows
3. PROCESSHow we transform each field, each file or data.
You are able to: › Map fields: Same value as origin. › Modify: Change or process the value. › Add: Create new fields from other fields or calculate
these fields.
3. PROCESSMap fields. Values are equal in both sides
3. PROCESSDefaultValue. Add a default value
3. PROCESSCallable. Values are related but a process is needed with a function
3. PERFORMANCE TIPS
4. Performance tips
1. Hardware improvements (ssd, cpu,..)2. Disabling functionalities (hook, modules)3. Incremental migration (hash)4. Multithreaded migrations
4. Performance tips
$ drush mi cm_articles_1_400 && \ drush mi cm_articles_401_800 &&
D8 : Bonnes pratiques en vrac
› Drush is 10x faster than composer› Librairie “composer-patches” : appliquer des
patchs via composer› Pre-commit hook : PHP Lint check› PHPCBF : PHP Code Beautifier & Formatter› Kint : debug in twig› Smacks: organize CSS & JS› Templating : set var in twig (no preprocess)
Bonus D7 : drush unsuck
MERCI !Questions ?