1 modélisation de linteraction avec objets déformables en temps-réel pour des simulateurs...
TRANSCRIPT
1
Modélisation de l’interaction avec objets déformables en temps-réel pour des simulateurs médicaux
Diego d’AulignacGRAVIR/INRIA Rhone-AlpesFrance
2
Medical Simulators
Motivations danger to patients cost certification
Objectives Geometric Models Physical Models
deformationinteraction
3
Problems
Simulation MUST be real-time! deformation resolution
Simulation MUST be realistic! model identification of parameters
Simulation MUST be interactive! collision detection haptic interaction
4
Plan
Deformation Models Mass-Spring vs. FEM
Real-time Resolution Techniques Static Dynamic
Echographic Simulator parameter identification
Liver Model interactive deformation
5
Deformable Object
GeometryElements
Springs [TW90] Tetrahedra FEM [OH99]
Comparison Realism Speed
6
Geometrical Model
56 surface points108 triangles57 total points120 tetrahedra230 edges
7
Mass-Spring Model
20
20
2
2L
LLE
:Strain
Initial length
Deformed length
8
Finite Element Method (FEM)
displacements
Small strain
Green’s strain
Cauchy Strain:
a
x
Deformation tensor:
Initial configuration Deformed configuration
a x
9
Strain-Stress
Snt
:Traction
EES
: stressLinearE
WS
ij
d
kijkkij
21
Lamé coefficients
force per unit area
Deformation Energy
10
Mass-Spring Model
•Springs are placed along the edges (230)
•Not very realistic: modeling a volume with springs!
•The force of each spring relatively cheap to evaluate
•globally fast
11
Finite Element Method (FEM)
•120 tetrahedra using Green’s strain tensor
•Continuum is modeled with volumetric element.
•Dilatation may be controlled
•Approximately four times slower than mass-spring network
12
Deformable Models (conclusions)
Mass-Spring One dimentional elements Unrealistic to model volume
Tetrahedral FEM Good realism for 3D continuum Control of dilatation Approximately 4 times slower to
evaluate forces
13
Contributions
Quantitative and qualitative comparison of mass-springs and tetrahedral elements
Interactive non-linear static resolutionFormal analysis of the real-time stability of
integration methods based on parameters
Identification of the parameters of a model from experimental data
Relevant medical applications
14
Plan
Deformation Models Mass-Spring vs. FEM
Real-time Resolution Techniques Static Dynamic
Echographic Simulator parameter identification
Liver Model interactive deformation
15
Real-time Resolution
Static Resolution linear resolution [Cotin97]
small displacements
Our approach: non-linear resolutionlarge displacements
Dynamic resolution explicit [Picinbono01] implicit [BW98]
externalfKuuDuM
externalfKu
16
Linear Static Resolution
Principle of virtual work:
internal and external forces are balanced
externalfKu
Linear case:• Pre-inversion (if enough space)
• No large strain
• No rotation
• No material non-linearity
17
Nonlinear Static Resolution
externalfuuK )(Non-linear case:•Stiffness matrix changes with displacement:
•geometric
•material
18
Newton Iteration
Full Newton-Rapson method:
•Reevaluation of Jacobian
•Faster convergence
Modified Newton-Rapson method:
•Constant Jacobian
•Slower Convergence
19
Dynamic Analysis
externalfKuuDuM
u
uY
u
uYf
)(
2nd order non-linear differential equation
Convertto
1st ordersystem
20
Explicit Integration
s
iijij kahYfk
10
Runge-Kutta method with s stages
0
2
4
6
8
10
1 2 3 4 5 6 7 8
Order of consistency (accuracy) vs. stages
j
s
jjkbYY
1
01
s
precision
21
Explicit Integration Stability
hM
K
M
D
M
Dhz
2
22
Im
Re Timestep is limited by the
the physical parameters!
externalfKuuDuM linearizing
22
Implicit Integation
011 )( YYhfY
)(1
)()(
0)()(
0
0
0
0 YffIh
Y
hYfYfhY
YfYfhY
YY
linearisation
Semi-implicit euler
B-stable implicit euler:
Stable for linear case (A-stable)
any timestep
any physical parameters
If you know your history, then you would know where you are coming from.
Bob Marley
Over-damped case
YYf )(
0Y
1Y
h
Y
23
Resolution (conclusions)
Static analysis non-linear resolution for large displacements
Dynamic explicit
strict stability criteria
implicitno limit on timestep, but resolution of non-linear
system
24
Contributions
Quantitative and qualitative comparison of mass-springs and tetrahedral elements
Interactive non-linear static resolutionFormal analysis of the real-time stability of
integration methods based on parameters
Identification of the parameters of a model from experimental data
Relevant medical applications
25
Plan
Deformation Models Mass-Spring vs. FEM
Real-time Resolution Techniques Static Dynamic
Echographic Simulator parameter identification
Liver Model interactive deformation
26
Thigh Echography
27
Echographic Simulator
Data AcquisitionModel of the thigh
Mass-Spring Neural
Interaction collision haptics
Generation of echographic image
baxxxf
)( axxf )(
28
Data Acquisition
64 sample points are marked on the thigh. For each, the forces for some given penetrations are measured
Two different probes
(a) Indentor shaped probe for punctual force-penetration data
(b) Probe with surface equal to that of a typical echographic probe
Two different probes
(a) Indentor shaped probe for punctual force-penetration data
(b) Probe with surface equal to that of a typical echographic probe
1- The end effector advances in small steps (2mm) in the direction normal to the surface of the thigh.
2- The force depending on the penetration distance is measured
(at LIRMM, Montpellier)
29
Data Acquisition: Data Acquisition: Experimental ResultsExperimental Results
The two probes do not offer the same resistance The two probes do not offer the same resistance difference in surface areadifference in surface area
Different curves for different pointsDifferent curves for different points different depth of soft tissuedifferent depth of soft tissue
Highly non-linear behaviour Highly non-linear behaviour
Indentor probeIndentor probe Surface probeSurface probedisplacement
Forc
e
displacement
Forc
e
[d’Aulignac et al.
MICCAI 99]
30
Echographic Simulator
Data AcquisitionModel of the thigh
Mass-Spring Neural
Interaction collision haptics
Generation of echographic image
baxxxf
)( axxf )(
Dynamic Model of the thighDynamic Model of the thigh
baxxxf
)( axxf )(
Incompressibility of the tissue
Elasticity of theepidermis
•Why mass-spring model?
•computationally efficient
•interior NOT discretized into tetrahedra
0LLx
32
Identification of Identification of thethe Parameters of a Parameters of aDynamic ModelDynamic Model
New parameters (elasticity, plasticity, collision stiffness ...)New parameters (elasticity, plasticity, collision stiffness ...)
Desired behaviourDesired behaviour
BehaviourBehaviour
ErrorError
OptimizationOptimization AlgorithmAlgorithm
ModelModel ResolutionResolution --
Measurements
For each sample point, 10-12 deformation/force values with each probe
=> Total of ~1200 measurements.
Parameter EstimationParameter Estimation
Least-squares minimisation:Least-squares minimisation:
1. 1. find (find (a,ba,b) for ) for eacheach non-linear spring non-linear spring
2. 2. find (find (a,ba,b) for ) for eacheach non-linear spring, non-linear spring, andand ( (aa) for ) for allall linear springs linear springs
Error of the model with respect to the experimental data
=> Overall error less than 5%=> Overall error less than 5%
Distribution of Nonzero Error Values
baxxxf
)(
axxf )(
(in collaboration with UC Berkeley)
[d’Aulignac et al., IROS 99]
Error (N)
=> Avoid local minima=> Avoid local minima
34
Dynamic Analysis
Explicit integration Euler stability
too small timesteps• no real-time
...or large mass• slow movement• no gravity
Implicit integration Semi-Implicit Euler
constant Jacobian100 steps per second
• h=1/100 (i.e. real time)
)(1
00 YffIh
Y YY
hYfY )( 0
35
Dynamic Resolution
100 Hz using semi-implicit integration
36
Neural Networks
Forces acting on particles: f
Displacement of particles: uufM )(
•Static Analysis
•Multi-layer perceptron is a general approximizer
•Network is trained directly on experimental data
•back-propagation64 inputs and outputs
37
Neural NetworksDisplacement (mm)
Force (N)
Experimental data Neural Model
38
Mass-Spring vs. Neural Model
Mass-spring topology chosen
based on measurements
dynamic resolution semi-implicit (100 Hz)
Neural model no assuption on topology static resolution
very fastno change of topology
baxxxf
)( axxf )(
39
Echographic Simulator
Data AcquisitionModel of the thigh
Mass-Spring Neural
Interaction collision haptics
Generation of echographic image
baxxxf
)( axxf )(
40
Interaction
Collision Detection
Collision Response
Force Feedback
xxxFcollision
41
Collision Detection
Finds polygons in the OpenGL viewing frustrum
Detects collision between simple rigid body and any other object quickly
42
Collision Response
Inter-penetration distance must be computed
Generates large forces (bad for haptics)
xxxFcollision
Penalty forces [Hunt and Crossley 1975]
43
Haptics
Haptic devices require high update frequency typically around 1kHz
….which the simulation normally can’t meet 100 Hz (dynamic
model)
44
Haptic Interaction
Local approximation of the contact simple local model
running in a separate threadfast collision detectionfast force computation
[Balaniuk 99]
Haptic loop (1kHz):
collision detection and response with local model
Simulation Loop (100Hz):
deformation
global collision detection and response
positionLocal model update
45
Haptic Feedback
time
With local model
Without local model
[d’Aulignac et al. ,
ICRA, 2000]
forc
e
46
Echographic Simulator
Data AcquisitionModel of the thigh
Mass-Spring Neural
Interaction collision haptics
Generation of echographic image
baxxxf
)( axxf )(
47
Echographic Image Generation
64 images aquired on each sample point
Voxel Map 120 Mb
Interpolation fill in the blanks
Provide image any rotation any position
[Vieira01] (in collaboration with TIMC-IMAG, France)
48
Echographic Image Deformation
Problem structures deform
differentlyveinbone, etc.
segmentation
Linear deformation Possible extension:
precalculated deformation maps [Troccaz et al, 2000]
49
A first Prototype
50
Echographic Simulator (conclusions)
Data AcquisitionModel of the thigh
Mass-Spring Neural
Interaction local model
Generation of echographic image linear deformation
baxxxf
)( axxf )(
51
Contributions
Quantitative and qualitative comparison of mass-springs and tetrahedral elements
Interactive non-linear static resolutionFormal analysis of the real-time stability of
integration methods based on parameters
Identification of the parameters of a model from experimental data
Relevant medical applications
52
Plan
Deformation Models Mass-Spring vs. FEM
Real-time Resolution Techniques Static Dynamic
Echographic Simulator parameter identification
Liver Model interactive deformation
53
Keyhole Surgery
Surgery involves soft tissues
Need to model deformation
…in real-time!
simulation
54
Human Liver
•Interior composed of parenchyma
•Surounded by elastic skin or Glisson’s capsule
•Venous network
•Approximate weight: 1.5 kg
55
Liver Model
Geometry
Physical Model
Dynamic Analysis explicit integration stability
Static Analysis non-linear resolution
56
Geometrical Model
•187 Vertices
•370 Triangles
•299 Particles
•1151 Tetrahedra
•1634 Edges
GHS3D
57
Physical Model[Boux et al., ISER, 2000]
Heterogenous Non-linear:
skin Parenchyma
Weight distributed equaly on all particles (i.e. approximately 5g each)
dddFspring
)( 23
1
Strain
Str
ess
58
Explicit Integration
280
1
105)105(2)105(2 6
2
66
KDDhz
280 steps per second
mass 5 grams
hM
K
M
D
M
Dhz
2
22
59
Stability Analysis
Im
Re
280
1
105)105(2)105(2 6
2
66
KDD
hz
60
Simulation
AchitectureSGI Onyx2
Compexity370 facets1151 tetrahedra3399 springs
Frequency150Hz
•Explicit not stable!
•...large mass
61
Static Resolution
The large deformations of the organ during operation require non-linear resolution techniques.
62
Calculate forces on nodes
Evaluate stiffness matrix K? (analytically)
Iteratively solve linear system for displacements u
Ku = f
by successive over-relaxation (SOR)
until residual forces < epsilon through Newton-Rapson iteration
Iterative Solution
63
Modified Newton-Raphson
•Accurate solution (many SOR iterations) does not allow faster solution
•Inexact Jacobian limits convergence speed
•Of special importance for strong nonlinearities
resi
dual
iterations
64
Newton-Raphson
•Less iteration to converge then modified NR
•Exact Jacobian allows faster convergence
•Global time gain when solving linear system accurately
iterationsre
sidu
al
65
Pseudo-Dynamic
Interactive resolution of the non-linear system.
66
Result
1157 tetrahedraIterative non-linear resolution
Rotational invarience
(N.B. Real-time animation)
1157 tetrahedraIterative non-linear resolution
Rotational invarience
(N.B. Real-time animation)
60 NR iterations/sec on SGI Octane 175Mhz
Pseudo-dynamic
67
Liver Model (conclusions)
Physical Model mass-springs
Dynamic Analysis explicit integration unstable
Static Analysis interactive non-linear resolution
68
Summary
Physical Models Mass-Spring or FEM?
Resolution Static
linear or non-linear?
Dynamicexplicit or implicit?
Medical Simulators The choice of numerical methods must be guided
by the application!
69
Contributions
Quantitative and qualitative comparison of mass-springs and tetrahedral elements
Interactive non-linear static resolutionFormal analysis of the real-time stability of
integration methods based on parameters
Identification of the parameters of a model from experimental data
Relevant medical applications