9.1 introduction 9.2 atam method 9.3 example 9.4 mpm methodohar/luennot/luennot2006/ohar9...

1Ohjelmistoarkkitehtuurit Syksy 2006 TTY Ohjelmistotekniikka

9. Evaluation of software architectures9. Evaluation of software architectures

9.1 Introduction9.2 ATAM method9.3 Example9.4 MPM method9.5 Summary


9.1 Introduction9.1 Introduction

What is "architectural"?

System S

components,subsystems

not "architectural"in the context of STypically not architectural:- data structures- algorithms- details of interfaces


Evaluation viewpoint to architecturesEvaluation viewpoint to architectures

Architecture description should contain the information needed to reason about the quality requirements of a system.

Architectural evaluation is usually performed against the quality requirements of a system, but alsofunctional requirements can be evaluated.

Architecture determines to what extent (most of)the quality requirements are satisfied.


Quality attributes to be evaluatedQuality attributes to be evaluated

• Performance• Reliability• Availability• Security• Modifiability• Portability• Variability• Usability• Memory efficiency…


Different quality attributesDifferent quality attributes

• Run-time quality attributes (e.g. performance)

• Development & evolution time quality attributes (e.g. modifiability)


What are the results of evaluation? What are the results of evaluation?

Evaluation typically yields answers to the following types of questions:

1) Is the planned architecture suitable for the system,satisfying essential quality requirements?If not, why?

2) Which of the alternative architectural solutionsis the most suitable for the system, and why?

3) How well a particular quality attribute can be achieved with the planned architecture?

Note 1: the answers are based on the architecture description, assuminga reasonable implementationNote 2: the results of the evaluation depend on the accuracy of the architecture description


Why should software architectures be Why should software architectures be evaluated?evaluated?

• Architecture is the first precise description of a system• Architecture contains the most critical decisions of a system• Architectural evaluation increases the understanding of the system

• Architecture determines many process and organization aspects as well


When should software architectures be When should software architectures be evaluated?evaluated?

• After the first architectural sketch (pre-evaluation)

• After the architecture is designed,before implementation is started (full evaluation)

• After the system is implemented (sensible only inthe case of legacy systems)


How to elicit information about the system'squality properties based on architecture?

• Scenarios• Checklists• Metrics• Simulations• Experts


ScenarioScenario--based evaluation methodsbased evaluation methods

SAAM (Software Architecture Analysis Method)• concentrates on modifiability, portability, variability• developed at SEI• based on evolution-time scenarios

ATAM (Architecture Tradeoff Analysis Method)• covers all quality attributes• developed at SEI• derived from SAAM

MPM (Maintenance Prediction Method)• concentrates on maintainability (cost of maintenance)• developed by Jan Bosch• based on maintenance scenarios


9.2 ATAM (Architecture Tradeoff Analysis 9.2 ATAM (Architecture Tradeoff Analysis Method)Method)

1. Presentation

2. Analysis

3. Testing

4. Reporting

Phase 1

Phase 2


Presentation (Phase 1)Presentation (Phase 1)

1. Present the ATAM method• ATAM phases• ATAM techniques (scenarios, quality attribute trees, etc.)

2. Present the business drivers• most important functions of the system• business goals of the system• constraints (economic, political etc.)

3. Present the architecture• technical constraints (operating system, hardware etc.)• external interfaces of the system• architecture description


Analysis (Phase 1)Analysis (Phase 1)

4. Identify architectural solutions• identify and name used styles, patterns etc.• add explanations on how the solution achieves certain quality attributes

5. Produce quality attribute tree• refine quality requirements to the level of scenarios• prioritize by importance and difficulty

6. Analyze the architectural solutions against thescenarios

• architecture is probed against scenarios• risks, nonrisks, sensitivity points and tradeoff points identified


Quality attribute treeQuality attribute tree

Quality

Performance

Modifiability

Availability

Security

quality attributes

Throughput 1000 servicerequests/sec (H,M)Authentication responsein less than 1 sec. (H,M)

Change to Web UIin 1 month (M,H)Change to Linux in 6 months (L,H)Restart after disk failurein 5 minutes (L,H)

Restart after auth servercrash in 5 minutes (M,M)

Credit card transactionsecure 99.999% (H,L)

scenarios

TransactionthroughputResponse time

Change UIChange OS

Hardware failureServer crash

Data confidentiality

refinements …


RisksRisks

Risk = potentially problematic architectural decision

example:

The rules for writing business logic modules in the second layer of your three-layer architecture arenot clearly articulated. (decision/fact in architecture)This could result in replication of functionality (rationale),thereby compromising modifiability (quality atttribute implication).


NonrisksNonrisks

Nonrisk = good decision/property in the architecture based on safe assumptions

example:

because components do not have to know the components responding to their events (rationale).

Assuming that only the actually responding componentsare registered as observers (assumption),

the use of the Observer pattern in the communication betweenthe components (decision)improves modifiability (quality attribute implication),


Sensitivity pointsSensitivity points

Sensitivity point = an architectural aspect that is critical for achieving a particular quality attribute

examples:The portability of the system is sensitive to the use of theMVC model in the GUI implementation.

The modifiability of the system is sensitive to the use ofthe Abstract Factory pattern for the creation of the driverobjects.


Tradeoff pointsTradeoff points

Tradeoff point = Sensitivity point that affects more than one quality attribute

examples:Use of the State pattern in the implementation of the statemachine improves modifiability but impairs performance.

Using XML for the input format improves interoperabilityof the system but impairs performance.


Testing (Phase 2)Testing (Phase 2)

7. Produce testing scenarios• all stakeholders brainstorm and rank scenarios from their perspectives • voting techniques for ranking scenarios • validate & complete quality attribute tree

8. (Re)analyze the architectural solutions • highly ranked scenarios run against architecture • map highly ranked scenarios to architectural solutions & quality attributes


Reporting (Phase 2)Reporting (Phase 2)

Output of ATAM

• Identification and analysis of architectural solutions• Identification of prioritized scenarios• Quality attribute tree• Identification of risks• Identification of nonrisks• Identification of sensitivity points and tradeoff points• Identification of risk themes


9.3 Example: Car service monitor system9.3 Example: Car service monitor systemA car control system needs to be extended with a subsystem that collects variouskinds of data during the running of the car, to be used for monitoring and service purposes. The control system is based on a CAN-bus. The bus is used to send messagesbetween the components in the system. The monitoring system (calledMS) listens to the messages sent along the CAN-bus. It should be possible to addlater various kinds of processing capabilities to MS, reading certain kinds of messages,performing arbitrary computation on the basis of the transferred data, possibly storing the computed data on a local database, and sending in certain cases information to a central service station through GSM. For example, MS may collect information about the usageof gears and breaks, about the speed, about engine temperatures, consumption etc.The driver should have access to the collected information through a graphical user interface and activate or passivate certain information collecting services. Since MS is not controlling critical functions, there are no hard real-timerequirements. It should be possible to receive monitored information also from externalunknown systems, and to receive monitoring requests from such external systems.The system should be easily configurable to various kinds of cars, and it should be possibleto upgrade the system at run-time.


MessageDispatcherIF

send(Msg)register(MsgType,Component)

Component

Msg

receive(Msg)

type(): MsgType

XMLMsg

GSMCompBrakeState

DBBrakeController

recordUsagecheckConditionregister(View)getState()setState()

handleEvent(Event)

BrakeViewIFupdate()

CANBus

CANFilter

MessageDispatcher

sendReport

Brake-View

BrakeModelIF


Analysis (1)Analysis (1)

4. Identify architectural solutions• identify and name used styles, patterns etc.• add explanations on how the solution achieves certain quality attributes

Architectural solutionUse of the message dispatcher architectural style in the communication of components

ExplanationSupports dynamic extensibility of the system, becausenew components can be loaded at run-time,interacting with other components through messages.


Analysis (2)Analysis (2)5. Produce quality attribute tree• refine quality requirements to the level of scenarios• prioritize by importance and difficulty

Quality

Modifiability

Extensibility

Performance

Availability

Usability...

Data modifiability

Functional modifiability

...

S1: Change database in1 month (H,M)S2: Change message formatin two weeks (L,H)

Change UI

Change data collecting

Component serviceresponse in 1 microsec

...


Analysis (3)Analysis (3)

6. Analyze the architectural solutions against thescenarios

• architectural solutions mapped to scenarios• risks, nonrisks, sensitivity points and tradeoff points identified

Scenario: S1: Change database in 1 monthQuality attribute: ModifiabilityStimulus: Old database no more maintainedResponse: Change done in 1 month Architectural solutions: Risk Nrisk Sens TroffAbstract interface for database NP1 SP1access (interface+DB comp)


NP1: Assuming that the persistence services can be provided by a newdatabase, the use of an abstraction layer (interface + DB component)between the other components and the database makes the change ofthe underlying database easy, thus improving the modifiability of the system.

SP1: The modifiability of the system is sensitive to the use of abstractionlayer between the components and the concrete database.


Testing (1)Testing (1)7. Produce testing scenarios• all stakeholders brainstorm and rank scenarios from their perspectives • voting techniques for ranking scenarios • validate & complete quality attribute tree

A scenario presented by a manager:It is possible that in the future the company will develop its own proprietaryinformation exchange format for onboard devices, instead of usingstandard CAN-bus. Therefore it should be possible to read data fromother sources than CAN-bus.

A scenario presented by a car engineer:Actually the behavior of the car is also affected by the external conditions.For example, if the temperature is low, the fuel consumption gets higher.Therefore information about external conditions from various sensors might be later added to the system.


Testing (2)Testing (2)8. (Re)analyze the architectural solutions • highly ranked scenarios run against architecture • map highly ranked scenarios to architectural solutions & quality attributes

Scenario: S27: Change CAN-bus to proprietary information source in two monthsQuality attribute: ModifiabilityStimulus: CAN-bus not any more used in companyResponse: The modification done in two monthsArchitectural solutions: Risk Nrisk Sens TroffSeparate source handler NP14 SP22Abstract interface for messages NP15 SP23


9.4 MPM: Maintenance Prediction Method9.4 MPM: Maintenance Prediction Method

1. Define scenario categories2. Define scenarios3. Assign weights for scenarios4. Impact analysis5. Maintenance cost prediction


Define scenario categoriesDefine scenario categories

Scenario categories • serve as the first step of finding scenarios • give structure for the scenario set• help to find a covering set of scenarios• can be based for example on enviroment change types, source of change etc.

Example:Scenario category I: New/changed processing functionalityScenario category II: New/changed hardware devicesScenario category III: New/changed infrastructureScenario category IV: New/changed implementation techniquesScenario category V: Changed UI


Define scenarios (1)Define scenarios (1)

Scenario category I: New/changed processing functionalityS1: Change the way a serious damage is inferred from engine temperature data ...

Scenario category II: New/changed hardware devicesS6: Add a new sensor for the temperature of the gear box, producing CAN messages to be monitored and analyzed statistically...

Scenario category III: New/changed infrastructureS11: Change the database system to another one with similar interface,both being relational...


Define scenarios (2)Define scenarios (2)

Scenario category IV: New/changed implementation techniquesS15: Change the message format from XML to binary...

Scenario category V: Changed UIS18: Add voice output for alarm situations...


Assign weights for scenariosAssign weights for scenarios

Scenario Normalized probability

S1

S22

Sum 1.0

0.05

0.12


Impact analysisImpact analysis

Scenario Affected components LOC

S1

...

Engine Controller component

Impact

20% 0.2x1500=300

... ... ...

S15 SourceHandlerComponentXComponentY...

... ... ... ...

30%5%5%...

0.3x1800=5400.05x200=100.05x500=25


Maintenance cost predictionMaintenance cost prediction

Weighted LOC impact for a scenario:

Weight * Impact = 300*0.12 + ... = X LOC

scenario ii i

Assume average cost of LOC in the enterprise: Y € / LOC

Assume expected number of scenarios in year: Z scenarios:

=> Expected maintenance cost per year: Z * X * Y €


9.5 Summary9.5 Summary

Assessment methods provide an opportunity for such discussions that should be carried out anyway during the process

Assessment methods can be regarded as exceptionally deepand systematic architectural review techniques

Methods are not (probably deliberately) very precise, giving a lotof freedom to tailor them for a particular case & company

Methods need a significant amount of resources and commitmentwhich is not always available (especially ATAM)


Course synthesisCourse synthesisArchitecture & software development processWell-defined & documented architecture gives theconstitution of a software system

Architecture & dependenciesArchitectural solutions reduce unnecessarydependencies in the system

Architecture & reuseProduct-line architecture and variation managementenable systematic reuse of enterprise know-how

Architecture & qualityArchitectural assessment, carried out as part of the process, ensures quality attributes

9.1 introduction 9.2 atam method 9.3 example 9.4 mpm methodohar/luennot/luennot2006/ohar9...

Documents