RAGs

Risk Assessment in Genetics

The RAGs project developed a system to assess genetic risk in breast and ovarian cancer, and to support professional counsellors helping people make personal healthcare decisions in these areas. RAGs applied decision theories and technologies developed in the StAR and PROMPT projects, and cognitive models and theories of risk perception from the COGENT project. veloped a system to assess genetic risk in breast and ovarian cancer, and to support professional counsellors helping people make personal healthcare decisions in these areas. RAGs applied decision theories and technologies developed in the StAR and PROMPT projects, and cognitive models and theories of risk perception from the COGENT project.

A collaboration between the ACL and Imperial Cancer Research Fund (now Cancer Research UK) units in London, Oxford and Edinburgh, and supported by the UK Economic and Social Research Council programme in cognitive engineering and the Imperial Cancer Research Fund (1996-2000).

Collaborators

Advanced Computation Lab., Imperial Cancer Research Fund (coordinator)

Cancer Research UK, Psychosocial Research Group at Western General Hospital, Edinburgh.

Cancer Research UK, Cancer Genetics Group (Guy's King's and St. Thomas's hospitals)

Context

For a number of years the ACL has been developing a general cognitive engineering approach to the design of medical and other decision support systems. Our basic aim is to understand human decision making in cognitive terms, rather than with reference to normative statistical theory, and to exploit this in designing decision support systems that are natural for people to understand. We view human decision making as a powerful symbolic reasoning process rather than a degenerate statistical one. This approach has proved to be productive in modelling human decision making (Fox and Cooper, 1997; Cooper and Fox, 1997; Yule, Cooper and Fox, 1998; Glasspool and Fox, 1999). It is also the basis of a formal theory of decision making under uncertainty based on a logic of argumentation (Fox et al. 1992; Fox, 1994; Krause et al., 1994, Parsons et al. 1999). This decision theory is embodied in PROforma, a generic technology for building decision support and intelligent agent systems (Fox et al., 1998).

The RAGs project (Risk Assessment in Genetics) aimed to apply this theoretical approach to gain an understanding of the cognitive basis for the assessment and c ommunication of risk. The focus of the project was the counselling of women at increased genetic susceptibility to breast and ovarian cancer, but the methods employed are intended to be applicable to any kind of genetic counselling and the management of personal risk generally.

Work on the project was carried out in two major streams. The first was concerned with developing a general theoretical approach to the representation of uncertainty, while the second applied this theoretical background to the development of decision support software for a specific medical application involving the communication of risk: genetic counselling of women at risk from breast cancer.

Cognitive modelling and theory

We established a programme of theoretical work on how people understand risk using the COGENT cognitive modelling system. The first piece of work explored the idea of using mental models (a la Johnson-Laird, 1983) augmented with symbolic argumentation to model people’s understanding of risk scenarios (Jungerman, 1985; Huys et al. 1997). We were able to show that the approach can provide an effective decision procedure in a simulated breast cancer management decision.

This model demonstrated in principle that a symbolic decision process can be pratically effective in a risk management setting, but does not provide an empirical test of its psychological validity.

We investigated this problem in the context of a central question: what do people understand by uncertainty terms like "possible", "probable", "improbable" etc? These form part of everyone’s routine vocabulary in talking about risk, and genetics; counsellors and other medical professionals depend heavily upon them. The orthodox interpretation is that the meaning of these modal terms is to be understood in terms of probabilities, as in "P is possible if prob(P) > 0)" (or analogous "fuzzy" quantities). In contrast our view is that an alternative role is to communicate a summary of the reasons to believe in a proposition or claim, as in "P is possible if there is at least one argument in support of P and none to exclude it".

We analysed data from an experiment in which subjects were asked to judge the equivalence or otherwise of modal sentences using verbal uncertainty terms and sentences which emphasise the underlying argumentation structure. (The experiment was carried out with the help of David Hardman and Peter Ayton at City University.) We then used COGENT to develop a model of the linguistic reasoning processes involved in this judgement based on mental models theory augmented with symbolic argumentation. We developed an elegant symbolic model that very accurately predicts the subject data (Glasspool and Fox, REF). This provides at least as good an account as that provided by a Bayesian variant (also implemented in COGENT) but has a more natural fit with current cognitive theory.

Our approach drew heavily on the desire to unify concepts and results from different subfields of cognitive science (reasoning, decision making, planning, knowledge representation etc.). In our view unification of, rather than competition between, cognitive theories is a prerequisite to developing an effective discipline of cognitive engineering. The collaboration between Imperial Cancer Research Fund (now Cancer Research UK) and Birkbeck College in the COGENT project proved to be highly productive in this respect, having made a significant contribution to the development of a unified theory of reasoning, decision making and action. This theory is summarised by the domino model shown in Figure 1 below

The Domino modelThe Domino model

The domino model represents a convergence of theoretical work on cognitive agents (Das et al. 1997) and interpretation of data from empirical studies of medical reasoning and decision making (Cooper, Fox, Yule 1998). Its formal semantics are well understood and we view it as an intuitive and plausible competence theory of much human reasoning and decision making in knowledge rich domains. We believe that it may offer a more general, plausible and productive basis for developing performance models of human cognition than available alternatives (c.f. reasoning models based on classical deduction; decision models based on expected utility and Bayesian formulations). The domino model is also the basic design framework for the PROforma knowledge representation language.

One of the results of the revolution in molecular genetics is the increasing availability of tests for the presence of genetic abnormalities which predispose certain individuals to contracting particular diseases at some point in their lives. For example, the location of the BRCA1 gene for familial breast cancer has recently been determined on chromosome 17, and it is confidently expected that a reliable test for the presence of the gene will be available within one or two years. In the UK alone this will impact of the order of 30,000 women who are estimated to carry the gene. Given a positive test (which implies an 80% lifetime risk of contracting the disease) such women will require supportive counselling to help them understand their individual circumstances and their options for monitoring, prophylaxis and therapy.

The aim of genetic counselling is to provide individuals with information about their personal risk of developing a disease in such a way as to encourage appropriate health care decisions (e.g. to attend for regular screening and increase the chances of early detection). Unfortunately, counsellors are not well served with techniques which are proven to facilitate understanding of personal risks.

RAGs decision support software

In parallel with the theoretical work described above, and drawing on it, we have developed a decision support system for breast and ovarian cancer (Coulson et al., submitted). The system is designed to assist the general practitioner (GP) in assessing and communicating genetic risk information to women who are worried about their personal risk. The tool consists of a user interface which allows a family tree to be constructed for the presenting patient, incorporating information on the incidence of cancers (see Figure 2). This interfaces with a risk calculation protocol written in PROforma which generates an assessment of risk level and recommendations for patient management, along with an explanation for the conclusions reached. The user interface component is designed to be generic and may interface with different protocol software to assess the risk of different genetic conditions.

Figures 2-3 illustrate RAGs being used in the field of breast cancer.

Figure 2 below shows the constrution of a family tree in RAGs; figure 3 shows a typical risk report created by the system.

RAGs family treeRAGs family tree
Figure 2: RAGs family tree

RAGs risk reportRAGs risk report

Figure 3: RAGs risk report

Clinical evaluation of RAGs has shown that it is an extremely effective instrument for constructing family histories and assessing risk. It has been the subject of two clinical papers in the British Medical Journal (1999; 319: 32-36 and 2000; 321: 28-32), two technical papers (one to appear in J. Informatics in Primary Care and one submitted to Methods of Information in Medicine), and various conference papers, invited talks and articles in the media.

The user interface design benefited greatly from work performed by Dr Jon Emery in Oxford. Dr Emery evaluated the program by having general practitioners (GPs) use it in mock consultations, with actors playing the parts of patients. The results of these trials have been very positive, and have provided important feedback on the appropriate user interface design for use in consultation sessions (Emery et al., 1999). Subsequent studies by Dr Emery have suggested that doctors make less mistakes when entering data into this system than when using other pedigree programs. Plans are currently under way to develop a version of the application for colorectal cancer.

RAGs Publications:

Publications from the RAGs project listed on the COSSAC publications database.

References for text:

Coulson AS, Glasspool DW, Fox J & Emery J. " RAGs: A novel approach to computerised genetic risk assessment and decision support from pedigrees." Methods of Information in Medicine 2001; 40; 315-322.
Appendices: Guidelines; Algorithm

Emery J., Walton R., Murphy M., Austoker J., Yudkin P., Chapman C., Coulson A., Glasspool D., Fox J. Computer support for recording and interpreting family histories of breast and ovarian cancer in primary care: comparative study with simulated cases British Medical Journal 2000;321:28-32 ( 1 July )

Emery, J., Walton, R., Coulson, A., Glasspool, D., Ziebland, S. & Fox, J. (1999). Computer support for recording and interpreting family histories of breast and ovarian cancer in primary care (RAGs): qualitative evaluation with simulated patients. British Medical Journal, 319, 32-36.

Cooper R, Fox J (1997). Learning to Make Decisions Under Uncertainty: The Contribution of Qualitative Reasoning. In Shafto MG and Langley P eds Proc. of 19th Annual Conference of the Cognitive Science Society, 125-130.

Cooper R, Fox J. and Yule P. (1998). COGENT, an authoring environment for cognitive modelling Behaviour Research Methods and Instrumentation. (details not available yet)

Das S K, Fox J, Elsdon D, Hammond P (1997) A Flexible architecture for autonomous agents. Journal of Experimental and Theoretical Artificial Intelligence, 9, 407-440.

Fox J, (1994). On the necessity of probability: Reasons to believe and grounds for doubt. Chapter 5, in G Wright and P Ayton (eds) Subjective Probability, Chichester: Wiley.

Fox J. and Cooper R. (1997) Cognitive processing and knowledge representation in decision making under uncertainty Psychologische Beitrage, Special Issue on Qualitative Aspects of Decision Making.

Fox J, Krause P and Ambler S. (1992). Arguments, Contradictions and Practical Reasoning. In: Neumann B (Ed) ECAI92, Vienna, Austria. Proc. 10th European Conf. on AI; 623-627.

Fox J, Johns N, Rahmanzadeh A (1998). Disseminating medical knowledge: the PROforma approach. Artificial Intelligence in Medicine, 14 (1,2), 157-182.

Glasspool, D.W. & Fox, J. (1999). Understanding probability words by constructing concrete mental models. To appear, 21st Annual Conference of the Cognitive Science Society.

Huys, J. Evers-Kiebooms, G. & d�Ydwalle, G. (1992). Decision making in the context of genetic risk: The use of scenarios. Birth Defects: Original Article Series. 28. 17-20.

Johnson-Laird, P. N. (1983). Mental Models. Cambridge: Cambridge University Press.

Jungermann H (1985). Inferential processes in the construction of scenarios. Journal of Forecasting, 4:7-34.

Krause P, Fox J, Judson P. (1994). An Argumentation Based Approach to Risk Assessment. Presented at IMA conf. on Risk: Analysis and Assessment, 14-15 April 1994. In IMA Journal of Mathematics Applied to Business and Industry, 5, 249-263.

Parsons S, Fox J, Coulson A. "Argumentation and risk assessment." Stanford Spring Symposium on Predictive Toxicology, Stanford CA, 1999.

Yule P, Cooper R, Fox J. (1998). Normative and Information Processing Accounts of Decision Making. Proceedings of the 20th Annual Conference of the Cognitive Science Society.

Acknowledgements

RAGs was designed and developed in the Advanced Computation Laboratory by Drs. Andrew Coulson and David Glasspool, with clinical input from Dr. Jon Emery who also carried out the clinical evaluation of the system.

Support

RAGs was funded by Cancer Research UK and the UK Economic and Social Research Council programme in cognitive engineering.