THE RESEARCH PROGRAMME

The research programme has five interacting components:

1. The Measurement Development Programme
2. Research relating to the Educational System
3. Research on Organisational Systems
4. Research into Societal Management arrangements
5. Research relating to the networks of social forces controlling the operation of society and the behaviour of individuals within it

1. The Measurement Development Programme

The Measurement Development Programme itself has 4 subthemes, each directly related to the main themes of Prof. Raven’s research.

Before, or whilst, reviewing the measurement development programme in each area, it may be useful for the reader to review the material on the main research programme in that area by clicking on the relevant link.

Introduction

Raven's Progressive Matrices and Vocabulary tests were based on observations and theories formulated by Spearman at the turn of the last century.

But Spearman himself noted two crucial problems with these theories. First, his "general factor" had emerged from studies of the intercorrelations between traditional tests of "educational" abilities - language, science, mathematics, etc. Yet these tests lack both construct and predictive validity - a point developed in more detail by John Raven in his book The Tragic Illusion: Educational Testing.

Second, Spearman noted that the way in which psychologists were trying to tackle the problem of describing and summarising individual differences was basically off-beam. Thus he wrote: "Every normal man, woman, and child is a genius at something. The problem is to identify at what. This must be a most difficult matter in that it occurs only in relation to a small proportion of circumstances. It certainly cannot be done with any of the psychometric procedures in current use."

The absurdity of trying to summarise and report individual differences in terms of two, five, or sixteen variables can be easily exposed by asking "Where would biologists have got to if they had sought to describe all the variation between plants (or animals) in terms of 2, 5, or 16 variables?" Clearly, if we are to improve our ability to describe individual differences and the emergent properties of groups it will be necessary to develop the equivalent of atomic theory in chemistry or a Linnaeus/Darwinian classification in biology.

But the way we think about and try to assess individual differences is not the only problem.

Exactly parallel problems arise in the way psychologists tend to think about the environment and the way it interacts with individual characteristics. They again try to "measure" variation between environments in terms of "variables". The problems this poses can again be illustrated by asking: "where biologists would have got to if they had tried to understand and describe the effects of environments on animals by first trying to summarize the variance between animals in terms of 16 variables, then the variance between environments in terms of 10, and then running linear multiple regressions between the two in an attempt to assess the effects of the environments on the animals?" An "ecological" model involving multiple interactions and feedback loops is required.

Both John Raven and his father devoted much of their lives to research in which they sought to advance thinking and assessment in these areas.

In addition to a range of books, this work has resulted in a range of preliminary assessment tools based on an alternative way of thinking about individual differences and behaviour.

Unfortunately, these tools cannot be immediately applied within our current workplaces, educational systems, and public management systems because the operation of these systems is determined, not by personal developmental or societal needs, but by a range of latent, rarely discussed, and hard to influence, sociological forces.

But this is not a cry of despair: The observations simply highlight another topic which has been widely neglected by psychologists: It tells us that human behaviour is not mainly determined by people’s internal properties - such as talents, attitudes, and values - but by external social forces. Such a transformation in psychological thinking and theorising is as great as the transformation Newton introduced into physics by noting that the movement of inanimate objects is determined neither by internal, "animistic", properties of the objects nor external animistic entities (such as the gods) but mainly by invisible external forces which act upon those objects - invisible forces that can nevertheless be mapped, measured, and harnessed.

So this brings us to our fifth conceptualisation and measurement topic: How are the external social forces which so much determine human behaviour to be conceptualised, mapped, measured, and harnessed in a manner analogous to the way in which Newton made it possible to harness the destructive forces of the wind and the waves to enable sailing boats to get to their destinations more effectively and reliably … and, in particular, to get to upwind destinations?

Prof. Raven would be extremely interested to hear from researchers tentatively interested in working in any of these areas.

Background

Prof. Raven’s measurement-development activities are grounded in a 50-year programme of research which shows that:

are not only not working well: they are, collectively, heading our species towards extinction at an exponentially increasing rate.

To appreciate the developments that are required in measurement theory and practice, it is really necessary to understand the deep-seated nature of the failures in each of the areas mentioned. However, we will skip over them here and return to them under separate sub-sections of this Research entry. To go directly to reviews of Raven's research in any of these areas please click on one of the following links:

The Educational System
Organisational Systems
Societal Learning and Management Arrangements
The Networks of Social Forces Controlling The Operation of Society And The Behaviour Of Individuals Within It

More specific observations on the requisite developments in measurement theory and practice in each of these areas are discussed below under the following headings which can again be reviewed independently by clicking on the appropriate link.

The Theoretical Bases for the Work
The Conceptualisation and Assessment of Competence
The Conceptualisation and Measurement of Developmental Environments
The Conceptualisation and Measurement of Climates for Organisational and Societal Innovation and Learning
The Conceptualisation and Measurement of Socio-Cybernetic Forces

Top of page

The Theoretical Bases for the Work

Prof. Raven's research work has highlighted a need for radical change, amounting to a paradigm shift, in the way we think about, not only human resources and institutional arrangements, but also in the way we assess them.

Among the developments required are:

Top of page

The Conceptualisation and Assessment of Competence

A century of work has demonstrated that, so long as we try to work within the mainstream psychometric tradition, it is impossible to get very far. As a title of a review article put it "g and not much else" works.

However, in parallel with this fruitless activity, a growing number (now amounting to some 700) studies of the competencies which distinguish more from less effective performance in a wide range of occupational roles have documented the importance of a range of generic high-level competencies, or motivational dispositions.

The problem is to develop a suitable framework for summarising, thinking about, and assessing such competencies.

Two possible frameworks have been published: One (in Competence in Modern Society) by Prof. John Raven and, the other by Lyle and Signe Spencer (in Competence at Work).

Both are grounded in the framework for thinking about, and assessing, "motives" developed by David McClelland and his co-workers.

Correctly understood, McClelland's measures are not measures of motivation, still less "personality", but measures of the competence to carry out valued activities ... defined in such terms as inventing new scientific theories, manipulating groups, organisations, or societies, or putting people at ease.

They measure the respondent's competence to carry out selected tasks by finding out how many of a number of identifiable, cumulative, and substitutable, components of competence he or she displays spontaneously while carry out specific kinds of activity.

What this procedure makes clear is that the usual, internal-consistency-based, measures of such things as "creativity", "self-confidence", etc. are off-beam: Someone who displays a great deal of creativity when, for example, putting people at ease is unlikely to do so when asked to think of as many uses as possible for a brick.

Pursuit of these insights leads to the realisation that psychologists have probably been misguided in their attempt to classify people in terms of scores on variables (as in physics). They should rather have been seeking to describe people in terms of their motives and the components of competence they display when carrying out valued tasks, in a manner analogous to the descriptive statements Chemists make about substances or biologists in classifying plants and animals.

Research in which this alternative way of thinking about competence and its assessment has been translated into practice and has led to some extraordinary reversals of some of the most "well established" findings in educational and occupational psychology.

Summaries of Raven’s work on the nature and assessment of competence will be found in Competence in Modern Society and Competence in the Learning Society.

Top of page

The Conceptualisation and Measurement of Developmental Environments

In the course of his research, Raven and his colleagues have studied the nature of development environments as they occur in homes, schools, universities, and workplaces. To their surprise, a common pattern has emerged.

In developmental environments, effective parents, mentors, and managers tend to create the following for their children, pupils, students, or subordinates:

Note how developmental environments engage with the motives or values of individuals: it is as irrelevant to record features of the "environment" which do not engage with those values as it is to record features of the environments of chemical substances which do not engage with the elements of the substance being studied.

What is needed, then, is a collection of measures of developmental environments which draw people's attention to these findings, enable them to take stock of the current situation, see what needs to be done to improve it, and monitor progress. The measures need to include questions to find out if there has been a serious attempt to identify individual's motives and talents and redeploy personnel in such a way as to use all available high-level talents for at least part of the time. And, to complement these, there is a need for tools to enable managers and others to recognise, develop and utilise the idiosyncratic talents of individuals.

Note that these tools are of crucial importance for the evaluation of teachers and managers: Have they been able to create developmental environments and climates of dedication and enthusiasm?

Preliminary versions of such measures have been developed, are listed among Dr Raven’s publications, and are available for further collaborative development.

Top of page

The conceptualisation and measurement of Climates for Organisational and Societal Innovation and Learning

As will be shown later, delivery of effective educational, health, and welfare services, and, more importantly, the evolution of a sustainable society, is dependent on the evolution of new forms of bureaucracy and democracy. These, in particular, need to incorporate new ways of thinking about the role of the public servant.

We need to charge our public servants with responsibility for creating a ferment of innovation and learning and also hold them accountable for initiating the collection of information, sifting it for good ideas, and acting on it in an innovative way in the long term public interest.

We therefore need tools which can be used to take stock of the extent to which the requisite climates of innovation and learning have been created and, by so doing, direct people's attention to the developments that are needed.

Many of the features which need to be incorporated into such measures are discussed below in connection with innovation in organisations. At a societal level, more attention will have to be paid to:

  1. the investigation of, and development of strategies to influence, the hidden sociological systems processes which deflect most public-improvement action from the achievement of its goals, and
  2. the formal arrangements for advancing understanding and especially, for obtaining comprehensive, systems-oriented, evaluations. [Current beliefs about the arrangements to be made to advance understanding are wide of the mark. The only way to move toward a focus on comprehensive, systems-oriented, thinking (instead of on accuracy in relation to isolated specifics) is to provide resources to a wide variety of people who have very different - currently unsubstantiated - perspectives.]

The features to be incorporated into tools designed to direct attention to the key features of new forms of public surveillance of public servants as managers charged with the duties which have been mentioned are less clear. But there is no mistaking the need to move from concepts of government and supervision based in hierarchy to concepts of democracy grounded in fluid, network-based, and issues-oriented arrangements.

Pilot versions of tools to assess both organisational climate (in relevant terms) and public perceptions of society and how it works, including the role of democracy, bureaucracy, and the market place have been developed and are available for further collaborative development.

Top of page

 

The Conceptualisation and Measurement of Socio-Cybernetic Forces and Societal Innovation and Learning.

As has been mentioned, one of the outcomes of Prof. Raven’s research has been the realisation that it is necessary to “turn psychology inside out” in a manner analogous to the way in which Newton turned physics inside out. Newton “de-animated” explanations of movement. No longer was it attributed to the internal properties of the object or to the whims of the gods.

In a similar way, we need to de-animate explanations of behaviour in psychology.

No longer are we to attribute most of the behaviour of teachers, public servants and politicians to their personal motives or ability deficits. Most of what they do is determined by the systems forces which act upon them. Indeed, it turns out that, despite what was said earlier, much of the variance in individual behaviour is to be explained in this way.

It is relatively easy to construct a systemogram illustrating the interacting and supporting nature of such forces (although mapping them in such a way that the map illustrates the main issues that require attention is more difficult). Illustrative material will be found in Conceptualising, Mapping and Measuring Social Forces and The Development and Use of maps of Socio-Cybernetic Systems to Improve Educational and Social Policy, with particular reference to sustainability.

The problem is to weight these linkages in such a way as to indicate their relative importance. Forrester (See Conceptualising, Mapping and Measuring Social Forces) has provided what might be an appropriate model by mapping the links between the economic and bio-physical resources associated with the survival of the planet and calculating their cumulative effect on a number of important outcomes, such as population and pollution. (See also Meadows et al in Conceptualising, Mapping and Measuring Social Forces

In Raven’s opinion, ways will be found to weight the relative importance of a number of variables which contribute to important collective behaviours if, as the work of Bookchin http://www.eyeonsociety.co.uk/resources/Bookchin.pdf, and http://www.eyeonsociety.co.uk/resources/GS09%20annotated.pdf indicates, the task of generating an extended version of our 1994 map of the Network of Social Forces Perpetuating a Non Sustainable Society is taken seriously.

The task is of inestimable importance to the future of humankind.

As far as the conceptualisation and measurement of these social forces is concerned, it is worth noting that, before Newton, there was no concept of force. There was just the wind and the waves and the gods.

Having educed the concept of “force” … something which was common to the wind, the waves, falling apples, and the movement of the planets … the first thing Newton did was show it was measureable.

He did this by seeing how far he could jump into the wind and how far he could jump with it. The difference between the two gave him a measure of the strength of the force in the wind. Raven suggests that similar insightful experiment will lead us to better conceptualise and measure social forces.

Be that as it may, Forrester (See Conceptualisation, Mapping and Measuring Social Forces) actually evades the problem of measuring forces by measuring, instead, such things as rates of forest depletion, the accumulation of particles in the atmosphere, and various indices of well being and economic growth. He then calculates the regression weights of each of these things against the others and the relative impact of intervention at crucial nodes controlling the rate of flow.

Unfortunately, what is entirely is missing from his model is representation and weighting of the sociological variables which, as we have seen, primarily determine human (social) response to the problem he documents. It is the absence of any attempt to map and understand these that leads him to the despairing conclusion that “we lack the political will” to take the steps indicated.

Top of page