Problems of artificial system - human subject interaction reliability
Prof. Ing. Votruba Zdeněk
Prof. MUDr. Faber Josef
*) Prof. Dr. Mirko Novák, Prof. MUDr. Josef Faber, Prof. Zdenek Votruba
Joint Laboratory of System Reliability,
Department of Control Engineering and Telematics
Faculty of Transportation Sciences
Czech Technical University, Prague,
Konviktská 20, 11000 Prague 1, Czech Republic
Institute of Computer Science,
Academy of Sciences of the Czech Republic,
Pod vodárenskou veží 2, 18207, Prague 8, Czech Republic
e-mail: mirko@fd.cvut.cz, mirko@cs.cas.cz
The problem of interaction reliability between artificial system and human being (operator, user), though extremely important, is still not solved satisfactorily, even the control of dominant part of complex and complicated systems is now realized by computer or at least by computer assistance.
The human operator, who has to interact with powerful, complicated and often also efficient artificial system (such as transport system, aircraft, express trains, large trucks, extensive power stations, security and defense systems etc.), is imposed to requirements on fast and correct reactions on very variable actual situations. Such exposition exists usually for considerably long time of his/her service.
The resulting high load of such operator´s brain and nervous system results necessarily to decrease of his/her vigilance and to degradation of his/her attention.
These demands on human operator ability are often jointed with influence of some internal and/or external factors. These are e.g.
* the long length of service,
* psychical isolation in the course of service,
* operators actual mental and physical state,
* climatic conditions,
* quality of environment in the particular cock-pit or control room,
* monotony of the scene or image, which operator has to observe etc.
The endeavor of our investigations is at present the diagnosis of attention decrease and of the related reduction of the reaction speed to unexpected emerging situation.
Such attention degradation can also result in real micro-sleep.
Both this states of operators brain,
* the long lasting attention decrease and
* the short lasting micro-sleep,
can be extremely dangerous and can result not only in huge material and financial damages, but also to losses of human lives.
The requirements on a human operator of an artificial system can be concentrated in the following main categories:
a) requirements on attention level and continuity,
b) requirements on the speed of operator reaction,
c) requirements on the correctness of operator decisions.
Within all three above-mentioned categories of the reliability of human operator - technical system interaction a correlation naturally exists.
A straightforward correlation exists between attention level and speed of reaction.
Operators commanding at high level of attention usually also possess very fast reactions.
On the other hand cases can appear, when fast, almost impulsive reaction may not be accompanied by very high level of the operator's concentration and attention.
Some people can react fast also when their attention is shared by different objects (they have very fast reflexes).
In addition, a high level of attention in the majority of cases leads to a very high probability of correct decisions and vice versa - if somebody is not concentrating enough, there is a rather low probability that his/her decision will be correct.
On the other hand, in the case of very fast reactions accompanied by a very low level of a human operator's attention, the probability of an incorrect decision can increase significantly. This is typical for a so-called surprise reaction, from which a transition to a panic reaction can sometimes be observed.
A drop in the attention level of a particular human operator can be caused by various external or internal reasons;
some of them have a general character;
the intensity of others depends significantly on the operator's individuality.
Among the general conditions causing the decrease of attention is:
- Extreme length of a particular operator's service without breaks,
- Operator's physical and mental exhaustion,
- Monotonous scene which the operator has to observe for a long time,
- Extreme temperature in which the operator has to serve (too high or too low),
- Extreme humidity in which the operator has to serve (too high or too low),
- Extreme air pressure,
- Air smell, dust density etc.
- Matters leading the operator to concentrate on problems other than from his/her main service.
The level of a operator's attention can be measured by the use of some real figure of merit LAT, expressed by real numbers, e.g. by the inverse of his/her reaction time RT. The time development of probands LAT has usually very variable character. In an ideal case, when only the factor of length of his/her operation service can be taken into account it has the form, sketched in Fig. 1.
Fig. 1: The typical shape of probands level of attention when no other factors besides the length of service are considered.
In the region of shallow attention decrease the probands reaction time is usually significantly prolonged (often 3 to 5 time). In addition to this the angle, in which he/she is able to react in adequate manner on visual stimuli is significantly narrowed and the probability of his/her false reaction in much higher. More over, some probands can stay in such stage for considerably long time. They are still able to operate the respective system in principle, but the probability that they operate it in safe manner is very lowered. Actually, they operate it in dangerous form.
When the particular proband is so lucky and causes no accident in the course of his/her system operation during the region of shallow attention decrease, he falls into the second region, where the level of his/her attention falls suddenly to zero. His/her eyes close and he starts the period of deep micro-sleep. Here his/her ability to control and operate the respective system well is reduced almost to zero.
Nevertheless, there is known, that some skilled professional truck drivers can drive also being in this region - of course their abilities to control the car are reduced to some small set of basic functions, like to keep the car in straight direction and to keep the constant speed. They relax for short time on such pieces of highways, which they know well as straight and quiet. Problems appear, when some unexpected situation happens - usually with result of heavy accident, because they do not react adequate.
In real situation, such ideal time-dependence of attention reduction in the course of long operator service appears only exceptionally. The particular operator attention decreases for some time. After that, it can be provoked by some alarming stimulus or situation. This is followed by another decrease, and so long.
However, after several repetitions of such cycles, the sensitivity of the tired operator to provocative stimuli appears to be much lower and his/her average level of attention it falls down definitely (see Fig. 2).
Fig. 2. To the temporary improvements of operators attention level
Till known, no on-board applicable method for direct measurement of the human subject reaction time RT is known.
Almost all direct methods are based on measurement of the time interval, which appears between the time of stimulus appearance and the time of human subject final reaction (inclusive the time of his/her particular effector function).
Such methods of measurement are considerably well realizable in laboratory. In on-board praxis, such provocation is usually unwanted because it can have negative influence on human operator. Therefore, for practical purposes the indirect methods of actual values of RT estimation are to be developed.
For this purpose one has to find a set of suitable, on-board measurable parameters which can be used for identification of the attention decrease and of onset of the micro-sleep.
Among such parameters belong:
* the electro-magnetic activity of brain,
* frequency of breath,
* frequency of hearth beats,
* eye movements,
* skin resistance,
* face grimaces etc.
All these parameters have some advantages and also disadvantages.
We have chosen the EEG activity as the dominant significant parameter for our investigations.
The reason is that this is probably the only one source of data, from which the almost immediate and reliable information about the brain function can be mined.
After mining information from many EEG time-series recorded on several tenths of human subjects in our laboratory we can analyze the procedure of the respective operator vigilance decrease and attention degradation almost immediately and with quite acceptable reliability.
As basis for such information mining we have used certain relations, which can be found among some components of EEG spectra. Here one has of course to take into account that the EEG time-series are of the quasi-periodical and quasi-stationary character in principle.
Fig. 3: Example of EEG time series.
Due the quasi-periodical and non-stationary character of these time series the standard methods of spectral analysis are of limited use here only. We have applied a Gabor filtration based on the use of polynomial filter of the 50 degree considerably successfully. Example of resulting spectra is presented in Fig 4.
 |
 |
Fig. 4a: Rail-road driver in relaxation stage, RT = 494 ms.
Fig. 4b: Rail-road driver in relaxation stage, RT = 810 ms.
The careful analyses of many such results allowed us to find such dependence like that shown schematically in Fig. 5
Fig. 5: Example of dependence among dominant amplitudes of two Gabor spectral lines and and reaction time RT.
Such dependence are of very individual character and differ for each proband, but seems to not change for long time of his life.
Beside this we search for much sharper analytical tool, by the use of which we can mine from the EEG time-series the information on the actual level of probands attention.
One of my graduate students, Ing. Petr Svoboda has tried to use for this purpose:
analysis of the poles and zeros locations of Z-transfors of some approximations of selected time-series windows,
analysis of time development of some parameters of differential equations describing the respective time-series.
Example of his results, which I consider as very interesting is shown in Fig. 6 and 7.
Fig. 6a
Fig. 6b 
Fig.7Having so some information about the particular proband attention behavior, we can construct a set of suitable attention predictors, which give us the information about the expected development of the particular operator attention in the course of his/her service.
Such predictors are to be individually adjusted to each operator under consideration. We expect that these predictors can work with acceptable prediction reliability (about 90%) for the prediction horizon about 2 - 3 minutes.
We hope that on the base of this knowledge, in the course of further research the development of on board-applicable measuring set will be possible. By the use of this we hope to be able to warn the system operator and eventually also his/her supervisor, that the particular subject is losing his/her attention and starts to operate the respective system dangerously.
The principal scheme of such a warning system for micro-sleeps is shown in Fig 8.
Fig. 8: Scheme of micro-sleep warning system. The yellow line represents the way for eventual prevention.
References:
[1]…Mišovic M., Malý V.: Reliability of Information Systems, CATE, Brno, 1993
[2]…Blackmore S.: The power of memes, Scientific American, Vol. 283, October 2000, No.
4, 52-63 [3]…Dawkins R.: The selfish gene, Oxford University Press, 1976, 1989
[4]…Blackmore S.: The meme machine, Oxford University Press, 1999
[5]…Dotlacil M., Kopriva P.: Networks Reliability Workshop 99, CTU, Prague, 1999, 41, 42
[6]... Nilsson T., Nelson T.M., Carlson D.: Development of fatigue symptoms during simulated driving Accid Anal Prev , 1997 July, 29(4), p. 479-488
[7]... Marottoli R.A., Richardson E.D., Stowe M.H., Miller E.G., Brass L.M., Cooney L.M. Jr, Tinetti M.E.: Development of a test battery to identify older drivers at risk for self-reported adverse driving events J. Am. Geriatr. Soc.,1998 May, 46(5), p. 562-568
[8]... Arnold P.K., Hartley L.R., Corry A., Hochstadt D., Penna F., Feyer A.M.: Hours of work, and perceptions of fatigue among truck drivers Accid, Anal, Prev, 1997 July, 29 (4), p.471-477
[9]… Novák M.: Theory of System Tolerances (in Czech), Academia, Prague, 1987
Control functions in micro-sleep
How the EEG signals can be measured ?
Thalamo - cortical oscillation consists of extremly complicated concert of very many almost parallel paths of electrical impulses travelling in membranes of axons connecting cortical and thalamic neurons.
These impulses have the potential of about 70 mV. However the resulting electromagnetic fiels is projected on the head surface with voltages about 10 - 30 mV only.
These electric waves are of quasi-periodical and quasi-stationary character and its main spectral components appears in the range from 1 to 30 Hz.
Faber glasses, scheme
Faber glasses, laboratory model
 |
 |
 |
EEG maps
Attention curve
Idealized EEG spectrum
RT versus P of wrong response
RT versus alpha, delta
How the micro-sleep can be detected?
By measuring of some significant human operator parameters (so called "markers"), e.g. breath temperature, eye movement, electro-cardiographic signals etc.
Among them the electroencephalographic (EEG) signals are most iportant.
Why?
EEG signals are the only marker which is immediately, primary producted my the brain activity influencing vigility and sleep. All other markers concern the secondary factors.
EEG signals are produced immediately by thalamo-cortical oscillations in brain structure.
| top |