WO2017145226A1 - Mental activity state assessment assisting apparatus, mental activity state assessment assisting system, and mental activity state assessment assisting method - Google Patents

Abstract

This mental activity state assessment assisting apparatus assists in the assessment of a mental activity state using a plurality of types of brain activity signals. A mental activity state assessment assisting method comprises steps of: holding error calculation information that shows the correspondence between error measurement signals that show the level of factors which change the plurality of types of brain activity signals and which are factors other than the mental activity state, and errors, caused by the factors, in estimated values of the mental activity state that correspond to each of the plurality of types of brain activity signals; obtaining the error measurement signals; referring to the error calculation information to calculate, from the obtained error measurement signals, the errors caused by the error factors in the estimated values of the mental activity state that correspond to the plurality of types of brain activity signals; and selecting a first type brain activity signal to be output from among the plurality of types of brain activity signals on the basis of the size of each calculated error.

Description

Mental activity state evaluation support device, mental activity state evaluation support system, and mental activity state evaluation support method

The present invention relates to a mental activity state evaluation support device, a mental activity state evaluation support system, and a mental activity state evaluation support method.

In order to evaluate various mental states (mental activity states) such as attention state, mood, and stress, the brain activity signal is measured, and the measured data is subjected to signal processing. A technique for calculating the value has been proposed. As a background art of this technical field, there is JP 2010-198232 A (Patent Document 1).

This gazette includes “a first brain activity data acquisition unit that acquires the first brain activity data of the user when the user indicates one intention, and the user when the user indicates the one intention”. The second brain activity data acquisition unit for acquiring the second brain activity data and the first brain activity data indicate that the indicated one intention corresponds to one intention identifier among a plurality of intention identifiers. From the first probability calculation unit that calculates a certain first probability for each intention identifier, and the second brain activity data, the indicated one intention is an intention corresponding to one intention identifier among a plurality of intention identifiers. A second probability calculation unit that calculates a certain second probability for each intention identifier, an intention determination unit that acquires an intention identifier using the first probability and the second probability, and an output unit that outputs the intention identifier With a brain information output device equipped with It is described as Ku intended can detect. "(See Abstract).

JP 2010-198232 A

For example, brain activity due to activities not included in the mental activity state to be calculated (for example, brain activity due to emotion or movement when the attention state is the difference calculation target), personal characteristics (for example, brain structure, blood vessel structure, habituation, development) ), And internal environment such as biological activities other than brain activity (eg, eye movement, muscle activity), and working environment (eg, body movement, posture), surrounding environment (eg, lighting frequency, illumination, temperature, humidity, etc.) The brain activity signal changes due to the influence of the external environment such as the measurement environment (for example, the accuracy and wearing state of the sensor).

That is, since the brain activity signal changes depending on factors other than the mental activity state to be calculated, the estimated value of the mental state calculated from the brain activity signal also changes depending on the factor, and the amount of change is the estimated value. And the true value of the mental activity state. That is, a biological signal that can always evaluate a certain mental activity state with high accuracy is not necessarily determined.

However, the technique described in Patent Document 1 does not assume that the error generated in the biological signal changes according to the internal environment or the external environment when the biological signal is measured. Therefore, one aspect of the present invention is to consider the influence of factors that cause an error between the estimated value of the mental activity state calculated from the brain activity signal and the true value of the mental activity state, The object is to select a brain activity signal that indicates a mental activity state with high accuracy from a plurality of types of brain activity signals.

In order to solve the above problems, one embodiment of the present invention employs the following configuration. A mental activity state evaluation support device for supporting evaluation of a mental activity state using a plurality of types of brain activity signals, comprising a processor and a storage device, wherein the storage device includes the plurality of types of brain activity. Correspondence between error measurement signal indicating degree of factor other than mental activity state changing signal and error due to factor of estimated value of mental activity state corresponding to each of plural types of brain activity signals Error calculation information is stored, and the processor acquires the error measurement signal, refers to the error calculation information, and corresponds to each of the plurality of types of brain activity signals from the acquired error measurement signal. An error due to the factor of the estimated value of the active activity state is calculated, a first type brain activity signal is selected from the plurality of types of brain activity signals based on the magnitudes of the calculated errors, and the first type Brain activity And outputs information indicating the issue, mental activity state evaluation support apparatus.

According to one aspect of the present invention, a brain activity signal that indicates a mental activity state with high accuracy can be selected from a plurality of types of brain activity signals.

Issues, configurations, and effects other than those described above will be clarified by the following description of the embodiments.

It is a schematic diagram which shows the structural example of the mental activity state evaluation system in Example 1. FIG. It is explanatory drawing which shows an example of the outline | summary of the attention state calculation process by the attention state calculation part in Example 1. FIG. 1 is a block diagram illustrating a configuration example of an information processing apparatus according to a first embodiment. FIG. 3 is a block diagram illustrating a configuration example of an attention state learning unit, an attention state calculation unit, and an attention state output unit in the first embodiment. 3 is an example of brain activity signal information in the first embodiment. 4 is an example of error measurement signal information in the first embodiment. It is an example of the attention state learning information in Example 1. 7 is an example of attention state learning feature amount information according to the first embodiment. 3 is an example of error learning information in the first embodiment. 6 is an example of error learning feature amount information in the first embodiment. 6 is an example of attention state learning accumulation information in the first embodiment. 6 is an example of error learning accumulation information in the first embodiment. 3 is an example of optimum signal learning information in the first embodiment. 7 is an example of attention state calculation feature amount information in the first embodiment. 3 is an example of attention state calculation information according to the first embodiment. 4 is an example of error calculation information in the first embodiment. 3 is an example of attention state storage information in the first embodiment. 3 is an example of error storage information in the first embodiment. It is an example of the optimal attention state selection information in Example 1. 6 is an example of optimal attention state information in the first embodiment. 6 is an example of error correction information in the first embodiment. 6 is a flowchart illustrating an example of processing by an attention state learning unit according to the first embodiment. 6 is a flowchart illustrating an example of processing by an attention state calculation unit according to the first embodiment. 6 is a flowchart illustrating an example of processing by an attention state output unit according to the first embodiment. It is explanatory drawing which shows an example of the correction process of the attention state learning accumulation | storage information in Example 2. FIG. It is a schematic diagram which shows the structural example of the mental activity state evaluation system in Example 4. FIG. It is a schematic diagram which shows the structural example of the mental activity state evaluation system in Example 5. FIG. 14 is an example of a measurement information display screen in Example 6. FIG. 10 is an example of a setting screen in Embodiment 6. FIG.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. It should be noted that this embodiment is merely an example for realizing the present invention, and does not limit the technical scope of the present invention. In each figure, the same reference numerals are given to common configurations.

This embodiment describes a method and system for evaluating an attention state, which is an example of a mental activity state, but this embodiment uses a process similar to the process shown below to perform a process other than the attention state such as mood and stress. It can be applied to the evaluation of various mental activities.

FIG. 1 shows a configuration example of a mental activity state evaluation system. The mental activity state evaluation system includes, for example, an information processing device 1, brain activity measurement devices 2 to 3, an error measurement device 4, an input device 5, and an output device 6. The information processing apparatus 1 includes an attention state learning unit 21, an attention state calculation unit 51, and an attention state output unit 71, and evaluates the user's attention state.

Each of the brain activity measuring device 2 and the brain activity measuring device 3 measures different types of brain activity signals for the information processing device 1 to evaluate the attention state. In this embodiment, the error measurement signal is also simply referred to as an error signal. The brain activity signal is a signal acquired from a living body for evaluating brain activity. The brain activity signal includes, for example, an electroencephalogram signal, an optical topography signal, a magnetoencephalogram signal, an fMRI signal, and a PET signal. For example, the brain activity measuring device 2 measures an electroencephalogram signal, and the brain activity measuring device 3 measures an optical topography signal.

The error measuring device 4 measures an error measurement signal for evaluating the error in the attention state. The error measurement signal is a signal that changes the brain activity signal and indicates the degree of a factor other than the mental activity state to be evaluated. The information processing apparatus 1 uses the error measurement signal to evaluate the error of the measured brain activity signal or the attention state error obtained from the measured brain activity signal and the influence of the factor causing the error. The error measurement signal includes, for example, electrooculogram, myoelectric potential, room temperature, and the user's posture.

The information processing apparatus 1 acquires a plurality of types of measured brain activity signals and a measured error measurement signal. The attention state learning unit 21 generates a relationship between each brain activity signal, the error measurement signal, each attention state, and each attention state error before calculating the attention state. The attention state calculation unit 51 calculates an error between each attention state and each attention state based on the relationship and each brain activity signal.

The attention state calculation unit 51 compares the errors of each attention state and selects the attention state that shows the most appropriate error, thereby performing a highly accurate attention state evaluation in which the influence of the error factor is suppressed. . The attention state output unit 71 outputs an error between the selected attention state and the attention state to the output device 6. Details of each device and terms in FIG. 1 are as follows.

The attention state is a state determined from a combination of the attention type and the attention level, for example. The attention type is synonymous with the type of attention defined academically, and includes selectivity attention, persistence attention, and diversion attention. The attention level is a value indicating the strength of the corresponding type of attention.

Note that mental activity states are defined for all mental activities as well as attention states. The mental activity state includes the type of the mental activity state and the intensity of the mental activity state. In addition, the kind of the said mental activity state may consist of one type.

The external signal is a signal that excludes the brain activity signal and the error measurement signal from the signals that control the operation of the information processing apparatus 1. The input device 5 is a device for inputting an external signal and includes, for example, an input device such as a mouse or a keyboard.

The output device 6 is, for example, a display device, a speaker, or a printer, and outputs the execution result of the program in a format that can be visually recognized by the operator. Further, the output device 6 may output a warning task. Details of the attention problem will be described later. An attention task is also simply called a task.

Although FIG. 1 illustrates an example in which the information processing device 1 is connected to two types of brain activity measurement devices and one type of error measurement device, the information processing device 1 can measure three or more types of brain activity. It may be connected, and may be connected to two or more kinds of error signal measuring devices. Hereinafter, in the present embodiment, the information processing apparatus 1 is connected to a plurality of brain activity measurement devices that measure different types of brain activity signals and one or more error measurement devices that measure different types of error measurement signals. It shall be.

The attention state learning unit 21 learns in advance the relationship between the brain activity signal, the attention state, and the attention state error in order to calculate the estimated value of the attention state and the attention state error with high accuracy. The attention state calculation unit 51 uses the brain activity signal, the attention state learning feature amount information, the error learning feature amount information, and the like to calculate the attention state estimation value and error in order to evaluate the attention state with high accuracy. . Details of the attention state learning feature amount information and the error learning feature amount information will be described later.

The attention state output unit 71 outputs the estimated value and error of the calculated attention state, attention state selection information, minimum attention error correction information, calculated feature amount group information, and the like. The information processing apparatus 1 may include other devices for the mental activity state evaluation system. In FIG. 1, the devices are connected to each other by wire, but may be connected to each other wirelessly via a network.

FIG. 2 shows an example of an outline of attention state calculation processing by the attention state calculation unit 51. The attention state calculation process 100 includes, for example, an attention level calculation process 101, an error calculation process 102, a one-dimensionalization process 103, a comparison process 104, and a determination process 105. FIG. 2 shows an example in which the information processing device 1 is connected only to the brain activity measuring device 2 and the brain activity measuring device 3.

2 The vertical axis of the graph in FIG. 2 indicates the index value of the brain activity signal, and the horizontal axis indicates the attention level. In the attention calculation process 101, the error calculation process 102, and the one-dimensionalization process 103, the upper graph shows the correlation between the index of the brain activity signal measured by the brain activity measuring device 2 and the attention, and the lower graph shows the brain activity. The correlation between the index of the brain activity signal measured by the measuring device 3 and the attention level is shown. The curves in the upper graph and the lower graph may be determined in advance or may be generated by learning. In this embodiment, it is assumed that the curve is predetermined.

The brain activity signal index is an index for evaluating the type of attention and the degree of attention, and is an example of a feature amount obtained from the brain activity signal. The brain activity signal itself is an example of an index of the brain activity signal. For example, the α wave power, β wave power, and P3 amplitude of the electroencephalogram are all indicators of the electroencephalogram signal. Hereinafter, the index of the brain activity signal is also simply referred to as an index.

First, the attention state calculation unit 51 calculates an estimated value of each attention level from each brain activity signal in the attention level calculation processing 101. Specifically, the attention state calculation unit 51 acquires brain activity signals at the same time from each of the brain activity measuring devices 2 to 3, and converts each acquired brain activity signal into a predetermined index. The attention state calculation unit 51 calculates the first estimated value of attention by substituting the index corresponding to the brain activity measuring device 2 into the function indicated by the curve in the upper graph in FIG. The second estimated value of the attention level is calculated by substituting the index to be inserted into the function indicated by the curve in the lower graph in FIG.

The attention state calculation unit 51 calculates an error of the estimated value of each attention level using an error function corresponding to each brain activity signal in the error calculation processing 102. Details of the error function determination method will be described later. The error function is a function for calculating an error range, and is a function having one or more error factors as independent variables and error as a dependent variable. The error function is, for example, a sigmoid function, but may be other functions.

In the one-dimensionalization process 103, the attention state calculation unit 51 represents each estimated value of the attention level and its error on a number line. Subsequently, in the comparison process 104, the attention state calculation unit 51 compares the error of each estimated value of the attention level, that is, the error indicated on the number line. Subsequently, the attention state calculation unit 51 determines the estimated value of the attention level by the determination process 105. In the example of the determination process 105 in FIG. 2, the attention state calculation unit 51 determines the first estimated value having a smaller error range as the estimated value of the attention level.

The attention state calculation unit 51 calculates an estimated value of each attention level and its error with respect to a plurality of types of brain activity signals, makes each error one-dimensional, and compares the magnitude of the error. The estimated value of attention can be determined with high accuracy in consideration of the influence. As a result, the attention state calculation unit 51 can suppress the influence of the error factor and can accurately evaluate the attention level.

FIG. 3A shows a configuration example of the information processing apparatus 1. The information processing apparatus 1 receives input of different types of brain activity signals and error measurement signals acquired by the brain activity measuring apparatus 2 and the brain activity measuring apparatus 3. The information processing apparatus 1 performs attention state learning and calculation processing using the input signal, stores the learned or calculated information, and stores the processing result and the information stored in the information processing apparatus 1 to the output device 6. Is output.

The information processing apparatus 1 is configured by, for example, a computer having an input unit 10, a calculation unit 20, a storage unit 80, and an output unit 15. The information processing apparatus 1 is a computer system configured on a plurality of computers that are physically configured on one computer or logically or physically, and operates on separate threads on the same computer. It may be possible to operate on a virtual machine constructed on a plurality of physical computer resources.

The input unit 10 is connected to another device and receives inputs such as brain activity signals, error measurement signals, and external signals from the other connected devices. The computing unit 20 is, for example, a CPU (Central Processing Unit) including a processor. The calculation unit 20 executes a program stored in the storage unit 80 and has a calculation function in the information processing apparatus 1.

The storage unit 80 stores a calculation result by the calculation unit 20 and a program for realizing each unit included in the calculation unit 20, and includes, for example, a memory and an auxiliary storage device. The memory includes a ROM that is a nonvolatile storage element and a RAM that is a volatile storage element. The ROM stores an immutable program (for example, BIOS). The RAM is a high-speed and volatile storage element such as a DRAM (Dynamic Random Access Memory), and temporarily stores a program executed by the processor and data used when the program is executed.

The auxiliary storage device is a large-capacity non-volatile storage device such as a magnetic storage device (HDD) or a flash memory (SSD), and stores a program executed by the processor and data used when the program is executed. That is, the program is read from the auxiliary storage device, loaded into the memory, and executed by the processor.

The output unit 15 outputs the calculation result by the calculation unit 20 and the information stored in the storage unit 80 to another device or a display. The output unit 15 is, for example, an output interface or a communication interface. The output interface includes VGA and HDMI. The communication interface is a network interface device that controls communication with other devices in accordance with a predetermined protocol.

The program executed by the processor is provided to the information processing apparatus 1 via a removable medium (CD-ROM, flash memory, etc.) or a network, and is stored in a nonvolatile auxiliary storage device that is a non-temporary storage medium. For this reason, the information processing apparatus 1 may have an interface for reading data from a removable medium.

The input unit 10 includes a plurality of brain activity signal input units 11, one or more error measurement signal input units 13, and an external signal input unit 14. Each unit included in the input unit 10 includes, for example, an input interface or a communication interface. The input interface includes a serial interface that accepts information input from other devices such as USB (Universal Serial Bus).

Each brain activity signal input unit 11 is connected to, for example, one brain activity measuring device, and receives an input of a brain activity signal from the brain activity measuring device. Each error measurement signal input unit 13 is connected to, for example, one error measurement device, and receives an input of an error measurement signal from the error measurement device. The external signal input unit 14 is connected to the input device 5 and receives an input of an external signal.

The calculation unit 20 includes an attention state learning unit 21, an attention state calculation unit 51, and an attention state output unit 71. For example, the arithmetic unit 20 operates as the attention state learning unit 21 by operating according to the attention state learning program loaded in the storage unit 80, and operates according to the attention state calculation program loaded in the storage unit 80. It functions as the attention state calculation unit 51. The same applies to the other units included in the calculation unit 20.

The storage unit 80 includes brain activity signal information 81, error measurement signal information 82, attention state learning information 83, attention state learning feature information 84, error learning information 85, and error learning features, which are created by the attention state learning unit 21, respectively. Quantity information 86, attention state learning accumulation information 87, error learning accumulation information 88, and optimum signal information 89 are stored.

In addition, the storage unit 80 calculates the calculated feature value group information 90, the attention state calculation information 91, the error calculation information 92, the attention state storage information 93, the optimum attention state selection information 95, and the optimum, which are respectively calculated by the attention state calculation unit 51. Attention state information 96 and error correction information 97 are stored. Details of each piece of information stored in the storage unit 80 will be described later.

In this embodiment and other embodiments, the information used by the mental activity state evaluation system may be expressed in any data structure without depending on the data structure. For example, a data structure appropriately selected from a table, list, database or queue can store the information. In this embodiment, an example in which information is stored in a table will be described.

FIG. 3B shows a configuration example of the attention state learning unit 21, the attention state calculation unit 51, and the attention state output unit 71. The attention state learning unit 21 includes a learning brain activity signal acquisition unit 22, a learning error measurement signal acquisition unit 24, an attention state learning information generation unit 25, an error learning information generation unit 26, an attention state learning feature amount calculation unit 29, and an error learning feature. An amount calculation unit 30, an attention state learning feature amount storage unit 33, an error learning feature amount storage unit 34, and an optimum signal learning unit 37 are included.

The learning brain activity signal acquisition unit 22 acquires brain activity signals input to the brain activity signal input units 11 during attention state learning, and creates brain activity signal information 81 corresponding to each brain activity signal during attention state learning. To do. The learning error measurement signal acquisition unit 24 acquires an error measurement signal input to each of the error measurement signal input units 13 during attention state learning, and creates error measurement signal information 82 corresponding to each error measurement signal during attention state learning. To do.

The attention state learning information creation unit 25 creates attention state learning information 83 corresponding to each brain activity signal by using the brain activity signal information 81 corresponding to each brain activity signal at the time of attention state learning. The error learning information creation unit 26 creates error learning information 85 corresponding to each brain activity signal using the brain activity signal information 81 and the error measurement signal information 82 corresponding to each brain activity signal at the time of attention state learning.

The attention state learning feature amount calculation unit 29 creates attention state learning feature amount information 84 corresponding to each brain activity signal, using the attention state learning information 83 corresponding to each brain activity signal. The error learning feature amount calculation unit 30 creates error learning feature amount information 86 corresponding to each brain activity signal by using the attention state learning feature amount information 84 and the error learning information 85 corresponding to each brain activity signal.

The attention state learning feature amount storage unit 33 uses the attention state learning feature amount information 84 corresponding to each brain activity signal to create attention state learning storage information 87 corresponding to each brain activity signal. The error learning feature amount accumulation unit 34 uses the error learning feature amount information 86 corresponding to each brain activity signal to create error learning accumulation information 88 corresponding to each brain activity signal. The optimum signal learning unit 37 creates optimum signal information 89 using the error learning accumulation information 88 and the error learning feature amount information 86.

The attention state calculation unit 51 includes a calculated brain activity signal acquisition unit 52, a calculation error measurement signal acquisition unit 54, a calculated feature amount group calculation unit 55, an attention state calculation information creation unit 58, an error calculation unit 59, and an attention state information storage unit 62. , An error information storage unit 63, an optimal attention state calculation unit 64, an optimal attention state selection unit 65, and an optimal attention error correction information calculation unit 66.

The calculated brain activity signal acquisition unit 52 acquires the brain activity signal input to each brain activity signal input unit 11 at the time of attention state calculation, and creates brain activity signal information 81 of each brain activity signal at the time of attention state calculation. The calculation error measurement signal acquisition unit 54 acquires the error measurement signal input to each of the error measurement signal input units 13 at the time of attention state calculation, and creates error measurement signal information 82 corresponding to each error measurement signal at the time of attention state calculation. To do.

The calculated feature value group calculation unit 55 creates calculated feature value group information 90 of each brain activity signal by using the brain activity signal information 81 of each brain activity signal at the time of attention state calculation. The attention state calculation information creation unit 58 creates attention state calculation information 91 corresponding to each brain activity signal by using the calculated feature amount group information 90 and attention state learning accumulation information 87 corresponding to each brain activity signal.

The error calculation unit 59 includes error learning accumulation information 88 corresponding to each brain activity signal, calculated feature value group information 90 corresponding to each brain activity signal, and each error measurement signal information 82 created by the calculation error measurement signal acquisition unit 54. Is used to create error calculation information 92 corresponding to each brain activity signal. The attention state information storage unit 62 uses the attention state calculation information 91 calculated by the attention state calculation information generation unit 58 to create the attention state storage information 93. The error information storage unit 63 uses the error calculation information 92 calculated by the error calculation unit 59 to create error storage information 94.

The optimal attention state selection unit 65 creates the optimal attention state selection information 95 using the error storage information 94. The optimal attention state calculation unit 64 creates optimal attention state information 96 using the attention state storage information 93 and the optimal attention state selection information 95. The optimal attention error correction information calculation unit 66 creates error correction information 97 using the optimal signal information 89 and the optimal attention state information 96.

The attention state output unit 71 includes, for example, an optimal attention state output unit 72, an optimal attention state selection information output unit 73, an error information output unit 74, and an optimal attention error correction information output unit 75. The optimal attention state output unit 72 outputs all or a part of the optimal attention state information 96 to the output unit 15. The optimal attention state selection information output unit 73 outputs all or part of the optimal attention state selection information 95 to the output unit 15.

The error information output unit 74 outputs all or part of the error storage information 94 to the output unit 15. The optimal attention error correction information output unit 75 outputs the determination result included in the error correction information 97 or information corresponding to the determination result to the output unit 15. The instruction for suppressing the influence of the error factor is an example of information corresponding to the determination result. Information corresponding to the determination result is stored in the storage unit 80 in advance and is acquired by the optimum attention error correction information output unit 75, for example. The attention state output unit 71 may output information created by the attention state learning unit 21 or the attention state calculation unit 51 other than the above output information.

4A to 4I are examples of information created by the attention state learning unit 21. FIG. FIG. 4A is an example of brain activity signal information 81. The brain activity signal information 81 includes, for example, a brain activity signal name column 811, a time column 812, and a signal column 813.

The brain activity signal name column 811 stores the name of the brain activity signal. The time column 812 stores the time when the signal is acquired. In the present embodiment, the elapsed time from the start of learning at the time of attention state learning, and the elapsed time from the start of calculation at the time of calculating the attention state are set as the time. The signal column 813 stores, for example, a numerical value obtained by A / D (analog / digital) conversion of the signal input to the brain activity signal input unit 11.

FIG. 4B is an example of the error measurement signal information 82. The error measurement signal information 82 includes, for example, an error signal name column 821, a time column 822, and a signal column 823. The error signal name column 821 stores the name of the error measurement signal.

The description of the time column 822 is the same as the description of the time column 812, and will not be repeated. Hereinafter, description of columns included in each piece of different information, such as the time column, will be made only when the column first appears. The signal column 823 stores, for example, the value of the signal input to the error measurement signal input unit 13 or a numerical value obtained by A / D converting the signal.

FIG. 4C is an example of the attention state learning information 83. The attention state learning information 83 includes, for example, an attention type field 831, an attention level field 832, a brain activity signal name field 833, a time field 834, and a signal field 835. The attention type column 831 stores attention types including persistence attention, selectivity attention, convertibility attention, distributive attention, and the like. The attention level column 832 stores the attention level. The degree of attention is given, for example, as a value between 0 and 100, and the greater the value, the stronger the degree of attention. Each cell in the attention type column 831 may include a plurality of attention types. At this time, the corresponding cell in the attention level column 832 stores the attention level corresponding to each of the attention types.

FIG. 4D is an example of attention state learning feature amount information 84. The attention state learning feature amount information 84 includes, for example, an attention type field 841, an attention level field 842, a brain activity signal name field 843, a time field 844, a signal field 845, and an index field 846. The index column 846 stores the index value of the corresponding brain activity signal.

Note that when a plurality of types of indices are calculated, the attention state learning feature amount information 84 includes an index column 846 corresponding to each of the plurality of indices. Each cell in the attention type column 841 may include a plurality of attention types. At this time, the corresponding cell in the attention level column 842 stores the attention level corresponding to each of the attention types.

FIG. 4E is an example of error learning information 85. The error learning information 85 includes, for example, a time column 851, a brain activity signal name column 852, a brain activity signal column 853, an error signal name column 854, and an error signal column 855. When a plurality of types of error measurement signals are input, the error learning information 85 includes an error signal name column and an error signal column corresponding to each of the plurality of types of error measurement signals.

FIG. 4F is an example of the error learning feature amount information 86. The error learning feature amount information 86 includes, for example, a time column 861, a brain activity signal name column 862, a brain activity signal column 863, a brain activity signal error range column 864, an index column 865, an index error range column 866, and an error signal name column 867. And an error signal column 868. The brain activity signal error range column 864 stores a value indicating the range of the brain activity signal that may vary due to the error. The index error range column 866 stores a value indicating a range of index values that can vary due to an error.

When the error ranges of a plurality of types of indicators are calculated, the error learning feature amount information 86 includes an index column 865 and an index error range column 866 corresponding to each of the plurality of indicator columns. When a plurality of types of error measurement signals are input, the error learning feature amount information 86 includes an error signal name column 867 and an error signal column 868 corresponding to each of the plurality of types of error measurement signals.

FIG. 4G is an example of the attention state learning accumulation information 87. The attention state learning accumulation information 87 includes, for example, a brain activity signal name field 871, an attention type field 872, and an attention state coefficient field 873. The attention state coefficient column 873 indicates coefficients in a predetermined relational expression for calculating the degree of attention from the index. When the predetermined relational expression is the linear function y = ax, the attention state coefficient is a.

One cell in the attention state coefficient column 873 may store a plurality of attention state coefficients. Specifically, for example, when the predetermined relational expression is a quadratic function y = ax ² + bx + c, there are three attention state coefficients a, b, and c. At this time, the attention state coefficient column 873 stores the three attention state coefficients in a format that can specify which of the three attention state coefficients corresponds to a, b, and c. Each cell in the attention type column 872 may include a plurality of attention types. At this time, the corresponding cell in the attention state coefficient column 873 stores the attention state coefficient corresponding to each of the attention types.

FIG. 4H is an example of the error learning accumulation information 88. The error learning accumulation information 88 includes, for example, a brain activity signal name column 881, an error signal name column 882, an error factor column 883, and an error coefficient column 884. The error factor column 883 stores a value indicating a factor that affects the error in the evaluation of the attention state. Brain activities other than attention, personal characteristics, other biological activities, work environment, and surrounding environment are all examples of error factors.

The error coefficient column 884 stores a coefficient of a predetermined relational expression for calculating an error from the error signal. Similarly to the attention state coefficient column 873, the error coefficient column 884 may store a plurality of error coefficients in one cell. When a plurality of types of error measurement signals are input, the error learning feature amount information 86 includes error signal name columns 882 corresponding to the plurality of types of error measurement signals.

FIG. 4I is an example of the optimum signal information 89. The optimum signal information 89 includes, for example, a time field 891, an error factor field 892, and brain activity signal fields 893 having the same number as the number of types of brain activity signals input to the brain activity signal input unit 11. The brain activity signal column 893 stores information about each brain activity signal, and includes, for example, a brain activity signal name column 894, an error column 895, and a cumulative error column 896. The error factor column 892 may store a plurality of error factors.

The error column 895 stores the error of the corresponding brain activity signal. When the error factor column 892 stores a plurality of error factors, each brain activity signal column 893 includes an error column corresponding to each error factor. The accumulated error column 896 stores the sum of errors corresponding to each error factor. Since there is only one error column 895 included in the brain activity signal column 893, the value stored in the total error column 896 and the error column 895 is the same. The last row of each error column 895 and each cumulative error column 896 stores the total value of the values included in the error column 895 and the total error column 896.

5A to 5H are examples of information created by the attention state calculation unit 51. FIG. FIG. 5A is an example of the calculated feature amount group information 90. The calculated feature amount group information 90 includes, for example, attention state calculated feature amount information 901 and error calculated feature amount information 902.

The attention state calculation feature amount information 901 includes, for example, a brain activity signal name field 9011, a time field 9012, a signal field 9013, and an index field 9014. The error calculation feature quantity information 902 includes, for example, a time column 9021, a brain activity signal name column 9022, a brain activity signal column 9023, a brain activity signal error range column 9024, an index column 9025, and an index error range column 9026.

FIG. 5B is an example of the attention state calculation information 91. The attention state calculation information 91 includes, for example, a time field 911, a brain activity signal name field 912, an attention type field 913, and an attention level field 914.

FIG. 5C is an example of the error calculation information 92. The error calculation information 92 includes, for example, a time column 921, a brain activity signal name column 922, a cumulative error column 923, an error factor column 924, and an error column 925.

FIG. 5D is an example of attention state storage information 93. The attention state storage information 93 includes, for example, a time column 931 and the same number of brain activity signal columns 932 as the number of types of brain activity signals input to the brain activity signal input unit 11. The brain activity signal column 932 stores information about each brain activity signal, and includes, for example, a brain activity signal name column 933, an attention type column 934, and an attention level column 935.

FIG. 5E is an example of the error storage information 94. The error storage information 94 includes, for example, a time column 941, an error factor column 942, and brain activity signal columns 943 of the same number as the types of brain activity signals input to the brain activity signal input unit 11. The brain activity signal column 943 stores information about each brain activity signal, and includes, for example, a brain activity signal name column 944, an error column 945, and a cumulative error column 946.

FIG. 5F is an example of the optimal attention state selection information 95. The optimal attention state selection information 95 includes, for example, a time field 951, an optimal signal name field 952, an optimal error field 953, and the same number of brain activity signals as the number of types of brain activity signals input to the brain activity signal input unit 11. Column 954.

The optimal signal name column 952 stores any name of the brain activity signal input to the brain activity signal input unit 11. The optimum error column 953 stores the error of the signal stored in the optimum signal name column 952. The brain activity signal column 954 stores information about each brain activity signal, and includes, for example, a brain activity signal name column 955 and a cumulative error column 956.

FIG. 5G is an example of the optimal attention state information 96. The optimal attention state information 96 includes, for example, a time field 961, an optimal signal name field 962, an attention type field 963, an attention level field 964, and an optimal error field 965.

FIG. 5H is an example of error correction information 97. The error correction information 97 includes, for example, a time column 971, a learning optimum signal column 972, a calculated optimum signal column 973, and a determination result column 974. The learning optimum signal column 972 stores the signal name of the brain activity measurement signal indicating the optimum error at the time of attention state learning. The calculated optimum signal column 973 stores the signal name of the brain activity measurement signal indicating the optimum error when calculating the attention state. The determination result column 974 stores a determination result indicating whether the brain activity signal names stored in the learning optimal signal column 972 and the calculated optimal signal column 973 match. For example, the determination result when the brain activity signals are the same is “OK”, and the determination result when the brain activity signals are different is “NG”.

Hereinafter, an operation example of the information processing apparatus 1 will be described. FIG. 6 shows an example of processing by the attention state learning unit 21. Hereinafter, the brain activity signal input unit 11 receives the first brain activity signal to the mth brain activity signal (m is an integer equal to or greater than 2), and the error measurement signal input unit 13 receives the first error measurement signal to the nth. An error measurement signal (n is an integer of 1 or more) is input. The type of brain activity signal input to each brain activity signal input unit 11 and the type of error measurement signal input to each error measurement signal input unit 13 are determined in advance.

Further, the brain activity signal information 81 corresponding to the i-th brain activity signal (i is an integer of 1 to m) is referred to as i-th brain activity signal information 81. Other information generated for each brain activity signal is also expressed in the same manner. The information generated for each error measurement signal is also expressed in the same manner. Unless otherwise specified, in the description of FIG. 6, the brain activity signal information 81 and the error measurement signal information 82 indicate the brain activity signal information 81 and the error measurement signal information 82 generated during attention state learning.

In step 121, for example, a task is presented to the user, and the user performs a specific action corresponding to the task. It is desirable that the task changes the attention state of the user when the user performs a specific action corresponding to the task. The Stroop task is an example of the task, and the answer to the Stroop task is an example of the specific action.

Note that, for example, another user such as an expert may present the assignment manually. In addition, for example, the storage unit 80 holds task information including one or more tasks, and the attention state learning unit 21 acquires one task that is arbitrarily selected or designated by another user such as an expert from the task information. Then, it may be transmitted to the attention state output unit 71, and the attention state output unit 71 may output the problem to the output device 6 to present the problem.

Note that the task information may further include information indicating the type of attention corresponding to each task and the level of attention corresponding to each task. The information indicating each degree of attention may be the degree of attention itself with respect to the corresponding task, or may be a function for calculating the degree of attention from the result of the user regarding the task.

The attention state learning unit 21 sets the output time of the task as the learning start time only in the first step 121, for example. The learning brain activity signal acquisition unit 22 acquires various brain activity signals input to the brain activity signal input unit 11, and the learning error measurement signal acquisition unit 24 inputs various error signals input to the error measurement signal input unit 13. Is acquired simultaneously with the learning start time set, and is executed until the specific action is completed. The attention state learning unit 21 receives, for example, an end notification of the specific action from the user.

In step 122, the attention state learning information creating unit 25 acquires the attention state corresponding to the task presented in step 121. The attention state learning information creation unit 25 may acquire an attention state input from another user such as an expert, or may acquire an attention state from the task information. Further, a plurality of attention states, that is, a plurality of types of attentions and attention levels of the respective attention types may correspond to one task.

In step 123, the learning brain activity signal acquisition unit 22 creates brain activity signal information 81 corresponding to each brain activity signal in the task presented in step 121. Specifically, the learning brain activity signal acquisition unit 22 acquires a brain activity signal from each brain activity signal input unit 11 and acquires a brain activity signal name associated with each brain activity signal input unit 11 in advance.

The learning brain activity signal acquisition unit 22 creates empty brain activity signal information 81 for each brain activity signal, and acquires the acquired brain activity signal name, the acquisition time of the brain activity signal, and the acquired brain activity signal as A. Each / D converted value is stored in the brain activity signal name column 811, the time column 812, and the signal column 813 of the brain activity signal information 81.

In step 124, for example, the learning brain activity signal acquisition unit 22 determines whether or not all brain activity signal information 81 for all tasks has been created. Specifically, for example, the learning brain activity signal acquisition unit 22 receives a task presentation end notification from a user or the like, and determines whether or not brain activity signal information 81 for all the presented tasks has been generated. Further, the learning brain activity signal acquisition unit 22 may determine whether all the tasks included in the task information are presented and the brain activity signal information 81 for all the presented tasks is generated.

When the learning brain activity signal acquisition unit 22 has created all the brain activity signal information 81 for all the tasks (step 124: YES), the learning brain activity signal acquisition unit 22 proceeds to step 125, and when there is uncreated brain activity signal information 81 (step 124: NO) returns to step 121. Note that other units included in the attention state learning unit 21 may execute the determination process of step 124. Hereinafter, the task presented in the loop process including the steps 121 to 124 in the k-th cycle is referred to as a k-th task.

In step 125, the learning brain activity signal acquisition unit 22 sets the value of the counter i to 1. In step 126, the learning brain activity signal acquisition unit 22 sets the value of the counter j to 1. In step 127, the learning brain activity signal acquisition unit 22 sets the value of the counter k to 1. Note that any unit included in the attention state learning unit 21 may set the value of each counter.

In step 128, the attention state learning information creation unit 25 determines whether there is a k-th task. If the attention state learning information creation unit 25 determines that there is the k-th task (step 128: YES), the attention state learning information creation unit 25 proceeds to step 129. Other units included in the attention state learning unit 21 may execute the determination process of step 128.

In step 129, the attention state learning information creating unit 25 determines whether or not there is the i-th brain activity signal information 81. The attention state learning information creation unit 25 proceeds to step 130 if it determines that there is the i-th brain activity signal information 81 (step 129: YES), and proceeds to step 141 if it determines that there is not (step 129: NO). Other units included in the attention state learning unit 21 may perform the determination process in step 129.

Note that the information described in steps 129 to 140 is information on the k-th issue unless otherwise specified. For example, the i-th attention state learning information 83 in step S130 indicates the i-th attention state learning information 83 in the k-th task.

In step 130, the attention state learning information creation unit 25 generates the i-th attention state learning information 83. Specifically, the attention state learning information creating unit 25 extracts the brain activity signal name field 811, the time field 812, and the signal field 813 from the i-th brain activity signal information 81, thereby obtaining the i-th attention state learning information 83. The brain activity signal name column 833, the time column 834, and the signal column 835 are obtained.

Subsequently, the attention state learning information creating unit 25 sets the attention type and the attention level included in the attention state of the kth task acquired in step S122, respectively, and the attention type column 831 and the attention level of the i-th attention state learning information 83, respectively. Store in column 832.

In step 131, the attention state learning feature amount calculation unit 29 creates the i-th attention state learning feature amount information 84. Specifically, for example, the attention state learning information creation unit 25 extracts the attention type field 831, the attention level field 832, the brain activity signal name field 833, the time field 834, and the signal field 835 from the i-th attention state learning information 83. By extracting, the values of the attention type column 841, attention level column 842, brain activity signal name column 843, time column 844, and signal column 845 of the i-th attention state learning feature amount information 84 are obtained.

Also, the attention state learning feature amount calculation unit 29 calculates an index value by performing a predetermined process on the value of the signal field 845 at each time, and stores the calculated index value in the index field 846. The attention state learning feature amount calculation unit 29 calculates each index value by performing signal processing for reducing noise such as a bandpass filter and trend removal, for example. Note that the attention state learning feature amount calculation unit 29 may calculate a plurality of types of indices.

In step 132, the attention state learning feature amount storage unit 33 creates the i-th attention state learning storage information 87. Specifically, for example, the attention state learning feature amount accumulating unit 33 extracts the brain activity signal name field 843 and the attention type field 841 from the i-th attention state learning feature amount information 84 to thereby obtain the i-th attention state learning. The values of the brain activity signal name column 871 and the attention type column 872 of the accumulated information 87 are obtained. The attention state learning feature amount accumulating unit 33 deletes a record having overlapping attention types stored in the attention type column 872 of the i-th attention state learning accumulation information 87.

Subsequently, the attention state learning feature amount accumulation unit 33 performs attention state coefficient calculation processing for each attention type stored in the attention type column 872. Specifically, the attention state learning feature amount storage unit 33, for example, a predetermined relational expression with unknown coefficients having an index as an independent variable and an attention level as a dependent variable, and the attention of the i-th attention state learning feature amount information 84. The coefficient of the relational expression is determined by the least square method using the values of the type attention level column 842 and the index column 846, and the determined coefficient is determined as the attention state coefficient of the type of attention. Note that the predetermined relational expression may be different for each attention type, for example.

In step 133, the learning error measurement signal acquisition unit 24 creates the j-th error measurement signal information 82 in the k-th task by the same method as the creation method of each brain activity signal information 81 in step S123. In step 134, the learning error measurement signal acquisition unit 24 increments the value of the counter j by 1.

In step 135, the learning error measurement signal acquisition unit 24 determines whether or not the jth error measurement signal information 82 exists. If the learning error measurement signal acquisition unit 24 determines that there is the j-th error measurement signal information 82 (step S135: YES), the learning error measurement signal acquisition unit 24 returns to step 133; .

In step 136, the error learning information creation unit 26 sets the value of the counter j to 1. In step 137, the error learning information creation unit 26 creates i-th error learning information 85. Specifically, the error learning information creation unit 26 merges the i-th error learning information 85 and the first error measurement signal information 82 to the j-th error measurement signal information 82, for example, using the time column as a key. Thus, the i-th error learning information 85 is created.

In step 138, the error learning feature amount calculation unit 30 creates the i-th error learning feature amount information 86. Specifically, the error learning feature amount calculation unit 30 uses the time column as a key for the i-th error learning information 85 and a table including the time column 844 and the index column 846 of the i-th attention state learning feature amount information 84. Are obtained in the time column 861, the brain activity signal name column 862, the brain activity signal column 863, the index column 865, the error signal name column 867, and the error signal column 868.

Subsequently, for each time, the error learning feature amount calculation unit 30 calculates, for example, the standard deviation of the value of the brain activity signal column 863 in a predetermined time window including the time, and calculates the calculated standard deviation at the time. The brain activity signal error range is determined and stored in the brain activity signal error range column 864. Similarly, the error learning feature amount calculation unit 30 calculates, for each time, for example, the standard deviation of the value in the index field 865 in a time window of a predetermined range including the time, and uses the calculated standard deviation as the index error at the time. The range is determined and stored in the index error range column 866.

In step 139, the error learning feature amount accumulation unit 34 creates the i-th error learning accumulation information 88. Specifically, the error learning feature amount storage unit 34 acquires the values of the brain activity signal name column 862 and the error signal name column 867 of the i-th error learning feature amount information 86.

Subsequently, the error learning feature amount accumulation unit 34 specifies an error factor corresponding to the acquired error signal name. When one type of error measurement signal is used, the error learning feature amount accumulation unit 34 acquires, for example, one or more predetermined error factors corresponding to the error measurement signal. For example, an eye movement that is an error factor corresponds to an electrooculogram that is an error measurement signal. Further, for example, the eye movement that is an error factor and the muscle activity may correspond to the electrooculogram that is the error measurement signal.

When a plurality of types of error measurement signals are used, the error learning feature amount accumulation unit 34 acquires, for example, one or more predetermined error factors corresponding to the combination of the plurality of types of error measurement signals. .

In addition, when a plurality of types of error measurement signals are used, the error learning feature amount accumulation unit 34 may specify an error factor using, for example, a clustering method. Specifically, the error learning feature amount accumulating unit 34 obtains the values of the plurality of types of error signals at each time from the error signal column 868, and obtains a vector at each time having each type of error signal value as an element. Generate. For example, a vector space composed of each type of error signal is classified into a plurality of clusters in advance, and an error factor is associated with each cluster in advance. The error learning feature amount storage unit 34 specifies one or more error factors according to the cluster to which each generated vector belongs.

The error learning feature amount accumulation unit 34 stores each of the identified one or more error factors in the error factor column 883. In addition, the error learning feature amount accumulation unit 34 stores the acquired brain activity signal name and error signal name in the brain activity signal name column 881 and the error signal name column 882, respectively.

Subsequently, the error learning feature amount accumulating unit 34 calculates an error coefficient. Specifically, the error learning feature amount accumulating unit 34, for example, a predetermined relational expression with unknown coefficients having an error signal as an independent variable, a brain activity signal error range or an index error range error as a dependent variable, and the i-th error learning. The coefficient of the relational expression is determined by the least square method using the values of the brain activity signal error range column 864 and the index error range column 866 or the error signal column 868 of the feature amount information 86, and the determined coefficient is an error. Decide on a coefficient. Note that the error learning feature amount accumulating unit 34 may calculate the error coefficient using, for example, a different relational expression for each error factor.

In step 140, the error learning feature value accumulating unit 34 increments the value of the counter j by one. In step 141, the attention state learning information creating unit 25 increments the value of the counter k by 1.

In step 142, the optimum signal learning unit 37 creates optimum signal information 89. The optimum signal learning unit 37 creates optimum signal information 89 by performing, for example, the following processing for all integers i of 1 to m.

The optimal signal learning unit 37 creates an i-th table in which i-th error learning feature amount information 86 is vertically combined for all the tasks. The optimum signal learning unit 37 stores the value of the time field 861 of the i-th table in the time field 891. It should be noted that it may be performed only in the first process concerned with storing a value in the time column 891. The optimum signal learning unit 37 specifies one blank brain activity signal field 893 and stores the name of the i-th brain activity signal in the brain activity signal name field 894 included in the specified brain activity signal field 893.

The optimum signal learning unit 37 performs the following processing for each record in the i-th table. The optimum signal learning unit 37 acquires the time, the error signal name, and the error signal value from the time field 861, the error signal name field 867, and the error signal field 868, respectively. The optimum signal learning unit 37 acquires an error factor and an error coefficient corresponding to the acquired error signal name from the i-th error learning accumulation information 88. The optimum signal learning unit 37 substitutes the error coefficient corresponding to the acquired error factor and the value of the acquired error signal into a predetermined relational expression corresponding to the error factor for each acquired error factor, An error value corresponding to the error factor is calculated.

The optimum signal learning unit 37 stores each calculated error value in the cell corresponding to the time in the error column 895 corresponding to the i-th brain activity signal, and the error value in the error column 895 corresponding to the i-th brain activity signal. The sum is stored in the cell corresponding to the time in the accumulated error column 896 corresponding to the i-th brain activity signal.

The optimum signal learning unit 37 adds the record indicating the total value to the lowest stage of the optimum signal information 89, for example, after executing the above-described processing for all the integers i, and stores each error column 895 and each accumulated error column. The total value of the values in each error column 895 and each cumulative error column 896 is stored in a cell in the added record 896.

When the unit or dimension differs between the error calculated using the error coefficient and the attention level calculated using the attention state coefficient, the optimum signal learning unit 37, for example, uses a predetermined relational expression. The error unit and dimension may be matched with the unit and dimension of attention level by converting the error in each error column 895 and each accumulated error column 896 of the optimum signal information 89.

As described above, the attention state learning unit 21 acquires a plurality of brain activity signals and one or more error measurement signals, and acquires the relationship between each brain activity signal, the attention state, and the attention state error. In addition, the attention state learning unit 21 can acquire an optimal error and a brain activity signal indicating the optimal error by using the optimal signal information 89, and thus a brain activity signal that can calculate a highly accurate estimated value of the attention state. Can be obtained.

FIG. 7 shows an example of processing by the attention state calculation unit 51. Hereinafter, the first brain activity signal to the mth brain activity signal (m is an integer of 2 or more) are input to the brain activity signal input unit 11, and the error measurement signal input unit 13 is input as in the description of FIG. The first error measurement signal to the nth error measurement signal (n is an integer of 1 or more) are input. Unless otherwise specified, in the description of FIG. 7, the brain activity signal information 81 and the error measurement signal information 82 indicate the brain activity signal information 81 and the error measurement signal information 82 generated when the attention state is calculated.

In step 151, the calculated brain activity signal acquisition unit 52 determines whether or not a brain activity signal is input to the brain activity signal input unit 11. If the calculated brain activity signal acquisition unit 52 determines that it has been input (step S151: YES), it proceeds to step 152. If it is determined that it has not been input (step S151: NO), the process of step 151 is executed again. The

In step 151, for example, the calculation error measurement signal acquisition unit 54 may determine whether or not an error measurement signal is input to the error measurement signal input unit 13. Further, for example, if signals are not input to all the brain activity signal input units 11 and the error measurement signal input units 13, the process of step 151 may be executed again.

In step 152, the calculated brain activity signal acquisition unit 52 creates brain activity signal information 81 corresponding to each brain activity signal, and the calculated error measurement signal acquisition unit 54 creates error measurement signal information 82 corresponding to each error measurement signal. To do.

In step 153, the calculated brain activity signal acquisition unit 52 generates the brain activity signal information 81 for all the brain activity signals, and the calculated error measurement signal acquisition unit 54 determines the error measurement signal for all the error measurement signals. It is determined whether or not the information 82 has been created. If it is determined that all signal information has been created (step S153: YES), the process proceeds to step 154. If it is determined that there is signal information that has not been created (S153: NO), the process returns to step 151.

In step 151, if no signal is input to both the brain activity signal input unit 11 and the error measurement signal input unit 13, when step 151 is repeated again, step 153 may be always followed by step 154.

In step 154, the error calculation feature amount acquisition unit 57 sets the value of the counter i to 1. It should be noted that any unit included in the attention state calculation unit 51 may set the counter value.

In step 155, the calculated feature value group calculation unit 55 determines whether or not there is the i-th brain activity signal information 81. If there is i-th brain activity signal information (step 155: YES), the process proceeds to step 156. If not (step 155: NO), the process proceeds to step 160. The processing in step 156 may be performed by another unit included in the attention state calculation unit 51.

In step 156, the calculated feature value group calculation unit 55 creates the i-th calculated feature value group information 90, that is, the i-th attention state calculated feature value information 901 and the i-th error calculated feature value information 902. Specifically, for example, the calculated feature value group calculation unit 55 calculates an index value corresponding to the value from each value in the signal column 813 of the i-th brain activity signal information 81 by the same method as in step 131. The calculated feature quantity group calculation unit 55 creates the i-th attention state calculated feature quantity information 901 by adding a column for storing the calculated index value to the i-th brain activity signal information 81.

Also, the calculated feature quantity group calculation unit 55 calculates the range of the brain activity signal error from each value in the signal column 813 of the i-th brain activity signal information 81 by the same method as in step 138. Further, the calculated feature quantity group calculation unit 55 calculates an index error range from each of the calculated index values by the same method as in step 138. The calculated feature quantity group calculation unit 55 includes, in the i-th brain activity signal information 81, a column for storing the calculated brain activity signal error range, a column for storing the calculated index value, and a column for storing the calculated index error range. By adding, the i-th error calculation feature quantity information 902 is created.

In step 157, the attention state calculation information creating unit 58 creates the i-th attention state calculation information 91. Specifically, for example, the attention state calculation information creation unit 58 acquires the values of the time column 9021 and the brain activity signal name column 9022 from the i-th error calculation feature amount information 902, and stores the i-th attention state calculation information 91. They are stored in the time column 911 and the brain activity signal name column 912, respectively.

Furthermore, the attention state calculation information creation unit 58 acquires all the attention types and attention state coefficients included in the i-th attention state learning accumulation information 87. The attention state calculation information creation unit 58 calculates the attention level of each attention type at each time using each attention state coefficient and a predetermined relational expression corresponding to the attention state coefficient. The attention state calculation information creation unit 58 specifies the maximum attention level and the attention type with the maximum attention level for each time. The attention state calculation information creation unit 58 stores the specified attention type and attention level in the attention type column 913 and the attention level column 914, respectively.

In step 158, the error calculation unit 59 creates the i-th error calculation information 92. Specifically, the error calculation unit 59 uses, for example, the values of the time column 9021 and the brain activity signal name column 9022 of the i-th error calculation feature amount information 902, and the time column 921 and the brain activity signal of the i-th error calculation information 92. Each is stored in the name column 922.

Subsequently, the error calculation unit 59 refers to the i-th error learning accumulation information 88 and acquires the error signal name from which the error measurement signal information 82 is created, and the error factor and error coefficient corresponding to the error signal name. . The error calculation unit 59 acquires the value of the error signal at each time from all the error measurement signal information 82. In addition, the error calculation unit 59 acquires the brain activity signal error range and the index error range at each time from the i-th error calculation feature amount information 902.

The error calculation unit 59 uses each acquired error coefficient, a predetermined relational expression corresponding to the error coefficient, and the acquired error signal, brain activity signal error range, and index error range values to determine the brain at each time. Each error of the activity signal is calculated. Further, the error calculation unit 59 calculates the sum of errors of each type for each time. The error calculation unit 59 stores the sum of the calculated errors at each time in the accumulated error column 923, the error factors at each specified time in the error factor column 924, and the calculated errors at each time in the error column 925. .

Note that if the unit or dimension differs between the error calculated using the error coefficient and the attention level calculated using the attention state coefficient, the error calculation unit 59 uses the same method as in step S142. Thus, by converting the error in each error column 925 and each accumulated error column 923 of the i-th error calculation information 92, the error unit and dimension may be matched with the attention level unit and dimension.

In step 159, the error calculation unit 59 increments the value of the counter i by 1. In step 160, the attention state information storage unit 62 creates attention state storage information 93, the error information storage unit 63 creates error storage information 94, and the optimal attention state selection unit 65 creates optimal attention state selection information 95. .

Specifically, the attention state information storage unit 62 creates the attention state storage information 93 by merging the first attention state calculation information 91 to the m-th attention state calculation information 91 using the time column 911 as a key. . Further, the error information storage unit 63 creates the error storage information 94 by merging the first error calculation information 92 to the m-th error calculation information 92 using the time column 921 as a key.

Further, the optimal attention state selection unit 65 refers to the error storage information 94 and selects an optimal signal that is a brain activity signal indicating an optimal error from the first brain activity signal to the mth brain activity signal for each time. Then, the value of the accumulated error column 946 of the selected optimum signal is acquired. The optimal attention state selection unit 65 stores the name of the selected optimal signal in the optimal signal name column 952 and stores the acquired accumulated error in the optimal error column 953 for each time.

Note that the optimal attention state selection unit 65 selects, for example, the brain activity signal having the smallest cumulative error as the optimal signal from the first brain activity signal to the m-th brain activity signal at each time. Further, the optimal attention state selection unit 65 may select the optimal signal from a plurality of signals selected in advance from the first brain activity signal to the mth brain activity signal, for example. Hereinafter, the accumulated error of the optimum signal is referred to as the optimum error.

Further, for example, the optimal attention state selection unit 65 may refer to the optimal signal information 89 and exclude brain activity signals having an average value of cumulative errors at each time that are equal to or greater than a predetermined value from the optimal signal selection candidates. . For example, the optimal attention state selection unit 65 may determine one brain activity signal randomly selected from the brain activity signals whose cumulative error is equal to or less than a predetermined value as the optimal signal at the time.

In step 161, the optimal attention state selection unit 65 sets the value of the counter i to 1. In step 162, the optimal attention state calculation unit 64 determines whether or not the i-th brain activity signal is an optimal signal. If the i-th brain activity signal is the optimum error (step 162: YES), the process proceeds to step 164. If the i-th brain activity signal is not the optimum error (step 162: NO), the process proceeds to step 163.

In step 163, the optimal attention state calculation unit 64 increments the value of the counter i by 1. In step 164, the optimal attention state calculation unit 64 creates optimal attention state information 96. Specifically, the optimal attention state calculation unit 64 includes a table including a time column 951, an optimal signal name column 952, and an optimal error column 953 of the optimal attention state selection information 95, and a time column 931 of the attention state storage information 93 and the optimal. The optimal attention state information 96 is created by merging the table including the attention type column 934 and the attention level column 936 in the signal with the time column as a key.

In step 166, the optimum attention error correction information calculation unit 66 creates error correction information 97. Specifically, the optimal attention error correction information calculation unit 66 sets the value of the time column 961 and the value of the optimal signal name column 962 of the optimal attention state information 96, the time column 971 of the error correction information 97, and the calculated optimal signal column. 973, respectively.

Also, the optimum attention error correction information calculation unit 66 selects the optimum signal at the time of learning from the total error of the brain activity signals in the last line of the optimum signal information 89, for example. The method for selecting the optimum signal at the time of learning is the same as the method for selecting the optimum signal by the optimum attention state selecting unit 65 in step 159.

The optimal attention error correction information calculation unit 66 stores the optimal signal at the time of learning in the learning optimal signal column 972. That is, the same value is stored in the learning optimum signal column 972. For example, the optimal attention error correction information calculation unit 66 stores “OK” in the determination result column 974 of the record in which the learning optimal signal and the calculated optimal signal are the same, and the determination result of the record in which the learning optimal signal and the calculated optimal signal are different. “NG” is stored in the column 974.

In the above-described attention state calculation process, the attention state calculation unit 51 uses the relationship between the plurality of types of brain activity signals acquired by the attention state learning unit 21 and the error between the attention state and the attention state, from each brain activity signal. An error between the estimated value of the state and the estimated value of the attention state is calculated. The attention state calculation unit 51 obtains the optimum error by comparing the calculated errors, and selects the estimated value of the attention state based on the brain activity signal indicating the optimum error. Accordingly, the attention state calculation unit 51 can accurately evaluate the attention state without being affected by the brain activity signal having a large error and the poor accuracy and the estimated value of the attention state by the brain activity signal.

When the attention state learning unit 21 and the attention state calculation unit 51 use the same kind of brain activity signals measured from a plurality of measurement channels (measurement points), for example, the average value of the brain activity signals in the plurality of measurement channels Alternatively, an indicator such as a phase difference or correlation having a certain time width between the plurality of measurement channels may be used as the type of brain activity signal. In this case, the attention state learning unit 21 and the attention state calculation unit 51 may select a brain activity signal of one channel from the plurality of channels using, for example, the same method as the selection method of the optimum signal. Good.

FIG. 8 shows an example of processing by the attention state output unit 71. In step 171, the optimal attention state output unit 72 determines whether output information has been created. The output information is information including arbitrary information stored in the storage unit. Hereinafter, the output information includes optimum attention state information 96, optimum attention state selection information 95, error storage information 94, and error correction information 97.

When the optimal attention state output unit 72 determines that the output information has been created (step S1711: YES), the process proceeds to step 172. When it is determined that the output information has not been created (step S1711: NO), step 171 is repeated. Note that the determination processing in step 171 may be performed by another unit of the attention state output unit 71.

In step 172, for example, the optimum attention state output unit 72 provides the optimum attention state information 96, the optimum attention state selection information output unit 73 provides the optimum attention state selection information 95, the error information output unit 74 provides the error storage information 94, and the optimum storage state 94. The attention error correction information output unit 75 outputs error correction information 97 to the output unit 15. When the output information includes other information, any unit included in the attention state output unit 71 may output the other information, or the output unit for outputting the other information It may be included in the state output unit 71.

In step 173, the optimum attention error correction information output unit 75 uses the determination result of the error correction information 97 so that, for example, the learning optimum signal and the calculated optimum signal are identical at all times in a time window having a predetermined width including the current time. Determine whether you are doing it. The optimal attention error correction information output unit 75 returns to step 171 when it is determined that they match (step 173: YES), and proceeds to step 174 when it is determined that they do not match (step 173: NO).

In step 174, the optimum attention error correction information output unit 75 refers to the optimum signal information 89, specifies, for example, an error factor indicating the maximum error in the learning optimum signal, and outputs an instruction to suppress the specified error factor. 15 is output. The error factor suppression instruction is determined in advance for each error factor, for example. The instruction “please reattach the electroencephalogram electrode” is an example of an instruction to suppress an electroencephalogram error factor.

As described above, the attention state output unit 71 indicates the error between the estimated value and the estimated value of the attention state calculated by the attention state calculation unit 51 and the information created by the attention state calculation unit 51 to the user and the measurer. be able to. The attention state output unit 71 can output an instruction for reducing the error by comparing the learning optimum signal and the calculated optimum signal. As a result, attention state evaluation is realized while suppressing the influence of error factors.

In the example of FIG. 8, the optimal attention error correction information output unit 75 outputs an error suppression instruction for the learning optimal signal, but selects other brain activity signals by referring to the error storage information 94 and selects them. An error suppression instruction for the brain activity signal may be output. Note that the attention state output unit 71 may output only a part of the created state. The attention state output unit 71 may output only the name of the optimum signal, for example, and the user may evaluate the attention state. At this time, the mental activity state evaluation system is a mental activity state evaluation support system that supports the evaluation of the mental activity state.

Also, for example, the optimal attention error correction information output unit 75 may output an instruction to suppress an error factor indicating the maximum error in each selected brain activity signal. Further, for example, the optimal attention error correction information output unit 75 may calculate an error of each of the error factors of the plurality of brain activity signals and output an instruction to suppress the error factor indicating the maximum error. Further, the error suppression instruction may be output to a target other than the user such as a measurer, such as a measurement setting or measurement environment changing instruction for reducing an error factor by the measurement device or the environment setting device.

Further, for example, each unit of the calculation unit 20 may perform a part of the processing according to an input from a user such as an expert. Moreover, each part of the calculating part 20 may perform a process, for example using the information set at the time of the information set beforehand, or the time of previous attention state learning or attention state calculation. Further, for example, a predetermined relational expression for calculating an attention state and an error of the attention state may be changed based on, for example, judgment of an expert.

The information processing apparatus 1 according to the present embodiment corrects the attention state learning accumulation information 87. In the present embodiment and the embodiments described later, differences from the first embodiment will be described.

FIG. 9 shows an example of correction processing for the attention state learning accumulation information 87. Hereinafter, an example in which the attention state learning feature amount storage unit 33 executes the correction process will be described. However, another part of the attention state learning unit 21 may execute the correction process, or the attention state learning unit 21 may correct the correction process. A unit that executes processing may be further included. The correction process 110 that may be performed by adding a new function unit to the function unit includes, for example, a measurement process 106, a relationship acquisition process 107, a collation process 108, and a correction process 109.

First, in the measurement process 106, the attention state learning feature amount accumulating unit 33 measures each brain activity signal at each attention degree by the same method as in the first embodiment. Subsequently, the attention state learning feature amount accumulation unit 33 generates, for each brain activity signal, a function (curve in the drawing) indicating the relationship between the brain activity signal and the attention state by fitting or the like.

In the matching process 108, the attention state learning feature amount storage unit 33 acquires, for example, one value of each brain activity signal indicating the first attention level in the measurement process 106. The attention state learning feature amount accumulation unit 33 calculates an estimated value of the attention level corresponding to each acquired brain activity signal value by substituting the value of each acquired brain activity signal into a function corresponding to the brain activity signal. Then, it is determined whether or not the difference between the calculated maximum value and the minimum value of each estimated value is equal to or less than a predetermined value. The predetermined value may be 0. In addition, the first attention level is preferably an average value of the attention level obtained by the measurement process 106, for example.

The attention state learning feature amount storage unit 33 executes the correction process 109 when it is determined that the difference between the maximum value and the minimum value of the calculated estimated value exceeds a predetermined value. The attention state learning unit 21 corrects the function in the correction process 109.

The attention state learning feature amount accumulation unit 33 corrects a function corresponding to a brain activity signal other than the optimum signal indicated by the optimum signal information 89, for example. For example, the attention state learning feature amount accumulating unit 33 identifies the optimum signal with reference to the optimum signal information 89, and the difference between the maximum value and the minimum value of the estimated attention level of each brain activity signal is equal to or less than a predetermined value. The coefficient of the function corresponding to the brain activity signal other than the optimal signal is changed at random. In addition, for example, the attention state learning feature amount accumulation unit 33 calculates an average value of the estimated attention level of each acquired brain activity signal, and the estimated value of the attention level of each acquired brain activity signal is the average value. You may change the coefficient of each function at random so that it may correspond.

Then, the attention state learning feature amount accumulation unit 33 recalculates the attention level of each brain activity signal at each time using the corrected relational expression, recalculates the attention state coefficient using the calculated attention level, The calculated attention state coefficient is stored in the attention state coefficient column 873.

For example, when the function is corrected without using the optimum signal information 89, the correction processing 110 is performed immediately before step 142. The correction processing 110 may be performed after step 142 both when the optimal signal information 89 is used for correcting the function and when it is not used. When the correction process 110 is performed after step 142, the optimum signal information 89 may be recreated by returning to step 142 again.

The attention state learning accumulation information 87 of the present embodiment can obtain a more accurate attention state coefficient by executing the correction process 110, and thus can improve the accuracy of estimating the attention state.

The information processing apparatus 1 according to this embodiment defines a new brain activity signal by combining a plurality of brain activity signals. Specifically, for example, the information processing apparatus 1 uses a combination of brain waves and optical topography as a new brain activity signal. Hereinafter, similarly to the first embodiment, it is assumed that the brain activity signal input unit 11 receives the first to mth brain activity signals (m is an integer of 2 or more).

In addition, the m + 1th brain activity signal to the m + a brain activity signal (a is one or more) which are new brain activity signals generated by a combination of a plurality of brain activity signals included in the first brain activity signal to the mth brain activity signal. Each) is defined. Hereinafter, these brain activity signals are referred to as a combined brain activity signal.

The attention state learning unit 21 performs the processing from step 130 to step 139 for the integer i not less than m + 1 and not more than m + a after it is determined in step 129 that there is no i-th brain activity signal information. That is, the attention state learning unit 21 corresponds to each combination brain activity signal, and attention state learning information 83, attention state learning feature amount information 84, error learning information 85, error learning feature amount information 86, and attention state learning accumulation information 87. Generate error learning accumulation information. Thereafter, the attention state learning unit 21 transitions to step 141.

The attention state calculation unit 51 performs the processing from step 156 to step 158 for an integer i that is greater than or equal to m + 1 and less than or equal to m + a after it is determined in step 155 that there is no i-th brain activity signal information. That is, the attention state calculation unit 51 creates the calculated feature amount group information 90, the attention state calculation information 91, and the error calculation information 92 corresponding to each combination brain activity signal. In step 160, the attention state calculation unit 51 determines the optimum from the first brain activity signal to the m + a brain activity signal, that is, from the brain activity signal input to the brain activity signal input unit 11 and the generated combined brain activity signal. Select a signal.

Note that the name, signal value, attention state, and index of the combined brain activity signal are determined as follows, for example. For example, the name of the combined brain activity signal may be predetermined, for example, or may be a combination of names of a plurality of brain activity signals constituting the combined brain activity signal. The attention type and attention level corresponding to the combined brain activity signal are determined in advance according to, for example, the attention type and attention level of a plurality of brain activity signals constituting the combined brain activity signal.

The attention state learning unit 21 and the attention state calculation unit 51 may determine the value of the combined brain activity signal as, for example, a vector composed of a plurality of brain activity signal values constituting the combined brain activity signal. The values obtained by substituting the values of the plurality of brain activity signals into a predetermined function may be determined.

Further, the attention state learning unit 21 and the attention state calculation unit 51 may determine the index value of the combined brain activity signal as, for example, a vector composed of the index values of a plurality of brain activity signals constituting the combined brain activity signal. Alternatively, it may be determined as a value obtained by substituting the index values of the brain activity signals constituting the plurality of brain activity signals into a predetermined function. For example, the correlation coefficient of a certain time width between the α wave power acquired from the electroencephalogram signal and the oxygenated hemoglobin amount change acquired from the optical topography signal is a combined brain activity signal composed of the electroencephalogram signal and the optical topography signal. It is an example of the indicator.

The information processing apparatus 1 according to the present embodiment acquires an index generated by a combination of indices having different properties included in the plurality of brain activity signals by combining the plurality of brain activity signals, and the index and the attention state And the error of the attention state can be acquired. The information processing apparatus 1 according to the present embodiment, for example, evaluates an optical topography signal index that can accurately evaluate the persistence attention and the selectivity attention when evaluating the convertibility attention combining the persistence attention and the selectivity attention. It is possible to evaluate the attention state with high accuracy by creating a new index by combining the index of the electroencephalogram signal that can be evaluated well.

The conditions for combining the indices are not limited to those described above. For example, the information processing apparatus 1 extracts the evaluation index of the selectivity attention extracted from the optical topography signal and the electroencephalogram signal in order to evaluate the convertibility attention. An evaluation index of persistence attention may be combined, or an optical topography signal index that can accurately evaluate attention level and an electroencephalogram signal index that can accurately evaluate switching of attention types may be combined .

The mental state evaluation system according to the present embodiment measures information on the mental activity object measured by the mental activity object measurement device. FIG. 10 shows a configuration example of the mental state evaluation system of this embodiment. The mental activity state evaluation system further includes one or more mental activity target measuring devices 7.

The mental activity target is a target to which the user is directing the mental activity state or a target inducing the user's mental activity state. When the attention state is evaluated as the mental activity state, the user's attention area is an example of the mental activity target. The attention area is an area where the user is paying attention. For example, the attention area is a part of the display included in the output device 6.

Note that the target of mental activity need not be the hardware itself. When the user is paying attention to one image among a plurality of images displayed on the display, a part of the display on which the one image is displayed may be the attention area, or the one image itself May be used as the attention area, or specific information included in the one image, for example, the character color or meaning of the character when the character is included in the one image may be used as the attention area.

The mental activity target measuring device 7 measures a mental activity target. When the mental state evaluation system evaluates an attention state as a mental activity state, the mental activity target measurement device 7 is a device for measuring a user's attention region, for example, a gaze measurement device. The coordinates indicating the region of the user's line of sight measured by the line-of-sight measurement device is an example of a signal measured by the line-of-sight measurement device. Hereinafter, it is assumed that the mental activity target measuring device 7 is an attention area measuring device.

Hereinafter, the configuration of the information processing apparatus 1 according to the present embodiment will be described. The input unit 10 further includes a unit that receives an input of a signal indicating the attention area measured by the mental activity target measuring device 7.

The attention state learning unit 21 and the attention state calculation unit 51 further include, for example, a functional unit that acquires a signal measured by the mental activity target measuring device 7 from the input unit 10. The attention state learning unit 21 further includes, for example, a function unit that generates attention region learning information, attention region learning feature amount information, and attention region learning feature amount accumulation information. In addition, the attention state calculation unit 51 further includes, for example, a function unit for calculating attention region calculation feature amount information and attention region information.

The attention state output unit 71 further includes, for example, a functional unit for outputting information on the attention region created by the attention state learning unit 21 and the attention state calculation unit 51. The information regarding the attention area is information that is not created in the first embodiment and is created in the present embodiment, and includes, for example, the attention area information, and is output in accordance with an instruction from the user, for example.

The attention area learning information, the attention area learning feature amount information, and the attention area learning feature amount accumulation information are respectively the attention state learning information 83, the attention state learning feature amount information 84, and the attention state column of the attention area learning feature amount accumulation information. , Attention level column, brain activity signal name column, indicator column, and attention state coefficient column are replaced with attention region type column, attention region column, signal name column, attention region index column, and attention region coefficient column, respectively. It is.

Also, each of the attention area calculation feature amount information and the attention area information includes the brain activity signal name column, the index column, the attention type column, and the attention level column of the attention state calculation feature amount information 901 and the attention state calculation information 91, respectively. The information is replaced with the signal name field, the attention area index field, the attention area type field, and the attention area field.

The attention state learning unit 21 and the attention state calculation unit 51 determine the attention region type and the attention region according to, for example, input by an expert or the like. In addition, the attention state learning unit 21 and the attention state calculation unit 51 may acquire the attention region type and the attention region that are predetermined for each task.

The attention state learning unit 21 and the attention state calculation unit 51 acquire, for example, a name predetermined for each functional unit that acquires a signal measured by the mental activity target measuring device 7 as a signal name. The line-of-sight area is an example of a signal name. The attention state learning unit 21 and the attention state calculation unit 51 may determine a name input by an expert or the like as a signal name.

The attention state learning unit 21 and the attention state calculation unit 51 calculate the attention region index value by performing predetermined processing from the signal measured by the mental activity target measurement device 7, for example. The attention state learning unit 21 and the attention state calculation unit 51 calculate the attention region index value by performing signal processing for reducing noise such as a bandpass filter and trend removal, for example.

Note that the attention state learning unit 21 and the attention state calculation unit 51 may calculate the attention region index value according to an algorithm for calculating the attention region index value input from an expert or the like. The area coordinates measured by the line-of-sight measurement device and the user's line-of-sight are both examples of the attention area index.

The attention state learning unit 21 and the attention state calculation unit 51 include, for example, a predetermined relational expression with unknown coefficients having the attention region index as an independent variable and the attention region as a dependent variable, and the attention region type corresponding to the attention region learning feature amount information. The attention area coefficient is calculated using the values of the attention area column and the attention area index field. The calculation method of the attention area coefficient is the same as the calculation method of the attention state coefficient.

Regarding the processing in the present embodiment, steps for executing processing different from that in the first embodiment will be described with reference to FIGS. 6 and 7.

In step 122, in addition to the processing in the first embodiment, the attention area learning information creation unit acquires an attention area corresponding to the instruction output in step 121. In step 123: In addition to the processing in the first embodiment, the attention area learning information creation unit acquires a signal to be stored in the attention area learning information, and generates attention area learning information.

If there is no k-th task in step 128 (step S128: NO), in addition to the processing in the first embodiment, before proceeding to step 142, the attention area learning feature amount accumulation creating unit stores the attention area learning feature amount accumulation information. create. In step 138, in addition to the processing in the first embodiment, the attention area learning feature amount creating unit creates attention area learning feature amount information.

In step 152, in addition to the processing in the first embodiment, the attention area calculation feature value creation unit acquires a signal to be stored in the attention area calculation feature value information, and creates attention area calculation feature value information.

In step 164, in addition to the processing in the first embodiment, the attention area information creation unit creates attention area information. Note that the attention area information creation process is not necessarily executed in step 164, but may be executed immediately before step 161 until step 166 is executed.

Thus, the information processing apparatus 1 according to the present embodiment can evaluate the target of the mental activity by using the signal measured by the mental activity target measuring apparatus 7. In particular, when the mental activity target measurement device 7 is an attention area measurement device, the information processing device 1 can evaluate a target to which the user is paying attention. It is known that the attention state changes due to various factors, and the information processing apparatus 1 can evaluate the cause of the change in the attention state by evaluating the object to which attention is directed.

The mental activity evaluation system of the present embodiment performs mental activity evaluation using autonomic nerve activity. FIG. 10 shows a configuration example of the mental activity evaluation system of this embodiment. The mental activity evaluation system further includes an autonomic nerve activity measuring device 8 and an autonomic nerve activity measuring device 9. The number of autonomic nerve activity measuring devices included in the mental activity evaluation system does not matter. That is, the mental activity evaluation system performs mental activity evaluation using one or more types of autonomic nerve activity signals.

An autonomic nerve activity signal is a signal that can be evaluated from autonomic nerve activity and acquired from a living body. An electrocardiogram, blood pressure, respiration rate and respiration interval, perspiration amount, skin resistance, skin temperature, and eardrum temperature are all examples of autonomic nerve activity signals. The autonomic nerve activity measuring device 8 and the autonomic nerve activity measuring device 9 acquire an autonomic nerve activity signal.

Hereinafter, the configuration of the information processing apparatus 1 according to the present embodiment will be described. The input unit 10 further includes an input unit for receiving an input of an autonomic nerve signal measured by the autonomic nerve activity measuring device 8 and the autonomic nerve activity measuring device 9. The attention state learning unit 21 and the attention state calculation unit 51 further include, for example, a functional unit that acquires the signals measured by the autonomic nerve activity measurement device 8 and the autonomic nerve activity measurement device 9 from the input unit 10.

The processing flow in this embodiment is as follows. At step 129, the attention state learning unit 21 determines that there is no i-th brain activity signal information, and then corresponds to each of the autonomic nerve activity signals, attention state learning information 83, attention state learning feature amount information 84, error learning. Information 85, error learning feature amount information 86, attention state learning accumulation information 87, and error learning accumulation information are generated. Thereafter, the attention state learning unit 21 transitions to step 141.

The attention state calculation unit 51 determines that there is no i-th brain activity signal information in Step 155, and then calculates the feature amount group information 90, the attention state calculation information 91, and the error corresponding to each of the autonomic nerve signals. Calculation information 92 is created. In step 160, the attention state calculation unit 51 selects an optimal signal from the first brain activity signal to the mth brain activity signal and all the autonomic nerve signals. The attention state learning unit 21 and the attention state calculation unit 51 may calculate each index from the autonomic nerve activity signal, or may calculate each index from the brain activity signal and the autonomic nerve activity signal.

The information processing apparatus 1 according to the present embodiment can acquire an index that can evaluate the attention state with higher accuracy by evaluating the autonomic nerve activity in addition to the brain activity. Autonomic nerves play an important role in maintaining homeostasis of the living body, and show a response to changes in mental activity states such as attention states. Moreover, the autonomic nerve is controlled by the hypothalamus located in the deep part of the brain, and it is difficult to evaluate it by noninvasive brain activity measurement such as an electroencephalogram or an optical topography signal. Therefore, the information processing apparatus 1 according to the present embodiment can evaluate the attention state with higher accuracy by performing the evaluation combining the brain activity and the autonomic nerve activity.

This example describes a GUI (Graphical User Interface) used when a user evaluates a mental activity state. FIG. 12 is an example of a measurement information display screen for evaluating a mental activity state. The measurement information display screen 200 includes, for example, an optimum signal display area 201, an error factor display area 202, a learning optimum signal display area 203, a mental activity state display area 204, a data selection area 205, a brain measurement signal display area 206, an error measurement. A signal display area 207 and a button 208 are included.

The optimum signal display area 201 includes, for example, the name of the calculated optimum signal of the error correction information 97, the type of the mental activity state under evaluation, and the mental activity state and the mental activity state calculated using the calculated optimum signal. Displays the error of. The error factor display area 202 displays, for example, an error factor indicating the maximum error, an error factor indicating the maximum error, and a method for suppressing the error due to the error factor among the errors of the mental activity state calculated using the optimum signal. .

The learning optimum signal display area 203 displays, for example, the learning optimum signal of the error correction information 97 and the determination result. In the example of FIG. 12, since the calculated optimal signal is an optical topography signal and the learning optimal signal is an electroencephalogram signal, the learning optimal signal display area 203 displays the determination result “NG”. The mental activity state display area 204 displays, for example, the name of the mental activity state under evaluation and time series data of the mental activity state under evaluation.

The data selection area 205 receives an input for selecting whether to use new measurement data or read existing data, for example, as a brain activity signal for evaluating a mental activity state. The data selection area 205 includes buttons for starting measurement and ending measurement. The brain measurement signal display area 206 includes, for example, the name of the brain activity measurement signal being measured, the error between the mental activity state and the mental activity state calculated using the brain activity signal, and the time series of the brain activity measurement signal. Display data.

The error measurement signal display area 207 includes, for example, the name of the error measurement signal being measured, the error that the error measurement signal has on the mental activity state indicated by each brain activity signal, and time-series data of the error measurement signal Is displayed. The button 208 is a button for selecting whether to display the measurement information display screen 200 or a setting screen described later.

FIG. 13 is an example of a setting screen for setting an evaluation parameter of the mental activity state. The setting screen 300 includes, for example, a brain activity signal setting area 301, an error measurement signal setting area 302, a calculation method setting area 303, a learning data correction setting area 304, a mental activity state setting area 305, a button 306, and a button 214. .

The brain activity signal setting area 301 is, for example, a brain activity signal of the type from the brain activity signal used in the information processing apparatus 1, the number of measurement points of the brain activity signal, and the same kind of brain activity signal obtained from a plurality of measurement points. Including a region for setting each value.

The error measurement signal setting area 302 includes, for example, an error measurement signal used in the information processing apparatus 1, an error factor corresponding to the error measurement signal, and an area for setting an error magnification. The error magnification is a magnification for weighting the error calculated by the error measurement signal.

The calculation method setting area 303 is an area for selecting a method for calculating an error between the mental activity state and the mental activity state. Specifically, when the user uses the calculation method setting area 303, for example, whether the learning data is used for the calculation or the learning data is not used, or if the learning data is used for the calculation, a new Whether to perform learning or to use existing data can be set.

The learning data correction setting area 304 is an area for setting the presence / absence of correction of learning data and a correction method for correction. The correction of the learning data corresponds to the correction process 110 in the second embodiment. The learning data correction method is, for example, a method for correcting the curves in FIG. 9 in order to match the estimated values when the estimated values of the mental activity state calculated from the learning data do not match. Indicates.

In the learning data correction setting area 304 of FIG. 12, the “optimal error” indicates the optimum error so that the “average value” matches the calculated estimated value average value. The estimated values of “Signal 1” and “Signal 2” are calculated using the brain activity signal 1 and the brain activity signal 2 respectively, so as to match the estimated value of the mental activity state using the brain activity signal. It is a radio button for correcting the curve so as to coincide with the estimated value of the mental activity state.

The mental activity state setting area 305 is an area for selecting a mental activity state to be evaluated. Specifically, the mental activity state setting region 305 is, for example, whether an existing mental activity state is evaluated as a mental activity state, a new mental activity state is evaluated, or an existing mental activity state. For example, a radio button for selecting whether to evaluate an attention state, a stress state, or a mood state is included. A button 306 is a button for saving parameters set on the setting screen 300.

With the GUI of this embodiment, when the user evaluates the mental activity state, the evaluation result can be easily understood, information used for evaluation such as learning data and evaluation parameters can be easily set, and the setting can be changed. Can be added, and deleted.

Note that the contents displayed on the measurement information display screen 200 and the setting screen 300 of this embodiment are merely examples. For example, the measurement information display screen 200 may display arbitrary information stored in the storage unit 80. In addition, the setting screen 300 may further include an area for setting information based on knowledge of an expert or the like.

In addition, this invention is not limited to the above-mentioned Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

In addition, each of the above-described configurations, functions, processing units, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor. Information such as programs, tables, and files for realizing each function can be stored in a memory, a hard disk, a recording device such as an SSD (Solid State Drive), or a recording medium such as an IC card or an SD card. In addition, the control lines and information lines are those that are considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. In practice, it may be considered that almost all the components are connected to each other.

Claims (15)

1. A mental activity state evaluation support device that supports the evaluation of mental activity state using multiple types of brain activity signals,
   Including a processor and a storage device;
   The storage device includes an error measurement signal indicating a degree of a factor other than the mental activity state that changes the plurality of types of brain activity signals, and estimation of the mental activity state corresponding to each of the plurality of types of brain activity signals. Holding error calculation information indicating the correspondence between the error due to the factor of the value,
   The processor is
   Obtaining the error measurement signal;
   Referring to the error calculation information, from the acquired error measurement signal, calculate an error due to the factor of the estimated value of the mental activity state corresponding to each of the plurality of types of brain activity signals,
   Based on the magnitude of each of the calculated errors, a first type brain activity signal is selected from the plurality of types of brain activity signals,
   A mental activity state evaluation support apparatus that outputs information indicating the first type brain activity signal.
2. The mental activity state evaluation support device according to claim 1,
   The processor is
   Obtaining the plurality of types of brain activity signals at a plurality of times and the error measurement signals at the plurality of times;
   For each of the multiple types of brain activity signals,
   In each of the plurality of times, to calculate the variation in the feature amount of the brain activity signal in the period including the time,
   Based on the correspondence between the error measurement signal at each of the plurality of times and the calculated variation, the error measurement signal included in the error calculation information and the estimated value of the mental activity state corresponding to the brain activity signal Mental activity state evaluation support device that determines correspondence with error.
3. The mental activity state evaluation support device according to claim 1,
   The storage device further holds mental activity state learning information indicating correspondence between each of the plurality of types of brain activity signals and the estimated value of the mental activity state,
   The processor is
   Obtaining the plurality of types of brain activity signals;
   Referring to the mental activity state learning information, calculating a first estimated value of the mental activity state from the first type brain activity signal;
   The information indicating the first type brain activity signal is a mental activity state evaluation support device including the first estimated value.
4. The mental activity state evaluation support device according to claim 3,
   The processor is
   Obtaining the plurality of types of brain activity signals at a plurality of times;
   Determining a mental activity state corresponding to each of the plurality of types of brain activity signals at each of the plurality of times according to a predetermined relationship;
   Based on the relationship between each of the brain activity signals at each of the plurality of times and the mental activity state corresponding to the brain activity signal at each of the plurality of times in each of the plurality of types of brain activity signals. A mental activity state evaluation support device that creates mental activity state learning information.
5. The mental activity state evaluation support device according to claim 3,
   Connected to the display device,
   The mental activity state evaluation support device, wherein the processor outputs the first estimated value and an error between the first estimated value to the display device.
6. The mental activity state evaluation support device according to claim 3,
   A first relationship for determining the mental activity state from a second type brain activity signal included in the plurality of types of brain activity signals;
   A second relationship for determining the mental activity state from a third type brain activity signal included in the plurality of types of brain activity signals is predefined,
   The processor obtains a first brain activity signal of the second type brain activity signal and a second brain activity signal of the third type brain activity signal;
   The difference between the mental activity state corresponding to the first brain activity signal indicated by the first relationship and the mental activity state corresponding to the second brain activity signal indicated by the second relationship is equal to or less than a first threshold value. Yes,
   The processor is
   The second estimated value of the mental activity state is calculated from the first brain activity signal with reference to the mental activity state learning information, and the third estimated value of the mental activity state is calculated from the second brain activity signal. To calculate
   Correcting the mental activity learning information,
   In the correction, the correspondence between the second type brain activity signal included in the mental activity state learning information and the estimated value of the mental activity state, and the third type brain included in the mental activity state learning information. The difference between the second estimated value calculated from the corrected mental activity state learning information and the third estimated value is at least one of the correspondence between the activity signal and the estimated value of the mental activity state. A mental activity state evaluation support device that corrects a threshold value to be equal to or lower than a threshold value.
7. The mental activity state evaluation support device according to claim 6,
   The mental activity state support apparatus, wherein the second threshold is 0.
8. The mental activity state evaluation support device according to claim 1,
   The factor changes a combined brain activity signal generated by combining a plurality of brain activity signals included in the plurality of types of brain activity signals,
   The error calculation information indicates a correspondence between the error measurement signal and an error due to the factor of the estimated value of the mental activity state corresponding to each of the combination brain activity signals,
   The processor is
   Referring to the error calculation information, from the acquired error measurement signal, calculate an error due to the factor of the estimated value of the mental activity state corresponding to each of the combination brain activity signal,
   The first type brain activity signal is selected from the plurality of types of brain activity signals and the combined brain activity signal based on the magnitudes of the errors of the plurality of types of brain activity signals and the errors of the combined brain activity signals. , Mental activity state evaluation support device.
9. The mental activity state evaluation support device according to claim 1,
   The processor is
   Receiving an input of a mental activity target measurement signal indicating a target to which the mental activity state is directed or a mental activity target that is a target to induce the mental activity state;
   Calculate the mental activity target based on the mental activity target measurement signal,
   A mental activity state evaluation support apparatus that outputs information indicating the calculated mental activity target.
10. The mental activity state evaluation support device according to claim 1,
    The factor changes the autonomic nerve activity signal,
    The error calculation information indicates a correspondence between the error measurement signal and an error due to the factor of an estimated value of a mental activity state corresponding to each of the autonomic nerve activity signals,
    The processor is
    With reference to the error calculation information, from the acquired error measurement signal, to calculate the error due to the factor of the estimated value of the mental activity state corresponding to each of the autonomic nerve activity signal,
    The first type brain activity signal is selected from the plurality of types of brain activity signals and the autonomic nerve activity signals based on the magnitudes of the errors of the plurality of types of brain activity signals and the errors of the autonomic nerve activity signals. , Mental activity state evaluation support device.
11. The mental activity state evaluation support device according to claim 1,
    The processor acquires a plurality of types of error measurement signals,
    The error calculation information indicates correspondence between the plurality of types of error measurement signals and the factors,
    The processor is
    With reference to the error calculation information, an error due to a factor corresponding to each of the plurality of types of error measurement signals of the estimated value of the mental activity state corresponding to the first type brain activity signal is calculated,
    A first type error measurement signal is determined from the plurality of types of error measurement signals based on the magnitudes of errors corresponding to the calculated factors.
    A mental activity state evaluation support apparatus that outputs information indicating the first type error measurement signal.
12. It is a mental activity state evaluation support device according to claim 11,
    Connected to the display device,
    The display device includes an area for inputting correspondence between the plurality of types of error measurement signals and the factors,
    The processor is
    Accepting inputs of correspondence between the plurality of types of error measurement signals and the factors,
    In accordance with the input, create a correspondence between the plurality of types of error measurement signals included in the error calculation information and the factor,
    A mental activity state evaluation support apparatus that refers to the error calculation information and outputs a factor corresponding to the first type error measurement signal as information indicating the first type error measurement signal.
13. The mental activity state evaluation support device according to claim 1,
    The mental activity state is an attention state composed of an attention type and an attention level,
    The mental activity state evaluation support apparatus, wherein the mental activity state error is the attention level error.
14. A mental activity state evaluation support system that supports the evaluation of mental activity state using multiple types of brain activity signals,
    A mental activity state measuring device, and an error measuring device for measuring the error measurement signal,
    The error measurement device acquires the error measurement signal,
    The mental activity measuring device
    The factors of an error measurement signal indicating the degree of factors other than the mental activity state that change the plurality of types of brain activity signals, and the estimated values of the mental activity states corresponding to the plurality of types of brain activity signals, respectively Holds error calculation information indicating the correspondence between the error and
    Obtaining the error measurement signal from the error measurement device;
    Referring to the error calculation information, from the acquired error measurement signal, to calculate an error of the estimated value of the mental activity state corresponding to each of the plurality of types of brain activity signals,
    Based on the magnitude of each of the calculated errors, a first type brain activity signal is selected from the plurality of types of brain activity signals,
    A mental activity state evaluation support system that outputs information indicating the first type brain activity signal.
15. A mental activity state evaluation support device is a method for supporting the evaluation of a mental activity state using a plurality of types of brain activity signals,
    The mental activity state evaluation support device comprises:
    The factors of an error measurement signal indicating the degree of factors other than the mental activity state that change the plurality of types of brain activity signals, and the estimated values of the mental activity states corresponding to the plurality of types of brain activity signals, respectively Holds error calculation information indicating the correspondence between the error and
    The method
    The mental activity state evaluation support device comprises:
    Obtaining the error measurement signal;
    Referring to the error calculation information, from the acquired error measurement signal, to calculate an error of the estimated value of the mental activity state corresponding to each of the plurality of types of brain activity signals,
    Based on the magnitude of each of the calculated errors, a first type brain activity signal is selected from the plurality of types of brain activity signals,
    A method of outputting information indicating the first type brain activity signal.

Images