1. A vehicle control system that gives an operation instruction to a control device that controls a control target of a vehicle, the vehicle control system comprising:

a voice recognition device that recognizes an external voice, which is a voice from outside the vehicle, and transmits an operation instruction for the control target recognized from the external voice to the control device after recognizing a predetermined term from the external voice; and

a user detection device that detects whether a user is present within a range of a prescribed distance from the vehicle,

the voice recognition device transmits, to the control device, an operation instruction for the control target recognized from the external voice even if the predetermined term is not recognized from the external voice when the user detection device detects the user.

2. The vehicle control system according to claim 1,

the vehicle control system includes an elapsed time measuring device that measures the elapse of a predetermined time from the time when the user detecting device detects the user,

the voice recognition device transmits an operation instruction for the control target recognized from the external voice to the control device even if the predetermined word or phrase is not recognized from the external voice when the elapsed time measurement device is measuring the elapse of the predetermined time,

when the elapsed time measuring device measures that the predetermined time has elapsed, the operation instruction for the control target recognized from the external voice is transmitted to the control device after the predetermined word is recognized from the external voice.

3. The vehicle control system according to claim 2,

the voice recognition device transmits the operation instruction for the control target recognized from the external voice to the control device after recognizing the predetermined term from the external voice even when the elapsed time measurement device measures the elapse of the predetermined time after transmitting the operation instruction for the control target recognized from the external voice to the control device.

4. The vehicle control system according to any one of claims 1 to 3,

the user detection device detects the presence of the user by detecting whether a portable machine carried by the user is located within a prescribed distance from the vehicle.

5. A vehicle control method executed by a voice recognition device that recognizes an external voice as a voice from outside a vehicle, and transmits an operation instruction for a control target of the vehicle recognized from the external voice to a control device that controls the control target after recognizing a predetermined word or phrase from the external voice,

when the user detection device detects a user within a range of a predetermined distance from the vehicle, the operation instruction for the control object recognized from the external voice is transmitted to the control device even if the predetermined sentence is not recognized from the external voice.

6. A recording medium on which a vehicle control program is recorded, the vehicle control program being executed by a processor of a voice recognition device that recognizes an external voice that is a voice from outside a vehicle, and transmits an operation instruction for a control target of the vehicle recognized from the external voice to a control device that controls the control target after recognizing a predetermined term from the external voice,

when the user detection device detects a user within a range of a predetermined distance from the vehicle, the processor transmits an operation instruction for the control object recognized from the external voice to the control device even if the predetermined sentence is not recognized from the external voice.

Background

Conventionally, there is known a technology for recognizing a voice of a user located outside a vehicle to perform vehicle control such as unlocking a door lock. For example, patent document 1 discloses the following system: the approach of a user is detected, a window of a vehicle is opened, a voice of the user located outside the vehicle is acquired by a microphone provided in the vehicle interior, and the vehicle is controlled in accordance with an operation indicated by the acquired voice.

Patent document 1: japanese patent laid-open publication No. 2019-167007

In recent years, as the system described in patent document 1, there is a system in which: after a predetermined sentence, so-called wake-up word, for starting voice recognition is recognized, the vehicle can be operated by voice. However, in such a system, a waiting time, which is an amount of time required for the user to say the wake-up word and for the system to recognize the wake-up word, is generated before the vehicle can be operated by voice. Therefore, when a user located outside the vehicle approaches the vehicle and tries to operate the vehicle by voice, such a system does not enter a state in which the user can operate the vehicle by voice until the user arrives at the vehicle, based on the time when the user utters the wakeup word, the speed of approaching the vehicle, and the like. For example, in such a system, there may occur a case where, when a user located outside the vehicle wants to unlock a door lock by voice and enter the vehicle, the user cannot enter the vehicle quickly although the user arrives at the vehicle.

Disclosure of Invention

The present invention has been made in view of the above circumstances, and an object thereof is to improve the usability of a vehicle when a user located outside the vehicle approaches the vehicle and operates the vehicle by voice.

One aspect of the present invention is a vehicle control system that instructs a control device that controls a control target of a vehicle to operate, the vehicle control system including: a voice recognition device that recognizes an external voice, which is a voice from outside the vehicle, and transmits an operation instruction for the control target recognized from the external voice to the control device after recognizing a predetermined term from the external voice; and a user detection device that detects whether or not a user is present within a range of a predetermined distance from the vehicle, wherein the voice recognition device transmits, to the control device, an operation instruction for the control object recognized from the external voice even if the predetermined sentence is not recognized from the external voice when the user detection device detects the user.

In the vehicle control system, the vehicle control system may be configured to include an elapsed time measuring device that measures elapsed time from a time when the user detecting device detects the user, the elapsed time measuring device measures the elapse of a predetermined time, and when the elapsed time measuring device measures the elapse of the predetermined time, transmitting an operation instruction for the control target recognized from the external speech to the control device even if the predetermined word or phrase is not recognized from the external speech, when the elapsed time measuring device measures that the predetermined time has elapsed, the operation instruction for the control target recognized from the external voice is transmitted to the control device after the predetermined word is recognized from the external voice.

In the vehicle control system, the voice recognition device may be configured to transmit the operation instruction for the control target recognized by the external voice to the control device after recognizing the predetermined sentence by the external voice even when the elapsed time measurement device measures the elapse of the predetermined time after transmitting the operation instruction for the control target recognized by the external voice to the control device.

In the vehicle control system, the user detection device may detect whether or not the user is present by detecting whether or not a portable device carried by the user is located within a predetermined distance from the vehicle.

Another aspect of the present invention is a vehicle control method executed by a voice recognition device that recognizes an external voice, which is a voice from outside a vehicle, and that, after recognizing a predetermined word or phrase from the external voice, transmits an operation instruction for a control target of the vehicle recognized from the external voice to a control device that controls the control target, wherein, when a user is detected by a user detection device within a range of a predetermined distance from the vehicle, the operation instruction for the control target recognized from the external voice is transmitted to the control device even if the predetermined word or phrase is not recognized from the external voice.

In another aspect of the present invention, there is provided a recording medium on which a vehicle control program is recorded, the vehicle control program being executed by a processor of a voice recognition device that recognizes an external voice that is a voice from outside a vehicle, and transmits an operation instruction for a control target of the vehicle recognized from the external voice to a control device that controls the control target after a predetermined term is recognized from the external voice, wherein the processor transmits the operation instruction for the control target recognized from the external voice to the control device even if the predetermined term is not recognized from the external voice when a user is detected by a user detection device within a predetermined distance from the vehicle.

Effects of the invention

According to the present invention, when a user located outside the vehicle approaches the vehicle and operates the vehicle, the convenience of use of the vehicle can be improved.

Drawings

Fig. 1 is a diagram showing a vehicle mounted with a vehicle control system.

Fig. 2 is a diagram showing a configuration of a vehicle control system.

Fig. 3 is a flowchart showing an operation of the vehicle control system.

Fig. 4 is a flowchart showing the operation of the operation instructing unit.

Fig. 5 is a diagram for explaining a relationship between a position of a user of the vehicle and an operation timing of the vehicle control system.

Description of the reference symbols

1: a vehicle; 2. 3: a portable machine; 100: a vehicle control system; 120: a user detection ECU (user detection means); 130: a voice recognition ECU (voice recognition device); 140: an elapsed time measurement ECU (elapsed time measurement device); 150: a door lock control ECU (control device); 151: a front door lock mechanism (control target); 152: a rear door lock mechanism (control target); 153: a tail gate lock mechanism (control target); 1311: a processor; 1323: a vehicle control program; p: a user.

Detailed Description

Fig. 1 is a diagram showing a configuration of a vehicle 1 on which a vehicle control system 100 is mounted.

The vehicle control system 100 is mounted on the vehicle 1, and instructs a control device to control a control target of the vehicle 1 based on a voice instruction from the user P.

Specifically, the vehicle control system 100 acquires a voice included in a voice outside the vehicle 1 (hereinafter referred to as "external voice") from, for example, a voice input unit 110A that is a microphone provided in the driver's seat door 1A of the vehicle 1 (hereinafter referred to as "external voice"). Then, the vehicle control system 100 recognizes the external voice, extracts an instruction voice instructing a certain motion to the vehicle 1 from the recognized external voice, and executes the motion indicated by the extracted instruction voice.

In this case, the vehicle control system 100 determines that the instruction voice acquired from the sound inputter 110A is the voice of the user P of the vehicle 1 and is the instruction voice directed to the vehicle 1, based on the utterance content of the user P and the presence or absence of the portable device 2 carried by the user P.

Here, the portable device 2 may be a so-called FOB (Frequency Operated Button) Button or a portable terminal device such as a smartphone that performs short-range communication.

In fig. 1, for example, a user P of a vehicle 1 approaches the vehicle 1 with a portable device 2, and says "good, my car, opens the door of the driver's seat". Here, the above-mentioned "good, my car" section is a word that is a word spoken by the user before the voice instruction is spoken to the vehicle 1, which is a so-called "wake-up word", and is defined in advance. In addition, the portion of "opening the door of the driver's seat" mentioned above corresponds to the instruction voice. In addition, "good, my car" is just one example of a wake word, and the wake word may be a sentence containing a predetermined arbitrary word.

First, the vehicle control system 100 detects the presence of the portable device 2 by communicating with the portable device 2, and detects that the user P is located around the vehicle 1. Then, the vehicle control system 100 recognizes the utterance of the user P after the utterance is acquired by the voice input device 110A, recognizes the presence of the wakeup word from the recognized utterance, and recognizes and extracts the instruction voice. Then, the vehicle control system 100 determines whether or not to transmit the voice instruction recognized or extracted from the utterance to the corresponding control device, based on the detection result of the portable device 2 and the utterance content of the user P.

In the above example, since the user P is around the vehicle 1 and the external voice includes the wakeup word, there is a high possibility that the instruction voice included in the external voice is an instruction voice uttered by the user P toward the vehicle 1. Therefore, in the above example, the vehicle control system 100 detects the portable device 2 and recognizes the wakeup word, thereby executing the operation indicated by the instruction voice, that is, unlocking the driver's seat door 1A. In the case of the present embodiment described later, the vehicle control system 100 unlocks the driver's seat door 1A after detecting the portable device 2 even if the wakeup word is not recognized.

Fig. 2 is a diagram showing the configuration of the vehicle control system 100 shown in fig. 1.

The vehicle Control system 100 includes a plurality of ECUs (Electronic Control units). That is, the vehicle control system 100 includes the sound obtainment ECU110, the user detection ECU120, the voice recognition ECU130, and the elapsed time measurement ECU 140.

The user detection ECU120 corresponds to the user detection means of the present invention. The voice recognition ECU130 corresponds to a voice recognition apparatus of the present invention. The elapsed time measurement ECU140 corresponds to the elapsed time measurement device of the present invention.

The sound obtaining ECU110, the user detection ECU120, the voice recognition ECU130, and the elapsed time measurement ECU140 are communicably connected to each other via an in-vehicle network bus 170. Here, the in-vehicle Network bus 170 is, for example, a CAN (Control Area Network) bus for performing communication based on a CAN communication standard.

The sound obtainment ECU110 obtains the external sound of the vehicle 1 from the sound inputter 110A, and outputs data representing the obtained external sound to the voice recognition ECU 130.

The user detection ECU120 detects whether the user P is within a prescribed distance range from the vehicle 1.

The user detection ECU120 includes a detection device control unit 121, a short-range wireless communication unit 122, and a detection device NW communication unit 123.

The detection device control unit 121 includes a processor 1211 that executes programs such as a CPU and an MPU, and a detection device storage unit 1212, and controls each unit of the user detection ECU 120. The detection device control unit 121 executes various processes in cooperation with hardware and software, so that the processor 1211 reads out a control program stored in the detection device storage unit 1212 and executes the processes.

The detection device storage unit 1212 has a storage area for storing a program executed by the processor 1211 and data processed by the processor 1211. The detection device storage unit 1212 stores a control program executed by the processor 1211 and other various data. The detection device storage unit 1212 has a nonvolatile storage area in which programs and data are stored in a nonvolatile manner. The detection device storage unit 1212 may include a volatile storage area, and may constitute a work area for temporarily storing a program executed by the processor 1211 or data to be processed.

The detection device storage unit 1212 stores identification information 1213 of a predetermined portable device 2. Here, the predetermined portable device 2 indicates a portable device 2 registered in advance as a portable device 2 of a user P (for example, an owner of the vehicle 1 and/or a borrower who obtains a license of the owner) having a use authority or an entry authority of the vehicle 1. In the following description, in order to distinguish a predetermined portable device 2 from portable devices 2 other than the predetermined portable device 2, the predetermined portable device 2 is referred to as a "portable device 3".

The detection device control unit 121 detects whether the user P is present within a predetermined distance range from the vehicle 1 by detecting whether the portable device 3 is present within the predetermined distance range from the vehicle 1. The detection device control unit 121 periodically attempts communication with the portable device 2 via the short-range wireless communication unit 122. When the communication is successfully performed, the detection device control unit 121 acquires the identification information 1213 from the portable device 2. When the acquired identification information 1213 matches the identification information 1213 of the portable device 3 registered in advance, the detection device control unit 121 detects that the portable device 3 is present within a predetermined distance range from the vehicle 1.

After attempting to communicate with the mobile device 2, the detection device control unit 121 transmits user detection information indicating the detection or non-detection of the mobile device 3 (that is, a predetermined mobile device 2) to the voice recognition ECU130 and the elapsed time measurement ECU140 via the detection device NW communication unit 123.

The short-range wireless communication unit 122 has communication hardware conforming to a predetermined short-range wireless communication standard, and performs wireless communication with the portable device 2 under the control of the detection device control unit 121. The short-range wireless communication unit 122 performs wireless communication with the portable device 2 held by the user P, with a range (for example, a range of 3.5m from the vehicle 1) of a predetermined distance from the vehicle 1 as a communicable range.

The detection device NW communication unit 123 is configured by, for example, a CAN transceiver that performs communication according to the CAN communication standard, and communicates with other ECUs such as the voice recognition ECU130 via the in-vehicle network bus 170 according to the control of the detection device control unit 121.

The voice recognition ECU130 is a device that recognizes an external voice. When the user detection ECU120 detects the user P, the voice recognition ECU130 transmits an operation instruction for the control object of the vehicle 1 recognized from the external voice to the control device.

In the present embodiment, the front door lock mechanism 151, the rear door lock mechanism 152, and/or the rear door lock mechanism 153 are exemplified as objects to be controlled by the vehicle 1, and the door lock control ECU150 is exemplified as a control device for controlling the objects to be controlled by the vehicle 1. Thus, the voice recognition ECU130 recognizes an operation instruction to unlock and/or lock the front door lock mechanism 151, the rear door lock mechanism 152, and/or the rear door lock mechanism 153 from an external voice, and transmits the operation instruction to the door lock control ECU 150. However, this is merely an example, and the control device instructed to operate by the vehicle control system 100 is not limited to the door lock control ECU 150. The vehicle control system 100 can issue an operation instruction to an arbitrary control device that controls an arbitrary control target of the vehicle 1 based on a voice instruction from the user P. Such control devices may include various electrical device control ECUs that control operations of various electrical devices (for example, an air conditioner, a heater, and a lamp) provided in the vehicle 1, respectively, and a drive control ECU that controls operations including start-up, torque generation, and the like of an internal combustion engine, for example, as a drive device of the vehicle 1.

The voice recognition ECU130 includes a voice recognition device control unit 131 and a voice recognition device NW communication unit 132.

The voice recognition device control unit 131 includes a processor 1311 such as a CPU and MPU that executes programs, and a voice recognition device storage unit 1312 (recording medium), and controls each unit of the voice recognition ECU 130. The voice recognition device control unit 131 executes various processes in cooperation with hardware and software so that the processor 1311 reads a control program stored in the voice recognition device storage unit 1312 and executes the processes. The speech recognition device control unit 131 functions as a speech recognition unit 131A, a wake word detection unit 131B, an instruction extraction unit 131C, and an operation instruction unit 131D by reading out and executing the vehicle control program 1323 stored in the speech recognition device storage unit 1312 by the processor 1311.

The voice recognition device storage 1312 has a storage area in which a program executed by the processor 1311 and data processed by the processor 1311 are stored. The voice recognition device storage unit 1312 stores a control program executed by the processor 1311, wakeup word data 1322 indicating a wakeup word that is a predetermined term, a vehicle control program 1323, and other various data. The voice recognition device storage unit 1312 has a nonvolatile storage area in which programs and data are stored in a nonvolatile manner. The voice recognition device storage unit 1312 may have a volatile storage area and constitute a work area for temporarily storing the program executed by the processor 1311 and data to be processed.

The voice recognition unit 131A recognizes an external voice included in the external voice indicated by the data received from the voice obtaining ECU 110. Then, the voice recognition section 131A sends the voice recognition result as a text to the awakened word detection section 131B and the instruction extraction section 131C.

The awakening word detection unit 131B determines whether or not an awakening word is included in the voice recognition result received from the voice recognition unit 131A. More specifically, the wakeup word detection unit 131B determines whether or not a word matching the wakeup word indicated by the wakeup word data 1322 is included in the result of speech recognition. The wakeup word detection unit 131B transmits the determination result, that is, wakeup word detection information indicating whether or not a wakeup word is included, to the operation instruction unit 131D.

The instruction extracting unit 131C extracts, for example, an instruction voice instructing an operation of the control target of the vehicle 1 as a text based on the result of the voice recognition received from the voice recognizing unit 131A. The instruction extracting unit 131C generates an operation instruction corresponding to the instruction voice from the extracted text.

The operation instruction generated by the instruction extracting unit 131C is a control instruction including a control target to which the operation instruction should be executed, the content of the operation, and device identification information of the corresponding control device. For example, the voice recognition unit 131A recognizes "good, my car," and opens the door of the driver seat. In the case of "such an external voice," an operation instruction including information indicating that the front door lock mechanism 151 is to be controlled, information indicating that the driver's door 1A is to be unlocked as operation content, and device identification information of the door lock control ECU150 is generated as a control command based on an instruction voice such as "open the driver's seat door".

The instruction extracting unit 131C transmits the generated operation instruction to the operation instructing unit 131D together with the extracted text.

The operation instructing unit 131D executes an operation described later, determines whether or not the operation instruction generated by the instruction extracting unit 131C should be transmitted to the corresponding control device, and transmits the operation instruction to the control device based on the determination result. This transmission is performed, for example, by transmitting a control command as an operation instruction to the in-vehicle network bus 170 via the voice recognition device NW communication unit 132. The transmitted control command is received by the control device corresponding to the device identification code included in the control command, and the received control device executes the operation content related to the control target indicated by the control command.

The voice recognition device NW communication unit 132 is configured by, for example, a CAN transceiver that performs communication according to the CAN communication standard, and communicates with another ECU such as the sound acquisition ECU110 via the in-vehicle network bus 170 according to the control of the voice recognition device control unit 131.

When the user detection ECU120 detects that the user P is present within the predetermined distance range from the vehicle 1, and the elapsed time measurement ECU140 detects this, the elapsed time measurement ECU140 measures the elapse of a predetermined time from the detected time. For example, when receiving user detection information indicating that the portable device 3 is detected from the user detection ECU120, the elapsed time measurement ECU140 measures the elapse of a predetermined time period from the time when it is determined that the user detection information is received.

As shown in fig. 2, the door lock control ECU150 is connected to the in-vehicle network bus 170. The door lock control ECU150 is connected to a front door lock mechanism 151, a rear door lock mechanism 152, and a rear door lock mechanism 153, and controls the locked and unlocked states of these mechanisms. The front door lock mechanism 151 is an actuator for locking and unlocking the door locks of 2 front doors located on the left and right sides of the driver seat and the passenger seat of the vehicle 1. The rear door lock mechanism 152 is an actuator for locking and unlocking the door locks of 2 rear doors located on the left and right sides of the rear passenger seat of the vehicle 1. The tailgate door lock mechanism 153 is an actuator for locking and unlocking a door lock of a door of a rear trunk room of the vehicle 1.

As shown in fig. 2, a lamp control ECU160 is connected to the in-vehicle network bus 170. The lamp control ECU160 is connected to a headlight 161, an interior lamp 162, and a tail lamp 163, and controls the lighting state of these lamps. The lamp control ECU160 controls the lighting state of the connected lamp in accordance with the operation state of the voice recognition ECU 130. The lamps whose lighting states are controlled by lamp control ECU160 are not limited to these lamps, and may be more or less.

Next, the operation of the vehicle control system 100 is explained.

Fig. 3 is a flowchart showing the operation of the vehicle control system 100.

The detection device control unit 121 of the user detection ECU120 of the vehicle control system 100 determines whether or not the presence of the user P of the vehicle 1 within a predetermined distance range from the vehicle 1 is detected (step SA 1). As described above, the detection device control unit 121 detects whether or not the user of the vehicle 1 is present within the predetermined distance range from the vehicle 1 by detecting whether or not the portable device 3 is present within the predetermined distance range from the vehicle 1.

When detecting that the user P of the vehicle 1 is present within the predetermined distance range from the vehicle 1 (yes in step SA1), the detection device control unit 121 transmits user detection information indicating that the portable device 3 is detected to the voice recognition ECU130 and the elapsed time measurement ECU140 (step SA 2).

The elapsed time measurement ECU140 starts measuring the elapse of a predetermined time period when receiving user detection information indicating that the portable device 3 has been detected (step SA 3).

Next, the sound obtainment ECU110 determines whether an external sound is detected based on the signal from the sound inputter 110A (step SA 4). For example, when the amplitude of the signal from the sound inputter 110A is above a prescribed level, the sound obtainment ECU110 makes an affirmative determination, i.e., determines that the external sound is detected, in step SA 4.

When the sound obtainment ECU110 determines that the external sound is not detected (step SA 4): yes), the detection device control unit 121 of the user detection ECU120 determines whether or not the presence of the user P of the vehicle 1 within a range of a predetermined distance from the vehicle 1 is detected (step SA 5).

If detecting device control unit 121 determines that user P of vehicle 1 is detected within a range of a predetermined distance from vehicle 1 (yes in step SA5), sound obtaining ECU110 executes the process of step SA4 again.

On the other hand, if the detection device control unit 121 determines that the user P of the vehicle 1 is not detected within the range of the predetermined distance from the vehicle 1 (no in step SA5), the process returns to step SA1, and the determination in step SA1 is performed again.

Returning to the description of step SA4, if the sound obtaining ECU110 determines that an external sound is detected (yes in step SA4), it transmits data indicating the detected external sound to the speech recognition ECU130 (step SA 6).

The voice recognition unit 131A of the voice recognition ECU130 performs voice recognition on the external voice from the voice obtaining ECU110, and recognizes the external voice included in the external voice (step SA 7). The voice recognition unit 131A sends the result of the voice recognition to the awakened word detection unit 131B and the instruction extraction unit 131C.

Next, the wake word detector 131B determines whether or not a wake word is included in the voice recognition result (step SA 8). The wakeup word detection unit 131B transmits wakeup word detection information indicating the determination result to the operation instruction unit 131D.

The instruction extracting unit 131C extracts an instruction voice from the result of voice recognition (step SA9), and generates an operation instruction from the extracted instruction voice (step SA 10). Then, the instruction extracting unit 131C transmits the text indicating the extracted instruction voice and the generated operation instruction to the operation instructing unit 131D.

Next, the operation instruction unit 131D performs an operation instruction transmission process (step SA 11).

Fig. 4 is a flowchart showing the operation of the operation instructing unit 131D in the operation instruction transmission process.

The operation instructing unit 131D determines whether or not the elapsed time measurement ECU140 is measuring the elapse of a predetermined time (step SB 1).

When it is determined that the elapsed time measurement ECU140 has not measured the elapse of the predetermined time (no at step SB1), in other words, when it is determined that the measurement of the elapse of the predetermined time has ended, the operation instructing unit 131D determines whether or not the awaking word has been detected from the external voice based on the awaking word detection information transmitted from the awaking word detecting unit 131B (step SB 3).

When determining that the wake word is detected from the external speech (step SB 3): yes), an operation instruction is transmitted to the control device (step SB 4).

On the other hand, when determining that the wakeup word has not been detected from the external voice (no in step SB3), the operation instructing unit 131D does not transmit the operation instruction to the control device (step SB 5).

Returning to the description of step SB1, when it is determined that the elapsed time measurement ECU140 has measured the elapse of the predetermined time (yes at step SB1), it is determined whether or not the operation instruction has been transmitted to the control device more than once since the elapsed time measurement ECU140 started the measurement of the elapse of the predetermined time (step SB 2).

When determining that the operation instruction unit 131D has transmitted the operation instruction to the control device more than once from the time when the elapsed time measurement ECU140 starts measuring the elapse of the predetermined time (yes in step SB2), the process proceeds to step SB 3.

On the other hand, when determining that the operation instruction has not been transmitted to the control device more than once since the elapsed time measurement ECU140 started the measurement of the elapse of the predetermined time (no at step SB2), the operation instructing unit 131D transmits the operation instruction (step SB 4).

When operation instructing unit 131D executes the process of step SB4 or the process of step SB5, the process of vehicle control system 100 returns to the flowchart shown in fig. 3, and the process of step SA4 is executed again.

Next, the operation of the vehicle control system 100 will be described with reference to specific examples.

Fig. 5 is a diagram for explaining a relationship between the position of the user P of the vehicle 1 and the operation timing of the vehicle control system 100.

Fig. 5 illustrates a case where the user P of the vehicle 1 attempts to enter the vehicle 1 by unlocking the door lock of the driver's seat with a voice while approaching the vehicle 1 from the position I1.

In fig. 5, a position I1 is a position where the user detection ECU120 cannot detect the user P, that is, a position outside the range of a prescribed distance from the vehicle 1.

Further, the position I2 is a position closer to the vehicle 1 than the position I1, and is a position farthest from the vehicle 1 where the user detection ECU120 can detect the user P.

Further, the position I3 is a position at which the user detection ECU120 can detect the user P, and is a position closer to the vehicle 1 than the position I2.

In addition, the position I4 is a position at which the user detection ECU120 can detect the user P, and is a position closer to the vehicle 1 than the position I3.

Further, the position I5 is a position at which the user detection ECU120 can detect the user P, and is a position closer to the vehicle 1 than the position I4.

Further, the position I6 is a position at which the user detection ECU120 can detect the user P, and is a position closer to the vehicle 1 than the position I5.

Further, the position I7 is a position at which the user detection ECU120 can detect the user P, and is a position closer to the vehicle 1 than the position I6. The position I7 is a position where the user P of the vehicle 1 is deemed to have arrived at the vehicle 1.

In fig. 5, a table CA indicates whether the user P of the vehicle 1 is present within a range of a predetermined distance from the vehicle 1. In table CA, the block diagram labeled with the letter "user not present" indicates that the user P of the vehicle 1 is not within a range of a predetermined distance from the vehicle 1, that is, the user detection ECU120 does not detect the presence of the user P of the vehicle 1. In table CA, the block diagram labeled with the letter "presence user" indicates that the user P of the vehicle 1 is within a range of a predetermined distance from the vehicle 1, that is, the user detection ECU120 can detect the presence of the user P of the vehicle 1.

In fig. 5, table CB shows a lighting state of the lamp connected to lamp control ECU 160. In table CB, the block diagram labeled with the letter "off" indicates that all the lamps connected to lamp control ECU160 are off. In table CB, the block diagram labeled with the letter "blink" indicates that at least one of the lamps connected to lamp control ECU160 is blinking. In table CB, the block diagram labeled with the letter "1 st lighting" indicates that at least one of the lamps connected to lamp control ECU160 is in a lighting state, which indicates that vehicle control system 100 is in a state of waiting for the utterance of user P. In table CC, the block diagram labeled with the letter "2 nd lighting" indicates that at least one of the lamps connected to lamp control ECU160 is in a lighting state indicating that vehicle control system 100 is executing processing based on the utterance of user P.

In fig. 5, a table CC shows an operation state of the vehicle control system 100. Table CC mainly shows the operation state of the voice recognition ECU 130. In table CC, the block diagram labeled with the word "sleep" indicates that the voice recognition ECU130 is in the sleep state. Note that, in table CC, the block diagram labeled with the word "activation processing" indicates that the voice recognition ECU130 is executing processing relating to activation. In table CC, the block diagram labeled with the letter "wait for utterance" indicates that the voice recognition ECU130 is waiting to receive data indicated by an external sound from the sound obtainment ECU 110. In table CC, the block diagram labeled with the word "signal processing" indicates that the vehicle control system 100 executes processing based on the external sound obtained by the sound obtaining ECU 110.

Set at time T1, user P of vehicle 1 is located at position I1. As shown in table CA, at position I1, the user detection ECU120 does not detect the presence of the user P of the vehicle 1. Therefore, as shown in table CC, the voice recognition ECU130 is in the sleep state at time TA. Further, the lamp connected to lamp control ECU160 is turned off.

The user P of the vehicle 1 approaches the vehicle 1 from the position I1, and the user P of the vehicle 1 is located at the position I2 at a time T2. At position I2, the user detection ECU120 can detect the presence of the user P of the vehicle 1. In fig. 5, at time T2, the user detection ECU120 does not perform detection of the portable device 3. Therefore, as shown in table CC, the voice recognition ECU130 is in the sleep state at time T2. Further, the lamp connected to lamp control ECU160 is turned off.

The user P of the vehicle 1 approaches the vehicle 1 from the position I2, and the user P of the vehicle 1 is located at the position I3 at a time T3. At position I3, the user detection ECU120 can detect the presence of the user P of the vehicle 1. In fig. 5, the time T3 is the time when the user detection ECU120 first detects the portable device 3 after the time T2. Therefore, as shown in table CC, at time T2, the voice recognition ECU130 starts the processing relating to the start. Further, since the voice recognition ECU130 is performing the processing related to the startup, at least one of the lamps connected to the lamp control ECU160 is in a blinking state. Further, elapsed time measurement ECU140 of vehicle control system 100 starts measuring the elapse of a predetermined time period after time T3.

The user P of the vehicle 1 approaches the vehicle 1 from the position I3, and the user P of the vehicle 1 is located at the position I4 at the time T4. At position I4, the user detection ECU120 can detect the presence of the user P of the vehicle 1. As shown in table CC, at time T4, the speech recognition ECU130 ends the processing relating to the start, and switches to a state of waiting for reception of data representing the external sound from the sound obtainment ECU 110. Further, at least one of the lamps connected to lamp control ECU160 is switched from a blinking state to a lighting state indicating a state of waiting for the sound production of user P.

The user P of the vehicle 1 further approaches the vehicle 1 from the position I4, and the user P of the vehicle 1 is located at the position I5 at the time T5. At position I5, the user detection ECU120 can detect the presence of the user P of the vehicle 1. At time T5, user P of vehicle 1 notices that vehicle control system 100 is in a state of waiting for the sounding of user P by the lighting state of the lamp, and says "open the door of the driver's seat". Then, as shown in table CC, after time T5, the vehicle control system 100 executes processing based on the external sound obtained by the sound obtaining ECU 110. Further, at least any one of the lamps connected to lamp control ECU160 is switched from a lighting state indicating that it is in a state of waiting for the utterance of user P to a lighting state indicating that vehicle control system 100 is executing processing based on the utterance of user P.

The user P of the vehicle 1 approaches the vehicle 1 further from the position I5, and at a time T6, the user P of the vehicle 1 is located at the position I6. In fig. 5, elapsed time measurement ECU140 measures the elapse of a predetermined time at least from time T3 to time T6. At position I6, the user detection ECU120 can detect the presence of the user P of the vehicle 1. After time T5, the vehicle control system 100 executes processing based on the external sound obtained by the sound obtaining ECU 110. In fig. 5, the utterance at time T5 is the utterance of the user P of the vehicle 1 for the first time after the user detection ECU120 detects the user P. Therefore, although the utterance does not include the wake-up word, the vehicle control system 100, in the operation shown in fig. 4, via step SB 2: since no executes the processing, the door lock of the driver seat is unlocked from time T5 to time T6.

The user P of the vehicle 1 approaches the vehicle 1 further from the position I6, and at a time T7, the user P of the vehicle 1 is located at the position I7. At position I7, the user detection ECU120 can detect the presence of the user P of the vehicle 1. Since the door lock of the driver's seat has been unlocked at position 6, the user P of the vehicle 1 can quickly enter the vehicle 1 after reaching the vehicle 1.

Here, it is assumed that the following structure is adopted: after the vehicle control system 100 recognizes the wake-up word, the door lock of the driver seat can be unlocked by an external voice. In the case of this configuration, in the vehicle control system 100, a waiting time corresponding to the time required for the user P to speak the wake-up word and for the system to recognize the wake-up word is generated until the door lock of the driver's seat can be unlocked by voice. Therefore, in the case of this configuration, depending on the timing of the utterance of the awakening word, the speed of the approaching vehicle 1, and the like, it is not always possible to unlock the door lock of the driver's seat before the user P is located at the position I7, and there is a possibility that the user P cannot rapidly enter the vehicle 1 even if the user P arrives at the vehicle 1. However, in the configuration of the vehicle control system 100 according to the present embodiment, when the user detection ECU120 detects the user P, the operation instruction recognized from the external voice is transmitted to the door lock control ECU150 without recognizing the wakeup word from the external voice. Thus, the vehicle control system 100 according to the present embodiment can unlock the door lock of the driver seat in advance by the time required for the user P to speak the wake-up word and for the system to recognize the wake-up word, and the user P can quickly enter the vehicle 1 after the user P arrives at the vehicle 1.

As described above, in fig. 5, in the vehicle control system 100, the elapsed time measurement ECU140 starts measurement of the elapse of the predetermined time after the time T3. When the elapsed time measurement ECU140 measures the elapse of the predetermined time after time T6, the vehicle control system 100 executes the operation shown in fig. 4 via step SB2 when the sound obtaining ECU110 obtains the external sound: is the process of (1). Therefore, even when the elapsed time measurement ECU140 measures the elapse of the predetermined time after the time T6, the vehicle control system 100 recognizes the wakeup word from the external voice and then transmits the operation instruction recognized from the external voice. In other words, when the elapsed time measurement ECU140 measures the elapse of the predetermined time after time T6, the vehicle control system 100 does not transmit the operation instruction recognized by the external voice when the wakeup word cannot be recognized by the external voice. Thus, after time T6, vehicle control system 100 does not transmit an operation instruction to the control device unless the wake-up word is included in the speech of user P of vehicle 1 or the third party. Therefore, the vehicle control system 100 can prevent the vehicle 1 from performing an operation not intended by the user P after the time T6. In addition, it is possible to prevent an illegal operation of the vehicle 1 by a third person, and to improve the safety of the vehicle 1.

As described above, the vehicle control system 100 that instructs the operation of the control device that controls the control target of the vehicle 1 includes: a voice recognition ECU130 that recognizes an external voice from outside the vehicle 1, and after recognizing a wakeup word from the external voice, transmits an operation instruction for the control target recognized from the external voice to the control device; and a user detection ECU120 that detects whether the user P is present within a range of a prescribed distance from the vehicle 1. When the user detection ECU120 detects the user P, the voice recognition ECU130 transmits an operation instruction for the control target recognized from the external voice to the control device even if the wakeup word is not recognized from the external voice.

In the vehicle control method executed by the voice recognition ECU130, when the user detection ECU120 detects the user P within a range of a predetermined distance from the vehicle 1, an operation instruction for the control target recognized from the external voice is transmitted to the control device even if the wakeup word is not recognized from the external voice.

The vehicle control program 1323 stored in the voice recognition device storage unit 1312 and executed by the processor 1311 of the voice recognition ECU130 causes the processor 1311 to perform the following operations: when the user detection ECU120 detects the user P within a range of a predetermined distance from the vehicle 1, the operation instruction for the control target recognized from the external voice is transmitted to the control device even if the wakeup word is not recognized from the external voice.

According to the vehicle control system 100, the vehicle control method, and the vehicle control program 1323 recorded in the voice recognition device storage unit 1312, it is possible to control the control target of the vehicle 1 while shortening the time required for the user P to say the wake word and for the system to recognize the wake word. Therefore, when the user P located outside the vehicle 1 approaches the vehicle 1 and attempts to operate the vehicle 1 by voice, the vehicle control system 100, the vehicle control method, and the vehicle control program 1323 recorded in the voice recognition device storage unit 1312 can set the state of the vehicle 1 to a state in which the user P can operate the vehicle 1 by voice before the user P arrives at the vehicle 1. Thus, the vehicle control system 100, the vehicle control method, and the vehicle control program 1323 recorded in the voice recognition device storage unit 1312 enable the user P to quickly operate the vehicle 1 after arriving at the vehicle 1, and can improve the usability of the vehicle 1 when the user P located outside the vehicle 1 approaches the vehicle 1 and operates the vehicle 1 by voice.

The vehicle control system 100 includes an elapsed time measurement ECU140, and the elapsed time measurement ECU140 measures the elapse of a predetermined time period from the time when the user detection ECU120 detects the user P. When the elapsed time measurement ECU140 measures the elapse of the predetermined time, the voice recognition ECU130 transmits an operation instruction for the control target recognized from the external voice to the control device even if the wakeup word is not recognized from the external voice. When the elapsed time measurement ECU140 measures that the predetermined time has elapsed, the speech recognition ECU130 transmits the operation instruction for the control target recognized from the external speech to the control device after recognizing the wakeup word from the external speech.

According to this configuration, since it is possible to prevent the vehicle 1 from being operated without the wakeup word for a long time after the user detection ECU120 detects the user P, it is possible to prevent an illegal operation of the vehicle 1 and to improve the safety of the vehicle 1.

After transmitting the operation instruction for the control target recognized from the external voice to the control device, even when the elapsed time measurement ECU140 measures the elapse of the predetermined time, the voice recognition ECU130 transmits the operation instruction for the control target recognized from the external voice to the control device after recognizing the wakeup word from the external voice.

According to this configuration, after the user detection ECU120 detects the user P, the wakeup word is not required only for the first operation of the vehicle 1, and the wakeup word is required for the second and subsequent operations of the vehicle 1. Therefore, the vehicle control system 100 can prevent the vehicle 1 from performing an unexpected action by the user P after the first operation of the vehicle 1 is performed. For example, even if the user P of the vehicle 1 unlocks by voice and the conversation includes a locked word, the vehicle control system 100 does not lock unless the conversation includes a wakeup word, and thus it is possible to prevent the user P from locking undesirably.

In addition, according to this configuration, the vehicle control system 100 can prevent an unauthorized operation of the vehicle 1 by a third person, and can improve the safety of the vehicle 1.

The user detection ECU120 detects the presence or absence of the user P by detecting whether the portable device 3 carried by the user P is located within a range of a prescribed distance from the vehicle 1.

According to this configuration, since the presence or absence of the user P is detected by detecting whether or not the portable device 3 is located within a predetermined distance from the vehicle 1, the user P may carry the portable device 3 when approaching the vehicle 1 and operating the vehicle 1 by voice. Therefore, the user P does not need to take a dedicated action for the user detection ECU120 to detect its presence. Therefore, when the user P located outside the vehicle 1 approaches the vehicle 1 and operates the vehicle 1 by voice, convenience can be improved.

The present invention is not limited to the configurations of the above embodiments, and can be implemented in various ways without departing from the scope of the invention.

For example, although the sound input device 110A is provided in the driver's seat door 1A of the vehicle 1 in the above-described embodiment, the present invention is not limited thereto. The sound inputter 110A may be disposed at one or more arbitrary positions where voice can be acquired from the user P located outside the vehicle 1. For example, the sound input device 110A may be provided not in the driver's seat door 1A but in the front passenger seat door, the rear seat left and right doors, and/or the back door of the vehicle 1, or may be provided in the driver's seat door 1A and the front passenger seat door, the rear seat left and right doors, and/or the back door of the vehicle 1.

In the above embodiment, the user detection ECU120 communicates with the portable device 2 via the short-range wireless communication unit 122 regardless of whether the portable device 2 is a FOB or a portable terminal such as a smartphone, but is not limited to this. For example, communication with the portable device 2 as a portable terminal such as a smartphone may be performed by the user detection ECU120, and communication with the portable device 2 as a FOB may be performed by another ECU mounted on the vehicle 1, for example, a so-called BCU (Body Control Unit). At this time, the user detection ECU120 can acquire information on whether or not the portable device 2 as the FOB is detected from the BCU via the in-vehicle network bus 170.

In the above embodiment, the user detection ECU120 detects whether or not the user P is present within a range of a predetermined distance from the vehicle 1 by communicating with the portable device 2 of the user P, but the present invention is not limited thereto. For example, the user detection ECU120 may detect whether the user P is within a range of a predetermined distance from the vehicle 1 by recognizing the approaching user P using the in-vehicle camera, without communicating with the portable device 2 through the short-range wireless communication unit 122. For example, according to the related art, the user detection ECU120 extracts the face of an approaching person from a captured image of the in-vehicle camera, and compares the extracted face image with a face image of the user P stored in advance, thereby making a determination (i.e., face authentication) as to whether the approaching person is the predetermined user P, thereby being able to detect the approach of the user P.

In this case, according to the related art, the user detection ECU120 may calculate the distance from the captured user P to the vehicle 1 based on the standard size of the head of the person and the angle of view of the face image extracted as described above, and the optical characteristics of the in-vehicle camera and the like. Such an in-vehicle camera may be provided, for example, on the outer side of the driver's seat door 1A, on the outer side of the passenger seat, the rear seat, and/or the tailgate, and/or on the inner side of the vehicle interior of the front window. For example, the user detection ECU120 can detect the approach of the user P using images from cameras provided at these locations to provide the driver with external environment images for improving safety.

For example, the vehicle 1 may be a manually drivable vehicle that is driven by a driver performing an operation related to driving, or may be an automatically drivable vehicle that is driven automatically without performing an operation related to driving. The vehicle 1 is a vehicle such as an engine-driven four-wheel vehicle, a motor-driven electric vehicle, or a hybrid vehicle having a motor and an engine mounted thereon. The vehicle 1 may be a vehicle other than a four-wheel vehicle.

For example, in the above embodiment, the modules shown in fig. 2 are schematic diagrams showing the components classified according to the main processing contents in order to facilitate understanding of the present invention, and may be classified into more components according to the processing contents. Further, the classification can be performed so that 1 component executes more processing.

For example, the step units of the operations shown in fig. 3 and 4 are divided according to the main processing contents in order to easily understand the operations of the vehicle control system 100, and the present invention is not limited by the dividing method and name of the processing unit. Or may be divided into more step units according to the processing contents. Further, the division may be performed so that more processes are included in one step unit. The order of the steps may be appropriately changed within a range not to impair the gist of the present invention.