1. An automated driving test method, the method comprising:

acquiring real road test data of an automatic driving vehicle;

generating N simulation scenes based on N data fragments under a target semantic scene in the real road test data, wherein N is a positive integer;

running an automatic driving algorithm for testing in the N simulation scenes to obtain results of first test indexes in the N simulation scenes;

and determining a target test result in the target semantic scene according to the result of the first test index in the N simulation scenes.

2. The method of claim 1, wherein the running an autopilot algorithm for testing in the N simulation scenarios, resulting in a result of a first test indicator in the N simulation scenarios, comprises:

running an automatic driving algorithm under the N simulation scenes for testing to obtain output data of the automatic driving algorithm under the N simulation scenes;

and calculating the result of the first test index under the N simulation scenes through the output data of the automatic driving algorithm under the N simulation scenes.

3. The method of claim 1, the determining a target test result in the target semantic scenario from the result of the first test indicator in the N simulation scenarios, comprising:

summarizing results of the first test indexes in the N simulation scenes, and determining results of second test indexes in the target semantic scene, wherein the second test indexes correspond to the first test indexes and are the same in number, and the target test results comprise the results of the second test indexes.

4. The method of claim 1, wherein after obtaining real road test data for the autonomous vehicle, further comprising:

performing semantic segmentation on the real road test data to obtain N data segments of the target semantic scene;

wherein any two data segments do not overlap in time.

5. The method of claim 1, wherein target data segments include segment environmental information perceived by the autonomous vehicle and at least one of:

segment start time and segment end time;

a segment starting position of the autonomous vehicle and motion state information at the segment starting position;

high-precision map information of the segments;

wherein the target data segment is any one of the N data segments.

6. An autonomous driving test apparatus, the apparatus comprising:

the test data acquisition module is used for acquiring real road test data of the automatic driving vehicle;

the generating module is used for generating N simulation scenes based on N data fragments under a target semantic scene in the real road test data, wherein N is a positive integer;

the test module is used for running an automatic driving algorithm to perform testing under the N simulation scenes to obtain the result of a first test index under the N simulation scenes;

and the determining module is used for determining a target test result in the target semantic scene according to the result of the first test index in the N simulation scenes.

7. The apparatus of claim 6, wherein the test module comprises:

the output data acquisition module is used for running an automatic driving algorithm under the N simulation scenes for testing to obtain output data of the automatic driving algorithm under the N simulation scenes;

and the result determining module is used for calculating the results of the first test indexes under the N simulation scenes through the output data of the automatic driving algorithm under the N simulation scenes.

8. The apparatus of claim 6, the determining a target test result in the target semantic scenario from the result of the first test indicator in the N simulation scenarios, comprising:

summarizing results of the first test indexes in the N simulation scenes, and determining results of second test indexes in the target semantic scene, wherein the second test indexes correspond to the first test indexes and are the same in number, and the target test results comprise the results of the second test indexes.

9. The apparatus of claim 6, further comprising:

the segmentation module is used for performing semantic segmentation on the real road test data after the test data acquisition module acquires the real road test data of the automatic driving vehicle to obtain N data segments of the target semantic scene;

wherein any two data segments do not overlap in time.

10. The apparatus of claim 6, wherein target data segments include segment environmental information perceived by the autonomous vehicle and at least one of:

segment start time and segment end time;

a segment starting position of the autonomous vehicle and motion state information at the segment starting position;

high-precision map information of the segments;

wherein the target data segment is any one of the N data segments.

11. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the autopilot testing method of any of claims 1-5.

12. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to execute the automated driving test method of any one of claims 1-5.

13. A computer program product comprising a computer program which, when executed by a processor, implements an autopilot testing method according to any one of claims 1-5.

Background

With the continuous development of automatic driving technology, more and more automatic driving vehicles are provided, and the automatic driving vehicles are more and more intelligent. In the actual operation process, the automatic driving algorithm encounters very complex and various scenes, and the automatic driving algorithm needs to be tested in the scenes.

At present, a commonly adopted method is simulation test, and in the process of the simulation test, automatic driving algorithm test is mainly performed under the scene of manual design.

Disclosure of Invention

The disclosure provides an automatic driving test method and device and electronic equipment.

In a first aspect, an embodiment of the present disclosure provides an automatic driving test method, including:

acquiring real road test data of an automatic driving vehicle;

generating N simulation scenes based on N data fragments under a target semantic scene in the real road test data, wherein N is a positive integer;

running an automatic driving algorithm for testing in the N simulation scenes to obtain results of first test indexes in the N simulation scenes;

and determining a target test result in the target semantic scene according to the result of the first test index in the N simulation scenes.

In the automatic driving test method of the embodiment, a manual design scene and a test in the manual design scene are not needed, but N simulation scenes are generated according to N data segments in a target semantic scene in real road test data for the automatic driving simulation test, and an automatic driving algorithm is operated in the N simulation scenes for the test to obtain results of first test indexes in the N simulation scenes, and the target test result in the target semantic scene is determined according to the results of the first test indexes in the N simulation scenes, that is, the automatic driving test in the specific semantic scene (i.e., the target semantic scene) can be realized, so that the automatic driving test effect in the specific semantic scene can be improved.

In a second aspect, one embodiment of the present disclosure provides an automatic driving test device, the device comprising:

the test data acquisition module is used for acquiring real road test data of the automatic driving vehicle;

the generating module is used for generating N simulation scenes based on N data fragments under a target semantic scene in the real road test data, wherein N is a positive integer;

the test module is used for running an automatic driving algorithm to perform testing under the N simulation scenes to obtain the result of a first test index under the N simulation scenes;

and the determining module is used for determining a target test result in the target semantic scene according to the result of the first test index in the N simulation scenes.

In a third aspect, an embodiment of the present disclosure further provides an electronic device, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the autopilot testing method of the present disclosure as provided in the first aspect.

In a fourth aspect, an embodiment of the present disclosure also provides a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the automated driving test method provided by the present disclosure as the first aspect.

In a fifth aspect, an embodiment of the present disclosure provides a computer program product comprising a computer program which, when executed by a processor, implements the automated driving test method of the present disclosure as provided in the first aspect.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is one of the flow diagrams of an automated driving test method of one embodiment provided by the present disclosure;

FIG. 2 is a second schematic flow chart of an automatic driving test method according to an embodiment of the present disclosure;

FIG. 3 is a test schematic diagram of an automatic driving test method according to an embodiment of the disclosure

FIG. 4 is a test result distribution plot of hard brake ratio obtained by an automated driving test method of one embodiment provided by the present disclosure;

FIG. 5 is a block diagram of an autopilot test apparatus of one embodiment provided by the present disclosure;

FIG. 6 is a block diagram of an electronic device for implementing an autopilot test method of an embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

As shown in fig. 1, according to an embodiment of the present disclosure, the present disclosure provides an automatic driving test method, the method including:

step S101: real road test data of an autonomous vehicle is obtained.

The real road test data is data recorded by the autonomous vehicle during the driving process of the real road, and may include, for example and without limitation, driving data (e.g., position, speed, and operating state) of the autonomous vehicle itself, environment information detected by an environment (i.e., environment information collected by the autonomous vehicle), and map information (e.g., may be high-precision map information), and for example, the environment information may include traffic environment data around the autonomous vehicle (e.g., data of static or dynamic other traffic participants (e.g., other vehicles, pedestrians, etc.), signal lights, and traffic signs, etc.).

Step S102: and generating N simulation scenes based on N data fragments under the target semantic scene in the real road test data, wherein N is a positive integer.

During the driving process of the automatic driving vehicle in the real road, data under various semantic scenes can be recorded, and it can be understood that the real road test data can include data segments under a plurality of semantic scenes, for example, the semantic scenes in the real road test data can include but are not limited to semantic scenes such as crossing straight-ahead driving, crossing left-turning, crossing right-turning, non-crossing straight-ahead driving, non-crossing lane changing and non-crossing straight-ahead driven vehicle cutting, and the data segments under each semantic scene in the real road test data can be one or more. The target semantic scene is one of a plurality of semantic scenes in real road test data, and can be understood as a scene expected to be tested by an automatic driving algorithm, namely a specific semantic scene to be tested.

In this embodiment, N simulation scenes may be generated by using N data fragments in a target semantic scene, that is, N simulation scenes may be simulated by using real N data fragments, and each data fragment corresponds to a module to generate one simulation scene, so that N different simulation scenes are obtained, that is, N data fragments correspond to N simulation scenes one to one. Because the simulation scenes are generated through the data segments recorded in the driving process of the real road, the reality degree of the obtained simulation scenes can be improved, and the automatic driving test effect can be improved.

Step S103: and running an automatic driving algorithm for testing in N simulation scenes to obtain the result of the first test index in the N simulation scenes.

The test in the embodiment is a simulation test, that is, a simulation vehicle is simulated to run through an automatic driving algorithm in a simulation scene, so that the simulation test is realized. The automatic driving algorithm runs under N simulation scenes respectively, so that the simulation test of automatic driving is realized, and the result of the first test index of the automatic driving algorithm under each simulation scene can be obtained, namely, the result of the first test index can be correspondingly obtained under each simulation scene. As one example, the first test metric may include, but is not limited to, at least one of a somatosensory metric, a safety metric, a driving efficiency metric, and an intelligence metric. For example, the somatosensory index may include the number of hard brakes, and the hard brakes may be detected according to the acceleration and the change rate of the acceleration of the simulated vehicle in the simulation test, for example, a behavior that the acceleration is smaller than a preset acceleration and the change rate of the acceleration is smaller than a preset change rate in the braking process may be defined as a hard brake. As the safety index, the number of collisions and the like may be included. As the running efficiency index, a running vehicle speed, which may be expressed by the running mileage divided by the running time, or the like, may be included. The intelligent index may include at least one of a stagnation number and a redundant lane change number, where the stagnation may also be understood as a stuck state, which may be defined as a phenomenon that a simulated vehicle is not stopped before passing, and the stopped state may be understood as a state where a time period during which the vehicle speed is less than a preset threshold value is longer than a preset time period. The redundant lane change can be understood as simulating that the vehicle has ineffective lane change actions, which can also be called ineffective lane change times, for example, the lane change is from an original lane to a target lane, and the lane change is from the target lane back to the original lane, and no obstacle is passed in the middle.

Step S104: and determining a target test result in the target semantic scene according to the result of the first test index in the N simulation scenes.

After the results of the first test indexes under the N simulation scenes are obtained, the results of the first test indexes under the N simulation scenes can be used for determining the target test result under the target semantic scene, and the test under the target semantic scene is realized.

In the automatic driving test method of the embodiment, a manual design scene and a test in the manual design scene are not needed, but N simulation scenes are generated according to N data fragments in a target semantic scene in real road test data for the automatic driving simulation test, and an automatic driving algorithm is operated in the N simulation scenes for the test to obtain results of first test indexes in the N simulation scenes, and the target test result in the target semantic scene is determined according to the results of the first test indexes in the N simulation scenes, that is, the automatic driving test in the specific semantic scene (i.e., the target semantic scene) can be realized, so that the automatic driving test effect in the specific semantic scene can be improved.

In one embodiment, running an autopilot algorithm for testing in N simulation scenarios to obtain a result of a first test indicator in the N simulation scenarios comprises: running an automatic driving algorithm under N simulation scenes for testing to obtain output data of the automatic driving algorithm under the N simulation scenes; and calculating the result of the first test index under the N simulation scenes through the output data of the automatic driving algorithm under the N simulation scenes. That is, in the present embodiment, as shown in fig. 2, there is provided an automatic driving test method including:

step S201: acquiring real road test data of an automatic driving vehicle;

step S202: generating N simulation scenes based on N data fragments under a target semantic scene in real road test data, wherein N is a positive integer;

step S203: running an automatic driving algorithm under N simulation scenes for testing to obtain output data of the automatic driving algorithm under the N simulation scenes;

step S204: calculating results of first test indexes under N simulation scenes through output data of an automatic driving algorithm under N simulation scenes;

step S205: and determining a target test result in the target semantic scene according to the result of the first test index in the N simulation scenes.

Steps S201 to S202 correspond to steps S101 to S102 one to one, and step S205 corresponds to step S104, which is not described herein again. The automatic driving algorithm is tested under each simulation scene, corresponding output data exist, therefore, in the test running process, the output data of the automatic driving algorithm under the N simulation scenes can be continuously recorded, the results of the first test indexes under the N simulation scenes can be calculated according to the output data of the automatic driving algorithm under the N simulation scenes, the obtained results of the first test indexes under the N simulation scenes are used for determining the target test result, the automatic driving test under the target semantic scene is completed, and the automatic driving effect is improved.

It should be noted that the automatic driving algorithm includes a plurality of sub-algorithms, and each sub-algorithm (it is understood that each sub-algorithm is an algorithm module) has corresponding output data during the test process, that is, the output data of the automatic driving algorithm includes the output data of the plurality of sub-algorithms. For example, the output data of the automatic driving algorithm may include a sensed output, a predicted output, a planned output, a control output, an output of a dynamic model, and the like, and may be recorded in a record form. In the process of calculating the result of the first test index, the recorded record form data can be analyzed, and the result of the first test index under N simulation scenes can be calculated according to the output data of the automatic driving algorithm. The sensing output can comprise obstacle information, signal light information and the like, and the prediction output can be understood as the position, the speed and the like of a predicted obstacle within a preset time length in the future after the simulated vehicle encounters the obstacle in the test process, wherein the future is relative to the current time. The planning output may be a sequence, which may be understood as content of a plan for simulating vehicle travel, which may include information about the planned location, the planned speed, and the length of time to reach the planned location. The control output may include control information for the throttle, control information for the steering wheel, and the like. The output of the dynamic model may include information on the position and motion state of the simulated vehicle.

In one embodiment, determining a target test result in a target semantic scenario according to a result of the first test index in N simulation scenarios includes:

and summarizing results of the first test indexes in the N simulation scenes, and determining results of second test indexes in a target semantic scene, wherein the second test indexes correspond to the first test indexes and are the same in number, and the target test results comprise the results of the second test indexes.

For example, if the number of the first test indexes is P, and P is a positive integer, the number of the second test indexes is P, and each simulation scenario corresponds to P test indexes. The P first test indexes correspond to the P second test indexes one by one, and the result of any one second test index is obtained by calculation according to the result of the first test index corresponding to the second test index under N simulation scenes, namely the result of any one second test index is determined according to the N results of the first test index corresponding to the second test index. That is, in this embodiment, the result of the second test index in the target semantic scene is determined by summarizing and counting the results of the first test indexes in the N simulation scenes, so that the automatic driving test is implemented, and thus, the effect of the automatic driving test can be improved.

As an example, the result of the second test index is positively correlated with the result of the corresponding first test index, or the result of the second test index is positively correlated with M, where M is the number of target simulation scenarios, and the target simulation scenarios are scenarios in which the result of the first test index corresponding to the second test index exceeds a preset value in the N simulation scenarios.

For example, the P first test indexes include a sudden braking number, a collision number, a running vehicle speed, a stagnation number, and a redundant lane change number, and correspondingly, the P second test indexes include a sudden braking ratio, a collision ratio, an average running vehicle speed, a stagnation ratio, and a redundant lane change ratio. The test index of the average running vehicle speed can be an average value of the running vehicle speeds under the N simulation scenes, and can be obtained by dividing the sum of the running vehicle speeds under the N simulation scenes by the total number N of the simulation scenes, namely, the average running vehicle speed is positively correlated with the running vehicle speeds under the N simulation scenes. The test index of the sudden braking ratio may be a ratio between the number of simulation scenes in which sudden braking occurs and the total number N of simulation scenes, that is, the sudden braking ratio is positively correlated with the number of simulation scenes in which sudden braking occurs, and the number of simulation scenes in which sudden braking occurs may be understood as the number of scenes in which the number of times of sudden braking exceeds a preset value (for example, may be 0, and if it exceeds 0, it indicates that sudden braking occurs) in the N simulation scenes. For example, the target semantic scene is an intersection straight-going scene, N is 100000, 100000 simulation scenes of the intersection straight-going scene are generated, wherein if sudden braking occurs in 1000 simulation scenes, the sudden braking ratio of the intersection straight-going scene is 0.01, that is, the result of dividing 1000 by 100000.

The test index of the collision ratio may be a ratio between the number of simulation scenes in which a collision occurs and the total number N of simulation scenes, that is, the collision ratio is positively correlated with the number of simulation scenes in which a collision occurs, and the number of simulation scenes in which a collision occurs may be understood as the number of scenes in which the number of collisions exceeds a preset value (for example, may be 0, and if it exceeds 0, it indicates that a collision occurs) in the N simulation scenes. The test index of the stagnation ratio may be a ratio between the number of simulation scenes in which stagnation occurs and the total number N of simulation scenes, that is, the stagnation ratio is positively correlated with the number of simulation scenes in which stagnation occurs, and the number of simulation scenes in which stagnation occurs may be understood as the number of scenes in which the stagnation number exceeds a preset value (for example, may be 0, and exceeds 0, indicates that stagnation occurs) in the N simulation scenes. The test index of the redundant lane change ratio may be a ratio between the number of simulation scenes in which the redundant lane change occurs and the total number N of simulation scenes, that is, the redundant lane change ratio is positively correlated with the number of simulation scenes in which the redundant lane change occurs, and the number of simulation scenes in which the redundant lane change occurs may be understood as the number of scenes in which the number of redundant lane change times exceeds a preset value (for example, may be 0, and if it exceeds 0, it indicates that the redundant lane change occurs) in the N simulation scenes.

In one embodiment, after acquiring the real road test data of the autonomous vehicle, the method further comprises:

performing semantic segmentation on the real road test data to obtain N data segments of a target semantic scene;

wherein any two data segments do not overlap in time.

That is, the real road test data can be segmented according to different semantic scenes, data segments under multiple semantic scenes can be obtained, data segments under a target semantic scene, namely N data segments of the target semantic scene, can be obtained from the real road test data, any two data segments are not overlapped in time, interference among the data segments can be reduced, the N data segments of the target semantic scene are used for generating a simulation scene, running test of the automatic driving algorithm under the target semantic scene is achieved, and therefore the automatic driving test effect can be improved.

In one embodiment, the target data segment includes segment environmental information perceived by the autonomous vehicle and at least one of:

segment start time and segment end time;

segment starting positions of the autonomous vehicle and motion state information at the segment starting positions;

high-precision map information of the segments;

the target data segment is any one of the N data segments.

The real road test data is test data with respect to time, and since the data segment is a segment in the real road test data, segment environment information, i.e., a segment in the detected environment information, may be included, and each data segment may have its corresponding start time and end time, i.e., a segment start time and a segment end time. The data segment is a segment in the real road test data, and the target data segment may include the start position in the segment and the motion state information of the start position in the segment, that is, the segment start position and the motion state information of the start position in the segment. The high-precision map information can be understood as a whole high-precision map, the data segments are segments in real road test data, the corresponding surrounding map information is different due to different positions of the automatic driving vehicles, and correspondingly, the segment high-precision map information is a segment in the high-precision map and can correspond to the segment starting position of the automatic driving vehicle.

That is, in the present embodiment, the data segment for generating the simulation scene may include segment environment information perceived by the autonomous vehicle, and may further include at least one of segment start time and segment end time, segment start position of the autonomous vehicle, motion state information at the segment start position, and segment high-precision map information.

The process of the automatic driving test method is described in detail below with an embodiment.

As shown in fig. 3, the automatic driving test method of the present embodiment mainly includes three processes, namely, generation of simulation scene set, batch operation of scenes, result summarization, and target test result output.

For the generation of the simulation scene set, the simulation scene provided by the embodiment of the disclosure is derived from real road test data, data generated by the automatic driving vehicle in the real road test process can be stored in a data center, and N data segments of a target semantic scene (namely a specific semantic scene) can be found out from the real road test data by means of scene mining, for example, semantic scenes such as crossing straight-ahead vehicles and non-crossing straight-ahead switched vehicles. Then, the N data fragments in the target semantic scene are respectively converted into simulation scenes, namely N simulation scenes are obtained, so that a simulation scene set in a specific semantic scene (namely in the target semantic scene) is obtained, wherein the simulation scene set comprises N simulation scenes.

For scene batch operation, after a simulation scene set is ready, batch operation calculation of scenes can be performed in a large-scale cluster, an automatic driving algorithm can be operated in parallel for testing under N simulation scenes, each simulation scene operates in a single docker container, and the whole operation process is divided into two main steps: a scenario run process (which may be understood as running an automated driving algorithm in a simulation scenario) and a metric detection process (which may be understood as a process of determining the result of the first test indicator).

In the scene operation process, the automatic driving algorithm operation environment and scene data are deployed firstly, then the automatic driving algorithm is started to operate, output data of each sub-algorithm module of the automatic driving algorithm in different simulation scenes are continuously recorded in the operation process, and the output data can be recorded in a record form.

In the measurement detection process, the recorded output data of record driving is analyzed, and indexes in aspects of body feeling, safety, driving efficiency and intelligence under each simulation scene are calculated by using the output data of the automatic driving algorithm.

For result summarization and target test result output, after the result of the first test index in each of the N simulation scenes is obtained in the operation link, the results of the first test index in the N simulation scenes can be summarized to obtain the expression of the second test index in the target semantic scene. If a simulation scene set comprising 100000 simulation scenes of one intersection straight-going semantic scene is operated, wherein sudden braking occurs in 1000 simulation scenes, the sudden braking ratio of the intersection straight-going scene is 0.01, and the result of a second test index of the sudden braking ratio can be obtained.

The automatic driving test method provided by the disclosure can obtain a result closer to a real road test, and the selection of the scene number with the same magnitude for evaluation in the automatic driving test method is a key factor influencing the evaluation result. In the real road test data, two identical scenes do not exist under the same semantic meaning, for example, an automatic driving vehicle turns left at the same intersection twice, and the two times have difference points, for example, the types, speeds and the like of other nearby traffic participants have differences, which is determined by the complexity of the traffic environment. However, from a probability perspective, it can be assumed that when the number of tests of the same semantic scenario is large enough, the performance index of the autonomous vehicle tends to converge, as shown in fig. 4, where the abscissa represents the total number of simulation scenarios used for the test, i.e., N, and the ordinate is the sudden braking ratio, and when the number of simulation scenarios is gradually increased, the sudden braking ratio tends to converge. Therefore, in order to obtain a reliable test result, it is necessary to select a simulation scenario of sufficient magnitude, and how many scenarios to select for testing can be determined according to the convergence condition.

The automatic driving test method disclosed by the invention can objectively and comprehensively evaluate the performance of the automatic driving algorithm in a certain specific semantic scene from a macroscopic view. Because the simulation scene is obtained by mining the road test data through scene semantics, and scene difficulty is not distinguished, the distribution of the whole simulation scene set is basically consistent with the distribution of the road real scene, and the situation that new problems are introduced after the road test bad case is repaired and cannot be found is avoided. The method is more real in the aspect of single scene, compared with a manually designed scene, the method retains the detail information of the real road data to the greatest extent, and the problem of the automatic driving algorithm discovered through testing is better in quality.

As shown in fig. 5, the present disclosure also provides an automatic driving test device 500 according to an embodiment of the present disclosure, the device including:

a test data obtaining module 501, configured to obtain real road test data of an autonomous vehicle;

a generating module 502, configured to generate N simulation scenes based on N data segments in a target semantic scene in real road test data, where N is a positive integer;

the test module 503 is configured to run an automatic driving algorithm for testing in N simulation scenarios to obtain results of the first test index in the N simulation scenarios;

the determining module 504 is configured to determine a target test result in the target semantic scene according to a result of the first test indicator in the N simulation scenes.

In one embodiment, the test module 503 includes:

the output data acquisition module is used for running the automatic driving algorithm under the N simulation scenes for testing to obtain the output data of the automatic driving algorithm under the N simulation scenes;

and the result determining module is used for calculating the result of the first test index under the N simulation scenes through the output data of the automatic driving algorithm under the N simulation scenes.

In one embodiment, determining a target test result in a target semantic scenario according to a result of the first test index in N simulation scenarios includes:

and summarizing results of the first test indexes in the N simulation scenes, and determining results of second test indexes in a target semantic scene, wherein the second test indexes correspond to the first test indexes and are the same in number, and the target test results comprise the results of the second test indexes.

In one embodiment, the apparatus 500 further comprises:

the segmentation module is used for performing semantic segmentation on the real road test data after the test data acquisition module acquires the real road test data of the automatic driving vehicle to obtain N data segments of a target semantic scene;

wherein any two data segments do not overlap in time.

In one embodiment, the target data segment includes segment environmental information perceived by the autonomous vehicle and at least one of:

segment start time and segment end time;

segment starting positions of the autonomous vehicle and motion state information at the segment starting positions;

high-precision map information of the segments;

the target data segment is any one of the N data segments.

The automatic driving test device of each embodiment is a device for implementing the automatic driving test method of each embodiment applied to the first vehicle, and has corresponding technical features and technical effects, which are not described herein again.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

The non-transitory computer readable storage medium of the embodiments of the present disclosure stores computer instructions for causing a computer to perform the automated driving test method provided by the present disclosure.

The computer program product of the embodiments of the present disclosure includes a computer program for causing a computer to execute the automatic driving test method provided by the embodiments of the present disclosure.

FIG. 6 illustrates a schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 6, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)602 or a computer program loaded from a storage unit 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the device 600 can also be stored. The calculation unit 601, the ROM 602, and the RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

Various components in the electronic device 600 are connected to the I/O interface 605, including: an input unit 606 such as a keyboard, a mouse, or the like; an output unit 607 such as various types of displays, speakers, and the like; a storage unit 608, such as a magnetic disk, optical disk, or the like; and a communication unit 609 such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows the electronic device 600 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

The computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated artificial intelligence (I) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 601 executes the respective methods and processes described above, such as the automatic driving test method. For example, in some embodiments, the autopilot testing method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 600 via the ROM 602 and/or the communication unit 609. When the computer program is loaded into RAM603 and executed by the computing unit 601, one or more steps of the autopilot testing method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the autopilot test method in any other suitable manner (e.g., by means of firmware). Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), the internet, and blockchain networks.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server can be a cloud Server, also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server incorporating a blockchain.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel or sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.