1. An urban road abnormity perception processing method is applied to a perception terminal arranged on a public transport means, and the method comprises the following steps:

step S10, collecting urban road images through a camera arranged on the perception terminal;

step S11, carrying out anomaly analysis processing on the collected image, and triggering a reporting process when determining that the collected image is an urban road anomaly event;

step S12, generating urban road abnormity report information according to the urban road abnormity event, wherein the urban road abnormity report information carries image information, positioning information, abnormity type and time information;

step S13, for the abnormal reporting information of the urban road, confirming whether the repeated reporting information exists in the reported list; the reported list is synchronous with the reported list of the control center in timing;

and step S14, when the confirmation result is that no repeated reporting information exists, sending the abnormal reporting information of the urban road to a control center.

2. The method of claim 1, wherein the step S11 further comprises:

judging the type of the urban road abnormal event by a driver of the public transport means, sending a voice instruction to a perception terminal, and triggering a reporting process; or

Inputting the collected image into a pre-trained abnormity judgment neural network, outputting urban road abnormity event confirmation information, and triggering a reporting flow; the pre-trained anomaly judgment neural network is used for judging the acquired image and determining whether the acquired image belongs to an urban market road anomaly event and an anomaly event type, wherein the urban road anomaly event type comprises the following steps: road exception events, human exception events, and vehicle exception events.

3. The method of claim 2, wherein the step S13 further comprises:

according to the positioning information of the urban road abnormal event, selecting a previous urban road abnormal event for comparison in the reported list;

comparing the abnormal events of the urban road with the abnormal events of the selected prior urban road one by one, and comparing the positioning information and the image similarity to determine whether repeated abnormal events exist in the reported list;

and when the comparison result is at least one of the following, determining that repeated abnormal events exist in the reported list: the positions of the two are different within a preset distance threshold value, or the image scene similarity of the two is larger than a preset similarity threshold value.

4. The method as claimed in claim 3, wherein the step of comparing the image similarity of the current abnormal event of the urban road with the image similarity of the previous abnormal event of the urban road is one of the following methods:

calculating histograms of the current urban road abnormal event and the prior urban road abnormal event, calculating the correlation of the two histograms, and determining the correlation as the image scene similarity of the two events; or

Inputting the urban road abnormal event and the prior urban road abnormal event into a pre-trained similarity judgment neural network, calculating through the neural network, and outputting the image scene similarity of the two events.

5. The method of claim 4, wherein in the step S14, further comprising:

the method comprises the steps of prompting current urban road abnormity report information to a public transport vehicle driver through voice, and sending the urban road abnormity report information to a control center after receiving voice confirmation of the public transport vehicle driver.

6. An urban road perception processing method is applied to a control center, and the method comprises the following steps:

step S20, receiving the abnormal information of the urban road reported by the sensing terminal installed on the public transport means; the urban road abnormal information comprises image information, positioning information, abnormal types and time information, wherein the image information is an image shot by a camera on the sensing terminal;

step S21, analyzing the abnormal information of the urban road to obtain the image information, the positioning information, the abnormal type and the time information carried by the abnormal information of the urban road;

step S22, the abnormal information of the urban road is sent to the processing terminal corresponding to the abnormal type and written into the reported list;

and step S23, after receiving the processing result from the processing terminal, deleting the record corresponding to the abnormal information of the urban road in the reported list.

7. The method of claim 6, wherein the step S22 is preceded by further comprising:

in the reported list, carrying out duplicate checking processing on the abnormal information of the urban road; inquiring whether the stored reported list has the reported information aiming at the same urban road abnormal event or not according to the analyzed content of the reported information, and if so, discarding the urban road abnormal information;

or/and the step S23 further includes:

carrying out voice prompt on the urban road abnormal information, recognizing the voice of a manual confirmation result, and issuing the urban abnormal information to a processing terminal corresponding to the abnormal type when the recognized confirmation result is that the urban road abnormal information is real and effective; otherwise, the flow returns to step S20.

8. An urban road abnormity perception processing device is characterized by comprising:

the acquisition unit is used for acquiring urban road images through a camera arranged on the sensing terminal;

the reporting triggering unit is used for carrying out anomaly analysis processing on the acquired image and triggering a reporting flow when determining that the acquired image is an urban road abnormal event;

a report information generating unit, configured to generate urban road exception report information according to the urban road exception event, where the urban road exception report information carries image information, positioning information, an exception type, and time information;

the filtering unit is used for confirming whether repeated reporting information exists in the reported list for the abnormal reporting information of the urban road; the reported list is synchronous with the reported list of the control center in timing;

and the reporting unit is used for sending the abnormal reporting information of the urban road to a control center through the communication device when the confirmation result shows that no repeated reporting information exists.

9. The apparatus of claim 8, wherein the reporting trigger unit further comprises:

the driver triggering unit is used for judging the type of the urban road abnormal event by a driver of the public transport means, sending a voice instruction to the sensing terminal and triggering a reporting process; or

The neural network triggering unit is used for inputting the acquired images into a pre-trained abnormity judgment neural network, outputting the confirmation information of the urban road abnormity events and triggering a reporting process; the pre-trained anomaly judgment neural network is used for judging the acquired image and determining whether the acquired image belongs to an urban market road anomaly event and an anomaly event type, wherein the urban road anomaly event type comprises the following steps: road exception events, human exception events, and vehicle exception events.

10. The apparatus of claim 8 or 9, wherein the filter unit further comprises:

the comparison object confirming unit is used for selecting a prior urban road abnormal event for comparison in the reported list according to the positioning information of the urban road abnormal event;

the filtering processing unit is used for comparing the abnormal events of the urban road with the abnormal events of the selected prior urban road one by one, and performing positioning information comparison and image similarity comparison to determine whether repeated abnormal events exist in the reported list;

and when the comparison result is at least one of the following, determining that repeated abnormal events exist in the reported list: the position difference between the two is within a preset distance threshold value, or the image scene similarity of the two is greater than a preset similarity threshold value;

wherein, the filtering processing unit at least comprises an image similarity comparison unit, which comprises:

the histogram comparison unit is used for calculating histograms of the current urban road abnormal event and the prior urban road abnormal event, calculating the correlation of the two histograms, and determining the correlation as the image scene similarity of the two events; or

And the neural network comparison unit is used for inputting the urban road abnormal event and the prior urban road abnormal event into a pre-trained similarity judgment neural network, calculating through the neural network and outputting the image scene similarity of the two events.

11. An urban road perception processing device, comprising:

the reporting information receiving unit is used for receiving the abnormal information of the urban road reported by the sensing terminal arranged on the public transport means; the urban road abnormal information comprises image information, positioning information, abnormal types and time information, wherein the image information is an image shot by a camera on the sensing terminal;

the report information analysis unit is used for analyzing the urban road abnormal information to obtain image information, positioning information, abnormal types and time information carried by the urban road abnormal information;

the report information issuing processing unit is used for issuing the urban road abnormal information to the processing terminal corresponding to the abnormal type and writing the urban road abnormal information into a reported list;

and the processing result checking unit is used for deleting the record corresponding to the abnormal information of the urban road in the reported list after receiving the processing result from the processing terminal.

12. The apparatus of claim 11, further comprising one or more of the following elements:

the duplicate checking and processing unit is used for inquiring whether the stored reported list has the reported information aiming at the same urban road abnormal event or not according to the analyzed content of the reported information, and if so, discarding the urban road abnormal information;

and the voice confirmation unit is used for carrying out voice prompt on the abnormal information of the urban road, recognizing the voice of a manual confirmation result and issuing the abnormal information of the urban road to the processing terminal corresponding to the abnormal type when the recognized confirmation result is that the abnormal information of the urban road is real and effective.

13. An urban road perception processing system is characterized in that the system comprises perception terminals, a control center and a plurality of processing terminals, wherein the perception terminals, the control center and the processing terminals are arranged on a plurality of public transportation means, and the system comprises:

the perception terminal comprises a central processing unit, a plurality of cameras, a wireless communication module, a voice interaction device and a display, wherein the cameras, the wireless communication module, the voice interaction device and the display are connected with the central processing unit, and the central processing unit at least comprises the urban road abnormity perception processing device according to any one of claims 8-10;

the control center comprises a central processing module, and a voice prompt module, an interaction module, a network communication module and a display which are connected with the central processing module, wherein the central processing module at least comprises the urban road abnormity perception processing device according to any one of claims 11-12;

and the processing terminal is used for receiving the urban road abnormal information from the control center and sending the processing result of the abnormal event to the control center.

14. A computer-readable storage medium having stored thereon computer instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1-7.

Background

With the progress of society, various facilities in cities are more and more, and the damage of the facilities, such as street lamp failure or sewer cover damage, often occurs; in addition, various obstacles can also appear on the urban road, such as tires left after tire burst, various goods falling off from trucks and the like; and violation accidents occurring on roads, etc.

In the prior art, the management of urban roads is realized by a plurality of departments, such as street lamps, sewer well covers and the like, belonging to the administrative department of the roads; traffic accidents, driving, mobile phone-taking and other traffic violation behaviors belong to the management of traffic police departments; the information is obtained mainly by means of video monitoring, manual inspection, citizen telephone reporting and other modes of fixed places, but no method is available for full coverage and unified scheduling management of urban road abnormal conditions in the whole city, so that urban management and operation efficiency is low.

Disclosure of Invention

The technical problem to be solved by the present invention is to provide a method, an apparatus, a system and a storage medium for sensing and processing an urban road anomaly, which can improve the efficiency of reporting and processing the urban road anomaly event information, reduce the operation cost of the city and improve the management efficiency.

As an aspect of the present invention, there is provided an urban road anomaly awareness processing method applied to an awareness terminal provided on a public transportation, the method including:

step S10, collecting urban road images through a camera arranged on the perception terminal;

step S11, carrying out anomaly analysis processing on the collected image, and triggering a reporting process when determining that the collected image is an urban road anomaly event;

step S12, generating urban road abnormity report information according to the urban road abnormity event, wherein the urban road abnormity report information carries image information, positioning information, abnormity type and time information;

step S13, for the abnormal reporting information of the urban road, confirming whether the repeated reporting information exists in the reported list; the reported list is synchronous with the reported list of the control center in timing;

and step S14, when the confirmation result is that no repeated reporting information exists, sending the abnormal reporting information of the urban road to a control center.

Wherein the step S11 further includes:

judging the type of the urban road abnormal event by a driver of the public transport means, sending a voice instruction to a perception terminal, and triggering a reporting process; or

Inputting the collected image into a pre-trained abnormity judgment neural network, outputting urban road abnormity event confirmation information, and triggering a reporting flow; the pre-trained anomaly judgment neural network is used for judging the acquired images and determining whether the acquired images belong to road anomaly events, human anomaly events and vehicle anomaly events, and the urban road anomaly event confirmation information contains the types of the urban road anomaly events.

Wherein the step S13 further includes:

according to the positioning information of the urban road abnormal event, selecting a previous urban road abnormal event for comparison in the reported list;

comparing the abnormal events of the urban road with the abnormal events of the selected prior urban road one by one, and comparing the positioning information and the image similarity to determine whether repeated abnormal events exist in the reported list;

and when the comparison result is at least one of the following, determining that repeated abnormal events exist in the reported list: the positions of the two are different within a preset distance threshold value, or the image scene similarity of the two is larger than a preset similarity threshold value.

The step of comparing the image similarity of the current urban road abnormal event with the image similarity of the previous urban road abnormal event is specifically one of the following methods:

calculating histograms of the current urban road abnormal event and the prior urban road abnormal event, calculating the correlation of the two histograms, and determining the correlation as the image scene similarity of the two events; or

Inputting the urban road abnormal event and the prior urban road abnormal event into a pre-trained similarity judgment neural network, calculating through the neural network, and outputting the image scene similarity of the two events.

Wherein, in the step S14, the method further comprises:

the method comprises the steps of prompting current urban road abnormity report information to a public transport vehicle driver through voice, and sending the urban road abnormity report information to a control center after receiving voice confirmation of the public transport vehicle driver.

Correspondingly, the invention also provides an urban road perception processing method, which is applied to a control center and comprises the following steps:

step S20, receiving the abnormal information of the urban road reported by the sensing terminal installed on the public transport means; the urban road abnormal information comprises image information, positioning information, abnormal types and time information, wherein the image information is an image shot by a camera on the sensing terminal;

step S21, analyzing the abnormal information of the urban road to obtain the image information, the positioning information, the abnormal type and the time information carried by the abnormal information of the urban road;

step S22, the abnormal information of the urban road is sent to the processing terminal corresponding to the abnormal type and written into the reported list;

and step S23, after receiving the processing result from the processing terminal, deleting the record corresponding to the abnormal information of the urban road in the reported list.

Wherein the step S22 is preceded by:

in the reported list, carrying out duplicate checking processing on the abnormal information of the urban road; and inquiring whether the stored reported list has the reported information aiming at the abnormal event of the same urban road according to the analyzed content of the reported information, and if so, discarding the abnormal information of the urban road.

Wherein the step S23 further includes:

carrying out voice prompt on the urban road abnormal information, recognizing the voice of a manual confirmation result, and issuing the urban abnormal information to a processing terminal corresponding to the abnormal type when the recognized confirmation result is that the urban road abnormal information is real and effective; otherwise, the flow returns to step S20.

Correspondingly, the invention also provides an urban road abnormity perception processing device, which comprises:

the acquisition unit is used for acquiring urban road images through a camera arranged on the sensing terminal;

the reporting triggering unit is used for carrying out anomaly analysis processing on the acquired image and triggering a reporting flow when determining that the acquired image is an urban road abnormal event;

a report information generating unit, configured to generate urban road exception report information according to the urban road exception event, where the urban road exception report information carries image information, positioning information, an exception type, and time information;

the filtering unit is used for confirming whether repeated reporting information exists in the reported list for the abnormal reporting information of the urban road; the reported list is synchronous with the reported list of the control center in timing;

and the reporting unit is used for sending the abnormal reporting information of the urban road to a control center through the communication device when the confirmation result shows that no repeated reporting information exists.

Wherein the reporting trigger unit further includes:

the driver triggering unit is used for judging the type of the urban road abnormal event by a driver of the public transport means, sending a voice instruction to the sensing terminal and triggering a reporting process; or

The neural network triggering unit is used for inputting the acquired images into a pre-trained abnormity judgment neural network, outputting the confirmation information of the urban road abnormity events and triggering a reporting process; the pre-trained anomaly judgment neural network is used for judging the acquired images and determining whether the acquired images belong to road anomaly events, human anomaly events and vehicle anomaly events, and the urban road anomaly event confirmation information contains the types of the urban road anomaly events.

Wherein the filter unit further comprises: the comparison object confirming unit is used for selecting a prior urban road abnormal event for comparison in the reported list according to the positioning information of the urban road abnormal event;

the filtering processing unit is used for comparing the abnormal events of the urban road with the abnormal events of the selected prior urban road one by one, and performing positioning information comparison and image similarity comparison to determine whether repeated abnormal events exist in the reported list;

and when the comparison result is at least one of the following, determining that repeated abnormal events exist in the reported list: the positions of the two are different within a preset distance threshold value, or the image scene similarity of the two is larger than a preset similarity threshold value.

Wherein, the filtering processing unit at least comprises an image similarity comparison unit, which comprises:

the histogram comparison unit is used for calculating histograms of the current urban road abnormal event and the prior urban road abnormal event, calculating the correlation of the two histograms, and determining the correlation as the image scene similarity of the two events; or

And the neural network comparison unit is used for inputting the urban road abnormal event and the prior urban road abnormal event into a pre-trained similarity judgment neural network, calculating through the neural network and outputting the image scene similarity of the two events.

Correspondingly, the invention also provides an urban road perception processing device, which comprises:

the reporting information receiving unit is used for receiving the abnormal information of the urban road reported by the sensing terminal arranged on the public transport means; the urban road abnormal information comprises image information, positioning information, abnormal types and time information, wherein the image information is an image shot by a camera on the sensing terminal;

the report information analysis unit is used for analyzing the urban road abnormal information to obtain image information, positioning information, abnormal types and time information carried by the urban road abnormal information;

the report information issuing processing unit is used for issuing the urban road abnormal information to the processing terminal corresponding to the abnormal type and writing the urban road abnormal information into a reported list;

and the processing result checking unit is used for deleting the record corresponding to the abnormal information of the urban road in the reported list after receiving the processing result from the processing terminal.

Wherein, further include: and the duplicate checking and processing unit is used for inquiring whether the stored reported list has the reported information aiming at the same urban road abnormal event or not according to the analyzed content of the reported information, and if so, discarding the urban road abnormal information.

Wherein, further include: and the voice confirmation unit is used for carrying out voice prompt on the abnormal information of the urban road, recognizing the voice of a manual confirmation result and issuing the abnormal information of the urban road to the processing terminal corresponding to the abnormal type when the recognized confirmation result is that the abnormal information of the urban road is real and effective.

Correspondingly, the invention also provides an urban road perception processing system, which comprises perception terminals, a control center and a plurality of processing terminals, wherein the perception terminals, the control center and the processing terminals are arranged on a plurality of public transport means, and the perception processing system comprises:

the perception terminal comprises a central processing unit, a plurality of cameras, a wireless communication module, a voice interaction device and a display, wherein the cameras, the wireless communication module, the voice interaction device and the display are connected with the central processing unit;

the control center comprises a central processing module, and a voice prompt module, an interaction module, a network communication module and a display which are connected with the central processing module, wherein the central processing module at least comprises the urban road abnormity perception processing device;

and the processing terminal is used for receiving the urban road abnormal information from the control center and sending the processing result of the abnormal event to the control center.

The present invention also provides a computer-readable storage medium having stored thereon computer instructions which, when run on a computer, cause the computer to perform the aforementioned method.

The embodiment of the invention has the following beneficial effects:

in the method, the device and the system for processing the urban road abnormity perception provided by the invention, the perception terminal arranged on the public transport means can conveniently report the urban road abnormity events to the control center in time; and after receiving the urban road abnormal information, the control center timely issues the urban road abnormal information to processing terminals of all functional departments so as to remind the functional departments of timely processing. Therefore, the efficiency of reporting and processing the urban road abnormity can be improved, the running cost of the city can be reduced, and various emergency events can be responded in time;

meanwhile, whether repeated reporting information exists or not is confirmed in the latest reported list before the abnormal information of the urban road is reported, so that the phenomenon that the same urban market road abnormal event is repeatedly reported for many times can be avoided, the repetition rate is reduced, and the workload of a control center can be reduced;

in addition, the voice confirmation function of the driver of the public transport means is set, so that the false alarm rate can be further reduced, and the reporting accuracy is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is within the scope of the present invention for those skilled in the art to obtain other drawings based on the drawings without inventive exercise.

Fig. 1 is a schematic main flow chart of an embodiment of an urban road anomaly sensing processing method provided by the present invention;

FIG. 2 is a more detailed flowchart of step S13 in FIG. 1;

fig. 3 is a schematic main flow chart of an abnormal urban road perception processing method according to another embodiment of the present invention;

FIG. 4 is a schematic structural diagram of an embodiment of an urban road anomaly sensing processing device according to the present invention;

fig. 5 is a schematic structural diagram of the report triggering unit in fig. 4;

FIG. 6 is a schematic view of the filter unit of FIG. 4;

FIG. 7 is a schematic diagram of the image similarity comparing unit associated with FIG. 6;

fig. 8 is a schematic structural diagram of another embodiment of an urban road anomaly sensing processing device provided by the invention;

FIG. 9 is a schematic structural diagram of an embodiment of an urban road anomaly awareness processing system according to the present invention;

FIG. 10 is a schematic diagram of the structure of one embodiment of the sensing terminal of FIG. 9;

fig. 11 is a schematic structural diagram of an embodiment of the control center in fig. 9.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings.

In the embodiment of the invention, the sensing terminal is arranged on public transport means such as a bus, a taxi and the like, and the sensing terminal is arranged on the public transport means such as the bus, the taxi and the like, so that abnormal events (such as street lamp fault events, traffic accident events, violation events and the like) on urban roads can be detected in real time, corresponding information is sent to the control center, and the control center timely issues the abnormal information of the urban roads to relevant functional departments for processing. The management efficiency of the whole city can be improved, and the operation cost is reduced.

Specifically, as shown in fig. 1, a main flow diagram in an embodiment of an urban road anomaly perception processing method provided by the present invention is shown; in this embodiment, together with fig. 2, the urban road anomaly sensing processing method is applied to a sensing terminal disposed on a public transportation means, and the method includes:

step S10, collecting urban road images through a camera arranged on the perception terminal; in a specific example, the sensing terminal is arranged on public transportation means such as a taxi, a taxi and a network appointment car, for example, the sensing terminal is arranged on the roof of the car, and is provided with four cameras in four directions (front, back, left and right) for acquiring urban road images in real time;

step S11, carrying out anomaly analysis processing on the collected image, and triggering a reporting process when determining that the collected image is an urban road anomaly event;

specifically, in some examples, there are two triggering manners of this step S11:

firstly, a driver of a public transport means triggers a reporting process; judging the type of the urban road abnormal event by a driver of the public transport means, sending a voice instruction to a perception terminal, and triggering a reporting process; when a driver observes that an urban road abnormal event occurs, a voice instruction is sent to a perception terminal, for example, the voice instruction is 'reported abnormally', and when the perception terminal receives and analyzes the voice instruction, a triggering process is started;

secondly, triggering a reporting process by a sensing terminal; the sensing terminal inputs the acquired image into a pre-trained abnormity judgment neural network, outputs urban road abnormity event confirmation information and triggers a reporting process; the pre-trained anomaly judgment neural network is used for judging the acquired image and determining whether the acquired image belongs to an urban market road anomaly event and an anomaly event type, wherein the urban road anomaly event type comprises the following steps: road exception events, human exception events, and vehicle exception events.

It is understood that the abnormal events of the urban roads herein may include such events as damage to street lamps, obstacles existing on the roads, traffic accidents occurring on the roads, loss of manhole covers on the roads, etc.; the human abnormal events comprise events such as a driver driving a car to take a mobile phone, a driver driving a car to smoke, a driver driving a car to not send a safety belt and the like; the vehicle abnormal event comprises vehicles with forbidden behaviors, such as a restricted engineering vehicle, a forklift, a truck and the like. The method comprises the steps of training various types of urban road abnormal events by adopting a large number of training pictures in advance to form an abnormal judgment neural network, outputting a judgment result through the abnormal judgment neural network after inputting a corresponding picture, namely confirming whether the current picture belongs to the urban road abnormal event, and simultaneously outputting a specific abnormal type if the current picture belongs to the urban road abnormal event.

In a more specific example, the neural network may be divided into a behavior recognition network and an object detection network; the behavior recognition network can be used for recognizing and processing drivers who drive to take mobile phones, smoke and send safety belts, traffic accidents and the like; it can be understood that the deep learning network may adopt a common behavior recognition network, such as a space-time double-flow network, an R-C3D (Region 3-Dimensional restriction) network, and the like, in an embodiment of the present invention, the R-C3D network may be adopted to perform behavior recognition, and when the neural network recognizes behaviors of driving, hitting a mobile phone, driving, smoking, traffic accidents, and driving without sending a seat belt, the neural network performs reporting processing;

the target detection network can be used for identifying and processing the events such as the fault of a street lamp, the existence of obstacles on a road, the prohibition of vehicles, the loss of a manhole cover of a water channel on the road and the like; common target detection networks include yolo, ssd, yolov3 and the like, but in the embodiment of the invention, a yolov3 network can be adopted to detect a target, and when an abnormal target is detected, corresponding reporting processing is carried out; for example, in one example, the state of the headlight can be acquired first, and if the state of the headlight is off, the default is daytime, and no processing is performed; if the headlight is bright, the description is a darker state, since the headlight does not completely represent night and day; in the process of changing from day to night, multi-frame continuous monitoring is added, for example, if only one street lamp is not lighted in 5 frames and other street lamps are all lighted, the street lamp is considered to be damaged.

Step S12, generating urban road abnormity report information according to the urban road abnormity event, wherein the urban road abnormity report information carries image information, positioning information, abnormity type and time information;

step S13, for the abnormal reporting information of the urban road, confirming whether the repeated reporting information exists in the reported list; the reported list is synchronous with the reported list of the control center in timing; the timing synchronization process here may be that the control center periodically sends the latest reported list to each sensing terminal, or that each sensing terminal periodically or when needed requests the latest reported list from the control center. The reported list at least includes the reported and processed positioning information, abnormal type and time information corresponding to each urban road abnormal event, and in some examples, the reported list may also include the image information of each urban road abnormal event;

specifically, the step S13 further includes:

step S130, according to the positioning information of the urban road abnormal event, selecting a prior urban road abnormal event for comparison in the reported list; specifically, the reported information that is in the reported list and is in accordance with the vicinity of the current position and has the same abnormal type may be used as the compared previous road abnormal event;

step S131, comparing the abnormal events of the current urban road with the abnormal events of the selected prior urban road one by one, and comparing the positioning information and the image similarity to determine whether repeated abnormal events exist in the reported list;

and when the comparison result is at least one of the following, determining that repeated abnormal events exist in the reported list: the two positions are different within a predetermined distance threshold (e.g., 50 meters), or the image scene similarity of the two is greater than a predetermined similarity threshold (e.g., 70%). It can be understood that in different application occasions, the number of conditions that need to be satisfied specifically can be set, for example, in some occasions, if one of the conditions needs to be satisfied, the abnormal event is determined to be a repeated abnormal event; and in other occasions, if two of the abnormal events need to be met, the abnormal event is judged to be a repeated abnormal event. And the predetermined distance threshold and the similar threshold are preset or calibrated in advance.

In a specific example, in step S131, the step of comparing the image similarity between the current abnormal event of the urban road and the previous abnormal event of the urban road may specifically adopt two methods:

one is as follows: calculating histograms of the current urban road abnormal event and the prior urban road abnormal event, calculating the correlation of the two histograms, and determining the correlation as the image scene similarity of the two events; it is understood that in the prior art, commonly used indicators for comparing the correlation of histograms of two images include such as: correlation, chi-square, histogram intersection, bhattacharyya distance, land travel distance (EMD), etc.; in some examples of the present invention, an EMD method may be adopted, which is to solve the cost of converting one image (an image formed by splicing four images collected by a sensing terminal) into another image (an abnormal image of a previous urban road); the histogram can also be used for representing that the cost of converting one histogram into another histogram is obtained; if the cost is smaller, it means that the two graphs are more similar.

The second step is as follows: inputting the urban road abnormal event and the prior urban road abnormal event into a pre-trained similarity judgment neural network, calculating through the neural network, and outputting the image scene similarity of the two events.

And step S14, when the confirmation result is that no repeated reporting information exists, sending the abnormal reporting information of the urban road to a control center.

In some other examples, the step S14 further includes:

the method comprises the steps of prompting current urban road abnormity report information to a public transport vehicle driver through voice, and sending the urban road abnormity report information to a control center after receiving voice confirmation of the public transport vehicle driver.

It can be understood that, in the embodiment, the sensing terminal arranged on the public transport means can conveniently and timely report the abnormal events of the urban road to the control center; meanwhile, whether repeated reporting information exists or not needs to be confirmed in the latest reported list before reporting, so that the same urban market road abnormal event can be prevented from being repeatedly reported for many times, the repetition rate is reduced, and the workload of a control center is reduced; and thirdly, the false alarm rate can be further reduced and the reporting accuracy can be improved by setting the voice confirmation function of the driver.

Fig. 2 is a schematic main flow chart of another embodiment of the urban road perception processing method according to the present invention. In this embodiment, the urban road perception processing method is applied to a control center, and the method includes:

step S20, receiving the abnormal information of the urban road reported by the sensing terminal installed on the public transport means; the urban road abnormal information comprises image information, positioning information, abnormal types and time information, wherein the image information is an image shot by a camera on the sensing terminal;

step S21, analyzing the abnormal information of the urban road to obtain the image information, the positioning information, the abnormal type and the time information carried by the abnormal information of the urban road;

step S210, in the reported list, the abnormal information of the urban road is subjected to duplicate checking treatment; inquiring whether the stored reported list has the reported information aiming at the same urban road abnormal event or not according to the analyzed content of the reported information, and if so, discarding the urban road abnormal information;

step S22, the abnormal information of the urban road is sent to the processing terminal corresponding to the abnormal type and written into the reported list; wherein, different exception types can correspond to different processing terminals which are arranged in each functional department; the processing terminal can be a smart phone carried by related personnel in the functional department; related personnel in the functional department can go to corresponding positioning positions for processing according to the urban road abnormal information received by the processing terminal, and send corresponding processing results to the control center through the processing terminal after the processing is finished;

step S23, after receiving the processing result from the processing terminal, deleting the record corresponding to the abnormal information of the urban road from the reported list, so as to ensure that the abnormal information in the reported list is the latest information in processing (unprocessed).

It is understood that step S210 is an optional step.

Meanwhile, in some embodiments, the step S23 further includes:

carrying out voice prompt on the urban road abnormal information, recognizing the voice of a manual confirmation result, and issuing the urban abnormal information to a processing terminal corresponding to the abnormal type when the recognized confirmation result is that the urban road abnormal information is real and effective; otherwise, the flow returns to step S20.

The following description is provided in conjunction with an example of traffic accident handling to better understand the methods described above in connection with fig. 1-3.

The sensing terminal installed on the public transport shoots and processes the image, when the public transport means passes through the accident site, the sensing terminal processes the input image, judges the accident site and type of the accident traffic by using a neural network, and stores the image and the positioning information at the moment; judging whether the current traffic accident is reported, if so, not processing, if not, judging whether a perception terminal starts a voice prompt confirmation function, if so, prompting a driver by voice, and if so, judging whether the traffic accident occurs; after the driver confirms, reporting abnormal event information, and if the voice prompt function is not started, directly reporting the abnormal event information to a control center;

after receiving the abnormal information reported by the sensing terminal, the control center prompts the manual confirmation by voice if the manual confirmation is started, judges the type of the event if the manual confirmation is not started, and sends the event type to a processing terminal of a traffic management department;

when the related personnel of the traffic management department receive the abnormal event information, the related accidents are confirmed and processed on site, and after the processing is finished, the processing finished information is sent to the control center;

the control center eliminates the related abnormal events, and the whole processing process is completed.

As shown in fig. 4, a schematic structural diagram of an urban road anomaly sensing processing device provided by the invention is shown; as shown in fig. 5 to 7, in this embodiment, the urban road anomaly sensing processing device 1 is disposed in a sensing terminal, and includes:

the acquisition unit 11 is used for acquiring urban road images through a camera arranged on the sensing terminal;

a reporting triggering unit 12, configured to perform anomaly analysis processing on the acquired image, and trigger a reporting process when an urban road anomaly event is determined;

a report information generating unit 13, configured to generate urban road exception report information according to the urban road exception event, where the urban road exception report information carries image information, positioning information, an exception type, and time information;

a filtering unit 14, configured to determine whether there is repeated reporting information in the reported list for the abnormal reporting information of the urban road; the reported list is synchronous with the reported list of the control center in timing;

and the reporting unit 15 is configured to send the abnormal reporting information of the urban road to a control center through the communication device when it is determined that there is no repeated reporting information.

In a specific example, the reporting triggering unit 12 further includes:

the driver triggering unit 120 is used for judging the type of the urban road abnormal event by a driver of the public transportation means, sending a voice instruction to the sensing terminal and triggering a reporting process; or

The neural network triggering unit 121 is configured to input the acquired image into a pre-trained anomaly judgment neural network, output the urban road abnormal event confirmation information, and trigger a reporting process; the pre-trained anomaly judgment neural network is used for judging the acquired images and determining whether the acquired images belong to road anomaly events, human anomaly events and vehicle anomaly events, and the urban road anomaly event confirmation information contains the types of the urban road anomaly events.

In a specific example, the filtering unit 14 further comprises:

a comparison object confirming unit 140, configured to select, according to the positioning information of the current urban road abnormal event, a previous urban road abnormal event for comparison in the reported list;

a filtering processing unit 141, configured to compare the current abnormal event of the urban road with the abnormal events of the selected previous urban road one by one, and perform positioning information comparison and image similarity comparison to determine whether there is a repeated abnormal event in the reported list;

and when the comparison result is at least one of the following, determining that repeated abnormal events exist in the reported list: the positions of the two are different within a preset distance threshold value, or the image scene similarity of the two is larger than a preset similarity threshold value.

In a specific example, the filtering unit 140 at least includes an image similarity comparing unit 1400, which includes:

a histogram comparing unit 1401, configured to calculate histograms of the current urban road abnormal event and the previous urban road abnormal event, calculate a correlation between the two histograms, and determine the correlation as an image scene similarity of the two events; or

And the neural network comparison unit 1402 is configured to input the current urban road abnormal event and the previous urban road abnormal event into a pre-trained similarity determination neural network, calculate through the neural network, and output image scene similarity of the two events.

For more details, reference may be made to the foregoing description of fig. 1-2, which is not detailed herein.

Fig. 8 is a schematic structural diagram of an urban road perception processing device according to the present invention. In this embodiment, the urban road perception processing device 2 is disposed in a control center, and includes:

a reported information receiving unit 21, configured to receive the abnormal information of the urban road reported by the sensing terminal installed on the public transportation vehicle; the urban road abnormal information comprises image information, positioning information, abnormal types and time information, wherein the image information is an image shot by a camera on the sensing terminal;

a reported information analysis unit 22, configured to analyze the urban road abnormal information to obtain image information, positioning information, an abnormal type, and time information carried by the urban road abnormal information;

a duplicate checking and processing unit 23, configured to query, according to the content analyzed from the current reporting information, whether the stored reported list includes reporting information for an abnormal event of the same urban road, and if so, discard the current abnormal information of the urban road;

a reported information issuing processing unit 24, configured to issue the urban road abnormal information to a processing terminal corresponding to the abnormal type, and write the urban road abnormal information into a reported list;

and the processing result checking unit 25 is configured to delete the record corresponding to the abnormal information of the urban road in the reported list after receiving the processing result from the processing terminal.

It is to be understood that the duplication checking processing unit 23 is an optional unit, and may not be provided in some embodiments.

In some specific examples, the method further comprises: and the voice confirmation unit 26 is configured to perform voice prompt on the abnormal urban road information, recognize a voice of a manual confirmation result, and issue the abnormal urban road information to the processing terminal corresponding to the abnormal type when the recognized confirmation result is that the abnormal urban road information is true and valid.

For more details, reference may be made to the foregoing description of fig. 3, which is not detailed herein.

As shown in fig. 9, which shows a schematic structural diagram of an urban road perception processing system provided by the present invention, and is shown by combining fig. 10 and fig. 11, in this embodiment, the system includes a perception terminal, a control center and a plurality of processing terminals disposed on a plurality of public transportation vehicles, wherein:

the sensing terminal comprises a central processing unit, a plurality of cameras, a wireless communication module, a voice interaction device and a display, wherein the cameras, the wireless communication module, the voice interaction device and the display are connected with the central processing unit, and the central processing unit at least comprises the urban road abnormity sensing processing device 1 described in the figures 4 to 7;

the control center comprises a central processing module, and a voice prompt module, an interaction module, a network communication module and a display which are connected with the central processing module, wherein the central processing module at least comprises the urban road abnormity perception processing device 2 described in the figure 8;

and the processing terminal is used for receiving the urban road abnormal information from the control center and sending the processing result of the abnormal event to the control center, and can be an intelligent terminal arranged in each functional department, such as a smart phone, a tablet computer and the like.

It is understood that the voice prompt module, the interaction module, the network communication module, the display, the camera, the wireless communication module, and the like in the embodiments of the present invention are well known and easily implemented by those skilled in the art, and therefore, detailed description is not provided. For more details, reference may be made to the foregoing description of fig. 4 to 8, which is not detailed here.

Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is caused to execute the urban road anomaly perception processing method described in fig. 1 to 3 in the above method embodiment of the present invention.

The embodiment of the invention has the following beneficial effects:

in the method, the device and the system for processing the urban road abnormity perception provided by the invention, the perception terminal arranged on the public transport means can conveniently report the urban road abnormity events to the control center in time; and after receiving the urban road abnormal information, the control center timely issues the urban road abnormal information to processing terminals of all functional departments so as to remind the functional departments of timely processing. Therefore, the efficiency of reporting and processing the urban road abnormity can be improved, the running cost of the city can be reduced, and various emergency events can be responded in time;

meanwhile, whether repeated reporting information exists or not is confirmed in the latest reported list before the abnormal information of the urban road is reported, so that the phenomenon that the same urban market road abnormal event is repeatedly reported for many times can be avoided, the repetition rate is reduced, and the workload of a control center can be reduced;

in addition, the voice confirmation function of the driver of the public transport means is set, so that the false alarm rate can be further reduced, and the reporting accuracy is improved.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not to be limited to the disclosed embodiment, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.