1. An image processing method, comprising:

acquiring a road image acquired in real time, caching the road image, and identifying a target object in the road image;

tracking and detecting the target object to obtain a plurality of corresponding target state images within a preset time period;

coding and compressing each target state image to obtain a coded image;

analyzing each target state image respectively to judge whether the target object has a preset behavior or not;

and if so, decoding the encoded image so as to generate an image set corresponding to the target object by using the decoded image.

2. The image processing method according to claim 1, wherein after determining whether the target object has the preset behavior, the method further comprises:

and if the target object does not have the preset behavior, deleting the coded image corresponding to the target object.

3. The image processing method according to claim 1, wherein the acquiring the road image acquired in real time comprises:

acquiring real-time acquired video data aiming at a road;

the road image is extracted from the video data.

4. The image processing method according to claim 1, wherein the tracking detection of the target object to obtain a plurality of corresponding target state images within a preset time period comprises:

acquiring video data acquired within the preset time period;

and analyzing the video data to obtain a plurality of target state images for representing the lane and the motion state of the target object.

5. The image processing method according to any one of claims 1 to 4, wherein the performing encoding compression processing on each target state image to obtain an encoded image includes:

and carrying out coding compression processing on the I frame of each target state image based on ROI coding to obtain a coded image.

6. The image processing method according to any one of claims 1 to 4, wherein the performing encoding compression processing on each target state image to obtain an encoded image includes:

analyzing each target state image to determine a region of interest therein;

acquiring a first coding quality parameter aiming at the interested region and a second coding quality parameter aiming at a non-interested region;

and respectively carrying out compression coding processing on the interested region and the non-interested region based on the first coding quality parameter and the second coding quality parameter to obtain a coded image.

7. The image processing method according to claim 6, wherein the performing compression coding on the region of interest and the region of non-interest respectively based on the first coding quality parameter and the second coding quality parameter to obtain a coded image comprises:

based on the first coding quality parameter, carrying out compression coding on the region of interest, and based on the second coding quality parameter, carrying out compression coding on the region of no interest, and generating a coded image in a JPEG format;

correspondingly, the decoding the encoded image includes:

and decoding the encoded image to obtain a decoded image in a YUV format.

8. An image processing apparatus characterized by comprising:

the object identification module is used for acquiring and caching the road image acquired in real time and identifying a target object in the road image;

the tracking detection module is used for tracking and detecting the target object to obtain a plurality of corresponding target state images within a preset time period;

the image coding module is used for coding and compressing each target state image to obtain a coded image;

the image analysis module is used for respectively analyzing each target state image so as to judge whether the target object has a preset behavior or not;

and if so, decoding the encoded image so as to generate an image set corresponding to the target object by using the decoded image.

9. An electronic device, comprising:

a memory for storing a computer program;

a processor for implementing the steps of the image processing method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the image processing method according to any one of claims 1 to 7.

Background

With the development of traffic networks, the urban road traffic condition is increasingly complex, and the traffic jam problem causes the urban operation to be slow and consumes more resources. In such a severe traffic environment, some drivers may lower the basic rules for road traffic laws, generate illegal violations, and drive at will according to personal thoughts. In this case, the probability of occurrence of a traffic accident is greatly increased. By constructing an accurate traffic violation snapshot system, the violation of the traffic rules of the driver can be restrained to a certain extent, and traffic accidents are reduced.

At present, a traffic violation detection system mainly relies on a video acquisition unit on a road section to acquire image information and carry out detection and analysis on videos. Usually, the detected target image needs to be stored first, and once the comparison shows that the whole violation behavior process of the vehicle meets the violation penalty, evidence chain reporting penalty can be carried out on the violation vehicle process. However, under the scene of complex traffic conditions and large scene, the single acquisition unit is limited by the size of the physical cache, and only a limited number of passing targets can be subjected to behavior image storage, that is, all targets cannot be subjected to violation detection, so that violation identification efficiency is low at the same time. The traditional solution is to increase the DDR size to allocate more physical memory, however, this approach results in an exponential increase in hardware cost.

Therefore, how to solve the above problems is a great concern for those skilled in the art.

Disclosure of Invention

The present application aims to provide an image processing method, an image processing apparatus, an electronic device, and a computer-readable storage medium, which can effectively save the occupation of storage space.

To achieve the above object, the present application provides an image processing method including:

acquiring a road image acquired in real time, caching the road image, and identifying a target object in the road image;

tracking and detecting the target object to obtain a plurality of corresponding target state images within a preset time period;

coding and compressing each target state image to obtain a coded image;

analyzing each target state image respectively to judge whether the target object has a preset behavior or not;

and if so, decoding the encoded image so as to generate an image set corresponding to the target object by using the decoded image.

Optionally, after determining whether the target object has the preset behavior, the method further includes:

and if the target object does not have the preset behavior, deleting the coded image corresponding to the target object.

Optionally, the acquiring the road image collected in real time includes:

acquiring real-time acquired video data aiming at a road;

the road image is extracted from the video data.

Optionally, the tracking and detecting the target object to obtain a plurality of corresponding target state images within a preset time period includes:

acquiring video data acquired within the preset time period;

and analyzing the video data to obtain a plurality of target state images for representing the lane and the motion state of the target object.

Optionally, the encoding and compressing each target state image to obtain an encoded image includes:

and carrying out coding compression processing on the I frame of each target state image based on ROI coding to obtain a coded image.

Optionally, the encoding and compressing each target state image to obtain an encoded image includes:

analyzing each target state image to determine a region of interest therein;

acquiring a first coding quality parameter aiming at the interested region and a second coding quality parameter aiming at a non-interested region;

and respectively carrying out compression coding processing on the interested region and the non-interested region based on the first coding quality parameter and the second coding quality parameter to obtain a coded image.

Optionally, the performing, on the basis of the first encoding quality parameter and the second encoding quality parameter, compression encoding processing on the region of interest and the non-region of interest respectively to obtain an encoded image includes:

based on the first coding quality parameter, carrying out compression coding on the region of interest, and based on the second coding quality parameter, carrying out compression coding on the region of no interest, and generating a coded image in a JPEG format;

correspondingly, the decoding the encoded image includes:

and decoding the encoded image to obtain a decoded image in a YUV format.

To achieve the above object, the present application provides an image processing apparatus comprising:

the object identification module is used for acquiring and caching the road image acquired in real time and identifying a target object in the road image;

the tracking detection module is used for tracking and detecting the target object to obtain a plurality of corresponding target state images within a preset time period;

the image coding module is used for coding and compressing each target state image to obtain a coded image;

the image analysis module is used for respectively analyzing each target state image so as to judge whether the target object has a preset behavior or not;

and if so, decoding the encoded image so as to generate an image set corresponding to the target object by using the decoded image.

To achieve the above object, the present application provides an electronic device including:

a memory for storing a computer program;

a processor for implementing the steps of any of the image processing methods disclosed above when executing the computer program.

To achieve the above object, the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of any of the image processing methods disclosed in the foregoing.

According to the scheme, the image processing method provided by the application comprises the following steps: acquiring a road image acquired in real time, caching the road image, and identifying a target object in the road image; tracking and detecting the target object to obtain a plurality of corresponding target state images within a preset time period; coding and compressing each target state image to obtain a coded image; analyzing each target state image respectively to judge whether the target object has a preset behavior or not; and if so, decoding the encoded image so as to generate an image set corresponding to the target object by using the decoded image. According to the method and the device, after the target state image corresponding to the target object is collected, the target state image is subjected to compression coding processing, and is decoded and uploaded when a preset behavior exists, so that the occupation of a storage space can be effectively saved, and the cost of storage resources is reduced.

The application also discloses an image processing device, an electronic device and a computer readable storage medium, which can also realize the technical effects.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a flowchart of an image processing method disclosed in an embodiment of the present application;

FIG. 2 is a flow chart of another image processing method disclosed in the embodiments of the present application;

fig. 3 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure;

fig. 4 is a block diagram of an electronic device disclosed in an embodiment of the present application;

fig. 5 is a block diagram of another electronic device disclosed in the embodiments of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In general, in a video processing unit, an image frame is subjected to image circulation in a YUV format, and model operation is performed on YUV raw data to perform processing such as recognition and capture of an object such as a vehicle. In the video processing unit, the determination time for the target object is relatively short, and after the target object recognition is completed, the image frame is actually stored only in the DDR to wait for being finally used or discarded. In a general implementation scenario, a complete penalty process of the violation lasts for 10 seconds, and calculated according to 5 common intelligent processing frames, the entire buffer occupies 50 YUV frame buffer data, and the size of one frame of image data is 4096 × 2160 for example, and the overall DDR occupancy is as high as 650M. Once the illegal targets and the violation penalty types are increased in the same time frame, the YUV cache number in the cache of the processing unit is increased, and the demand for DDR is increased linearly.

Therefore, the embodiment of the application discloses an image processing method, which can effectively save the occupation of storage space, thereby reducing the cost of storage resources.

Referring to fig. 1, a flowchart of an image processing method disclosed in an embodiment of the present application, as shown in fig. 1, includes:

s101: acquiring a road image acquired in real time, caching the road image, and identifying a target object in the road image;

in the embodiment of the application, monitoring equipment pre-installed on a road is used for shooting vehicles, pedestrians and the like running on the road and caching the images into a storage space so as to acquire video data acquired in real time for the road and extract road images from the video data. By recognizing the road image, the target objects in the road image, including target vehicles, pedestrians and the like, are determined. In particular, the process of identifying images to determine target objects may be implemented using a deep learning network.

S102: tracking and detecting the target object to obtain a plurality of corresponding target state images within a preset time period;

after the target object is identified, tracking detection is performed on the target object to acquire a plurality of target state images corresponding to the target object within a preset time period. Specifically, video data acquired within a preset time period may be acquired, and each frame of video data is analyzed, so as to obtain a plurality of target state images, where lanes and positions where target objects are located and corresponding motion states may be specifically displayed in the target state images.

S103: coding and compressing each target state image to obtain a coded image;

it will be appreciated that after obtaining a plurality of target state images of the target object, each image will be compressed to obtain an encoded image. As a preferred implementation manner, the embodiment of the present application may implement, by using hardware, encoding and compressing processing on the target state image, so as to avoid performance impact on a processor caused by an encoding process.

S104: analyzing each target state image respectively to judge whether the target object has a preset behavior or not;

in this step, whether the target object has the preset behavior or not is determined by analyzing data of the target state image of the target object. If the target object is a vehicle, the corresponding preset behaviors can include but are not limited to behaviors of parking against regulations, pressing a parking line, parking in a red light, parking in a green light, backing in a car against regulations, running in a red light, driving without a lane, turning around against regulations, pressing a line, crossing a line and the like. For example, whether the vehicle has a driving behavior such as pressing a line, crossing a line, driving without a lane, etc. can be determined by comparing the position between the vehicle position and a calibrated lane line.

S105: and if so, decoding the encoded image so as to generate an image set corresponding to the target object by using the decoded image.

It can be understood that if it is determined that the target object does not have the preset behavior through the analysis of the state image of the target object, the encoded image corresponding to the target object may be deleted from the storage space, and the memory space may be released in time. If the target object is determined to have the preset behavior, the encoded image can be decoded to obtain a decoded image, and a corresponding image set is generated by using the decoded image. The image set can be specifically displayed as evidence information when penalty is carried out on dangerous behaviors of the target object in the follow-up process.

According to the scheme, the image processing method provided by the application comprises the following steps: acquiring a road image acquired in real time, caching the road image, and identifying a target object in the road image; tracking and detecting the target object to obtain a plurality of corresponding target state images within a preset time period; coding and compressing each target state image to obtain a coded image; analyzing each target state image respectively to judge whether the target object has a preset behavior or not; and if so, decoding the encoded image so as to generate an image set corresponding to the target object by using the decoded image. According to the method and the device, after the target state image of the target object is collected, the target state image is subjected to compression coding processing and then decoded and uploaded when a preset behavior exists, so that the occupation of a storage space can be effectively saved, and the cost of storage resources is reduced.

The embodiment of the application discloses an image processing method, and compared with the previous embodiment, the embodiment further explains and optimizes the technical scheme. Specifically, the method comprises the following steps:

referring to fig. 2, a flowchart of another image processing method provided in the embodiment of the present application is shown in fig. 2, and includes:

s201: acquiring a road image acquired in real time, caching the road image, and identifying a target object in the road image;

s202: tracking and detecting the target object to obtain a plurality of corresponding target state images within a preset time period;

s203: analyzing each target state image to determine a region of interest therein;

s204: acquiring a first coding quality parameter aiming at the interested region and a second coding quality parameter aiming at a non-interested region;

s205: respectively carrying out compression coding processing on the interested region and the non-interested region based on the first coding quality parameter and the second coding quality parameter to obtain a coded image;

as a feasible implementation manner, according to the embodiment of the application, by using the H264 coding characteristic of live coding, by obtaining a coded H264 code stream frame packet, extracting ROI region information in the frame packet and an I frame code stream packet for parsing, and further adjusting the ROI parameter of the I frame in the cache according to the expected size of a coded picture. And (4) performing coding compression processing by adopting the matched I frame of each target state image, and directly discarding if the I frame is a P frame. The ROI (Region Of Interest) coding method can employ different compression ratios in different spatial regions Of a compressed image. A lower compression ratio is adopted in the ROI area to ensure higher pixel precision, and a higher compression ratio is adopted in the non-ROI area to keep the basic outline of the image, so that the resolution of the compressed image in the ROI area is higher than that of the background part.

In the embodiment of the application, after a plurality of target state images corresponding to a target object in a preset time period are acquired, an interested area in the target state images is determined first. And acquiring a first coding quality parameter corresponding to the region of interest and a second coding quality parameter corresponding to the region of non-interest, and further respectively coding the corresponding image regions based on the two parameters to obtain a coded image.

In a possible implementation manner, the embodiment of the present application may generate an encoded image in JPEG format after respectively performing compression encoding on the region of interest and the region of non-interest based on the two encoding quality parameters. By differentially encoding the region of interest and the region of non-interest, the size of the image after encoding is far smaller than that of a YUV format, the quality can be accepted, and the image corresponding to the target object can be stored in the DDR in a smaller cache size.

S206: analyzing each target state image respectively to judge whether the target object has a preset behavior or not;

s207: and if so, decoding the encoded image so as to generate an image set corresponding to the target object by using the decoded image.

If the preset behavior of the target object is detected, the processing unit decodes the image corresponding to the target object, converts the image into a YUV format again, uploads the YUV format to an illegal evidence chain corresponding to the target object, splices the image according to the time sequence of the image, and outputs a final illegal evidence chain.

It should be further explained that after the video image data is collected, the video image data needs to be extracted by the detection frame of the processing unit, the detection frame is reported, and finally the processing unit releases the detection frame buffer. In the whole processing process, the detection frame has a certain buffer time, namely a life cycle. The video buffer cannot be released during the life cycle, but in practice it takes less than 80ms to implement the vehicle identification process during a life cycle of up to several seconds, after which the image frames are simply waiting for final output. After the intelligent period is finished, the image cache can be compressed and stored. Namely the intelligent departmentPhysical life cycle T_algIn the method, the cache size is not changed, and T is processed after the intelligent processing is finished_all-T_algIn the period, the cache size is stored at the compression ratio of s, and the final DDR utilization rate can be improved compared with that before optimizationWherein, T_allFor violation handling lifecycle time, T_algFor intelligent processing lifecycle, s is the compression ratio after encoding. For example, if the image is compressed by 0.5 time, the violation processing life cycle is 10s, the intelligent processing life cycle is 80ms, the overall storage performance utilization rate is improved to 198%, in the actual application process, the data can be compressed to about 1/10 of the source data, that is, the DDR utilization rate is improved by more than 10 times, and the image definition can meet the requirement.

Most of the current mainstream coding and compression algorithms are lossy compression processing, so that the contradiction between compression benefit and image definition exists in the process of coding, compressing and restoring pictures. In the application, the frame buffer in the YUV format is encoded into a data format with high compression ratio and capable of being restored. And the quality control of the coding can be realized by configuring corresponding coding parameters. Firstly, a target of original frame data is captured, analyzed and dynamically tracked to obtain an interested area, after the area and encoding quality parameters are set, I-frame encoding is adopted, the whole of the finally encoded image is higher than a background area, the solid definition is higher, and the image compression ratio is better. By the differential coding method, the vehicle cache image is coded once, the obtained image region-of-interest is high in definition, the coded image occupies less storage, and the performance balance can be considered while the performance of the coder is optimized, the storage space is reduced. When the image is stored in YUV format, the whole size isTypically in YUV420 format. After the frame is coded into an I frame format, the compression rate can reach 1/2 of FrameSize or even smaller on the premise of ensuring the definition of the whole size, so that the occupation of a frame buffer pool is saved.

The image decoding process specifically decodes the image into YUV format. This process is the inverse of encoding, and there is some pixel loss and performance loss during decoding. The traditional RIO picture coding technology needs to use the latest picture coding technology JPEG2000, but at present, the technology is not popularized yet, only few hardware platforms support hardware coding and decoding, and if the technology is used, software needs to use the JPEG2000 coding and decoding. In the application, the ROI advantage of JPEG2000 can be replaced by I-frame coding, and meanwhile, a chip hardware module is adopted, and the influence of performance loss caused by decoding coding is avoided by the physical performance of hardware.

In the application, the average cache size is reduced to be below 50% through encoding processing of the cache image, namely the utilization rate of the cache DDR is improved by more than 98%, the problem that an intelligent algorithm detects that a vehicle target excessively depends on hardware DDR under a multi-target scene is optimized, and meanwhile the problems that in the range of the expected image size, the image quality definition before and after encoding is low, and the hardware performance loss seriously affects the whole system scheme are solved.

An image processing apparatus provided in an embodiment of the present application is described below, and an image processing apparatus described below and an image processing method described above may be referred to with each other.

Referring to fig. 3, a structure diagram of an image processing apparatus according to an embodiment of the present disclosure, as shown in fig. 3, includes:

the object identification module 301 is configured to acquire and cache a road image acquired in real time, and identify a target object therein;

a tracking detection module 302, configured to perform tracking detection on the target object to obtain a plurality of corresponding target state images within a preset time period;

an image encoding module 303, configured to perform encoding and compression processing on each target state image to obtain an encoded image;

an image analysis module 304, configured to analyze each target state image to determine whether a preset behavior exists in the target object;

a set generating module 305, configured to decode the encoded image if yes, so as to generate an image set corresponding to the target object by using the decoded image.

For the specific implementation process of the modules 301 to 305, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details are not repeated here.

On the basis of the foregoing embodiment, as a preferred implementation manner, the image encoding module 303 may specifically include:

the region determining unit is used for analyzing each target state image to determine a region of interest in the target state image;

a parameter obtaining unit, configured to obtain a first encoding quality parameter for the region of interest and a second encoding quality parameter for a region of no interest;

and the compression coding unit is used for respectively carrying out compression coding processing on the interested region and the non-interested region based on the first coding quality parameter and the second coding quality parameter to obtain a coded image.

The present application further provides an electronic device, referring to fig. 4, a structure diagram of an electronic device provided in an embodiment of the present application, as shown in fig. 4, includes:

a memory 100 for storing a computer program;

the processor 200, when executing the computer program, may implement the steps provided by any of the foregoing embodiments.

Specifically, the memory 100 includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer-readable instructions, and the internal memory provides an environment for the operating system and the computer-readable instructions in the non-volatile storage medium to run. The processor 200 may be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor or other data Processing chip in some embodiments, and provides computing and controlling capability for the electronic device, and when executing the computer program stored in the memory 100, the steps of the image Processing method provided in any of the foregoing embodiments may be implemented.

On the basis of the above embodiment, as a preferred implementation, referring to fig. 5, the electronic device further includes:

and an input interface 300 connected to the processor 200, for acquiring computer programs, parameters and instructions imported from the outside, and storing the computer programs, parameters and instructions into the memory 100 under the control of the processor 200. The input interface 300 may be connected to an input device for receiving parameters or instructions manually input by a user. The input device may be a touch layer covered on a display screen, or a button, a track ball or a touch pad arranged on a terminal shell, or a keyboard, a touch pad or a mouse, etc.

And a display unit 400 connected to the processor 200 for displaying data processed by the processor 200 and for displaying a visualized user interface. The display unit 400 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch panel, or the like.

And a network port 500 connected to the processor 200 for performing communication connection with each external terminal device. The communication technology adopted by the communication connection can be a wired communication technology or a wireless communication technology, such as a mobile high definition link (MHL) technology, a Universal Serial Bus (USB), a High Definition Multimedia Interface (HDMI), a wireless fidelity (WiFi), a bluetooth communication technology, a low power consumption bluetooth communication technology, an ieee802.11 s-based communication technology, and the like.

While FIG. 5 shows only an electronic device having the assembly 100 and 500, those skilled in the art will appreciate that the configuration shown in FIG. 5 does not constitute a limitation of the electronic device, and may include fewer or more components than shown, or some components may be combined, or a different arrangement of components.

The present application further provides a computer-readable storage medium, which may specifically be: random Access Memory (RAM). The storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the image processing method provided by any of the preceding embodiments.

According to the method and the device, after the target state image of the target object is acquired, the target state image is subjected to compression coding processing and then decoded and uploaded when a preset action exists, so that the occupation of a storage space can be effectively saved, and the cost of storage resources is reduced; in addition, coding compression is carried out based on the ROI coding mode, so that the loss rate of image data can be reduced, and the quality of decoded images is guaranteed.

The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.

It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.