1. A method for performing inspection of a container using a robot, the method comprising:

when the robot reaches the checking position of the current container to be checked, acquiring a first image containing the current container to be checked, wherein the first image is shot by an image acquisition part on the robot;

determining the container number and the actual identification number of the current container to be detected based on the first image;

inquiring the reported identification number of the current container to be detected according to the container number;

and matching and verifying the actual identification number and the reported identification number based on the query result.

2. The method for container inspection using a robot as claimed in claim 1, wherein the step of determining the container number and the actual identification number of the container currently to be inspected based on the first image further comprises:

identifying the first image by using a container number identification model to obtain the container number;

detecting the first image by using a position detection model to obtain position information of an envelope in the first image;

based on the position information, controlling the direction of a holder on the robot to adjust, and controlling the image acquisition part to adjust the focal length and focus so that the seal is positioned at the center of a viewing frame of the image acquisition part;

acquiring a second image containing the seal, which is shot by the image acquisition part;

and identifying the second image by using an identification number identification model to obtain the actual identification number.

3. The method for checking a container using a robot as claimed in claim 2, wherein the step of verifying the matching between the actual identification number and the reported identification number based on the query result further comprises:

if the actual identification number is successfully matched with the reported identification number, controlling the robot to send out unpacking prompt information, and shooting an unpacking image after preset time;

and if the actual identification number is not matched with the reported identification number, controlling the robot to send out alarm prompt information and move to the checking position of the next container to be checked.

4. The method for container inspection using a robot as claimed in claim 3, wherein after the step of verifying the matching of the actual identification number and the reported identification number, the method further comprises:

if the actual identification number is successfully matched with the reported identification number, receiving and storing the first image, the second image and the unpacking image uploaded by the robot;

and if the actual identification number is not matched with the reported identification number, receiving and storing the first image and the second image uploaded by the robot.

5. The method for container inspection using a robot as claimed in claim 1, further comprising:

issuing an inspection instruction including a position number and an inspection order of a container to be inspected to the robot so that

And the robot establishes a navigation map based on the position numbers and the checking sequence, and performs container checking based on the navigation map.

6. A container verification system, comprising:

an acquisition module configured to acquire a first image containing a current container to be inspected, which is captured by an image acquisition part on the robot, after the robot reaches an inspection position of the current container to be inspected;

a determination module configured to determine a container number and an actual identification number of the current container to be inspected based on the first image;

a query module configured to query the reported identification number of the current container to be inspected according to the container number;

a verification module configured to perform matching verification on the actual identification number and the reported identification number based on a query result.

7. The container verification system of claim 6, wherein the determination module is further configured to determine the container number and the actual identification number of the current container to be inspected based on the first image by:

identifying the first image by using a container number identification model to obtain the container number;

detecting the first image by using a position detection model to obtain position information of an envelope in the first image;

based on the position information, controlling the direction of a holder on the robot to adjust, and controlling the image acquisition part to adjust the focal length and focus so that the seal is positioned at the center of a viewing frame of the image acquisition part;

acquiring a second image containing the seal, which is shot by the image acquisition part;

and identifying the second image by using an identification number identification model to obtain the actual identification number.

8. The container verification system of claim 6, wherein the verification module is further configured to verify the matching of the actual identification number and the reported identification number based on the query result by:

if the actual identification number is successfully matched with the reported identification number, controlling the robot to send out unpacking prompt information, and shooting an unpacking image after preset time;

and if the actual identification number is not matched with the reported identification number, controlling the robot to send out alarm prompt information and move to the checking position of the next container to be checked.

9. The container verification system of claim 8, further comprising a receiving storage module configured to:

if the actual identification number is successfully matched with the reported identification number, receiving and storing the first image, the second image and the unpacking image uploaded by the robot;

and if the actual identification number is not matched with the reported identification number, receiving and storing the first image and the second image uploaded by the robot.

10. The container verification system of claim 6, further comprising an issuing module configured to:

issuing an inspection instruction including a position number and an inspection order of a container to be inspected to the robot so that

And the robot establishes a navigation map based on the position numbers and the checking sequence, and performs container checking based on the navigation map.

11. A computer device, comprising:

a processor;

a memory adapted to store a plurality of program codes adapted to be loaded and run by the processor to perform the method of any of claims 1 to 5.

12. A computer readable storage medium having stored therein a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by a processor to perform the method of any of claims 1 to 5.

Background

With the rapid development of economic trade in China, the number of containers entering and leaving ports increases in geometric progression. In order to guarantee the authenticity of goods and prevent the occurrence of conditions such as smuggling, customs personnel are required to open the container to check each container. The current customs inspection process is as follows: checking the position information of the container to be checked in the same day by an inspector in the four-system checking system, manually inputting the container number and the identification number to an inspection site, checking the container number and the identification number with background data, manually opening the container after the checking is passed, opening the container by a camera device for shooting, and drawing out the container for checking according to the requirements of the four-system checking system.

It can be seen that the traditional customs inspection means is difficult to adapt to the requirement of container inspection which increases in geometric progression because the inspection process is completed mainly by manpower. Under the background, the pressure of the inspection personnel is high, the strength is high, manpower and material resources are wasted, and the overall information processing efficiency and level are influenced.

Accordingly, there is a need in the art for a new inspection method to solve the above problems.

Disclosure of Invention

In order to solve at least one of the above problems in the prior art, that is, to solve the problem of low efficiency of the conventional method for inspecting a container by using human power, the present application provides a method for inspecting a container by using a robot, the method including:

when the robot reaches the checking position of the current container to be checked, acquiring a first image containing the current container to be checked, wherein the first image is shot by an image acquisition part on the robot;

determining the container number and the actual identification number of the current container to be detected based on the first image;

inquiring the reported identification number of the current container to be detected according to the container number;

and matching and verifying the actual identification number and the reported identification number based on the query result.

In the above preferred embodiment of the method for checking a container by using a robot, the step of determining the container number and the actual identification number of the current container to be checked based on the first image further includes:

identifying the first image by using a container number identification model to obtain the container number;

detecting the first image by using a position detection model to obtain position information of an envelope in the first image;

based on the position information, controlling the direction of a holder on the robot to adjust, and controlling the image acquisition part to adjust the focal length and focus so that the seal is positioned at the center of a viewing frame of the image acquisition part;

acquiring a second image containing the seal, which is shot by the image acquisition part;

and identifying the second image by using an identification number identification model to obtain the actual identification number.

In the above preferred technical solution of the method for checking a container by using a robot, the step of performing matching verification on the actual identification number and the reported identification number based on the query result further includes:

if the actual identification number is successfully matched with the reported identification number, controlling the robot to send out unpacking prompt information, and shooting an unpacking image after preset time;

and if the actual identification number is not matched with the reported identification number, controlling the robot to send out alarm prompt information and move to the checking position of the next container to be checked.

In the above preferred technical solution of the method for checking a container by using a robot, after the step of "performing matching verification on the actual identification number and the reported identification number", the method further includes:

if the actual identification number is successfully matched with the reported identification number, receiving and storing the first image, the second image and the unpacking image uploaded by the robot;

and if the actual identification number is not matched with the reported identification number, receiving and storing the first image and the second image uploaded by the robot.

In a preferred embodiment of the method for inspecting a container by using a robot, the method further includes:

sending an inspection instruction comprising the position number and the inspection sequence of the container to be inspected to the robot; so that

And the robot establishes a navigation map based on the position numbers and the checking sequence, and performs container checking based on the navigation map.

The present application also provides a container inspection system, the system comprising:

an acquisition module configured to acquire a first image containing a current container to be inspected, which is captured by an image acquisition part on the robot, after the robot reaches an inspection position of the current container to be inspected;

a determination module configured to determine a container number and an actual identification number of the current container to be inspected based on the first image;

a query module configured to query the reported identification number of the current container to be inspected according to the container number;

a verification module configured to perform matching verification on the actual identification number and the reported identification number based on a query result.

In a preferred embodiment of the above container inspection system, the determining module is further configured to determine the container number and the actual identification number of the container currently under inspection based on the first image by:

identifying the first image by using a container number identification model to obtain the container number;

detecting the first image by using a position detection model to obtain position information of an envelope in the first image;

based on the position information, controlling the direction of a holder on the robot to adjust, and controlling the image acquisition part to adjust the focal length and focus so that the seal is positioned at the center of a viewing frame of the image acquisition part;

acquiring a second image containing the seal, which is shot by the image acquisition part;

and identifying the second image by using an identification number identification model to obtain the actual identification number.

In a preferred embodiment of the above container checking system, the verification module is further configured to perform matching verification on the actual identification number and the reported identification number based on a query result in the following manner:

if the actual identification number is successfully matched with the reported identification number, controlling the robot to send out unpacking prompt information, and shooting an unpacking image after preset time;

and if the actual identification number is not matched with the reported identification number, controlling the robot to send out alarm prompt information and move to the checking position of the next container to be checked.

In a preferred embodiment of the above container inspection system, the system further includes a receiving storage module, and the receiving storage module is configured to:

if the actual identification number is successfully matched with the reported identification number, receiving and storing the first image, the second image and the unpacking image uploaded by the robot;

and if the actual identification number is not matched with the reported identification number, receiving and storing the first image and the second image uploaded by the robot.

In a preferred embodiment of the above container inspection system, the system further includes an issuing module, and the issuing module is configured to:

sending an inspection instruction comprising the position number and the inspection sequence of the container to be inspected to the robot; so that

And the robot establishes a navigation map based on the position numbers and the checking sequence, and performs container checking based on the navigation map.

The present application further provides a computer apparatus comprising: a processor; a memory adapted to store a plurality of program codes adapted to be loaded and run by the processor to perform the method of any of the above preferred embodiments.

The present application also provides a computer readable storage medium having stored therein a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by a processor to perform the method according to any of the above preferred embodiments.

In a preferred embodiment of the present invention, a method for inspecting a container using a robot includes: when the robot reaches the checking position of the current container to be checked, acquiring a first image containing the current container to be checked, which is shot by an image acquisition part of the robot; determining a container number and an actual identification number of the current container to be detected based on the first image; inquiring the reported identification number of the current container to be detected according to the container number; and matching and verifying the actual identification number and the reported identification number based on the query result.

The inspection of the container to be inspected is realized by means of the robot, the automation of the inspection process is realized, the manpower and material resources are saved, the inspection efficiency is also improved, and the operation mode is more in line with the modern, scientific and intelligent customs inspection requirements. Through repeated tests, calculation, analysis and comparison of the inventor, the inspection time of a single container is about 1 minute (including the waiting time of 30 seconds for opening the container) when the inspection method is actually applied, and the inspection efficiency is greatly improved.

Furthermore, the position of the seal in the first image is obtained by detecting the first image, the direction of the holder and the focal length of the image acquisition part are adjusted and focused to obtain a second image containing the seal, the actual seal number is identified based on the second image, and the identification precision of the seal number can be improved by accurately positioning and further obtaining the image of the actual seal number, so that the inspection success rate and the inspection efficiency of the container are improved.

Furthermore, the robot is controlled to send out unpacking information and shoot unpacking images after the preset time when the actual identification number is successfully matched with the reported identification number, so that a worker can be prompted to carry out unpacking operation in time when the actual identification number is successfully matched with the reported identification number, and the unpacking images are automatically shot after the unpacking. On the other hand, when the matching between the actual identification number and the reported identification number fails, the controller sends alarm information and moves to the checking position of the next container to be checked, so that the alarm information can be reported in time when the matching fails and moves to the next checking position, and the checking efficiency is improved.

Furthermore, the robot can automatically establish a navigation map by issuing the position number and the checking sequence to the robot, and realize automatic checking based on the navigation map to realize automatic cruising operation of the robot.

Drawings

The method, system, apparatus and medium for container inspection using a robot according to the present invention will be described with reference to the accompanying drawings. In the drawings:

FIG. 1 is a block diagram of a robot in one possible embodiment of the present application;

FIG. 2 is a flow chart of a method of using a robot to inspect a container according to the present application;

FIG. 3 is a graph of the identification of container numbers in one possible embodiment of the present application;

FIG. 4 is a diagram of a sealing position result in one possible embodiment of the present application;

FIG. 5 is a graph of identification results of a seal number in one possible embodiment of the present application;

fig. 6 is a block diagram showing the structure of the container inspection system of the present application;

FIG. 7 is a logic diagram of one embodiment of the present application.

List of reference numerals

1. A main body; 2. a moving part; 21. a driving wheel; 22. a universal wheel; 3. an image acquisition unit; 31. an infrared camera; 32. a high-definition camera; 33. a driving camera; 4. a holder; 41. a body; 42. a second rotating shaft; 5. sounding; 61. a multiline laser radar; 62. an ultrasonic radar; 7. a connecting portion; 8. a system for container inspection; 81. an acquisition module; 82. a determination module; 83. a query module; 84. a verification module; 85. receiving a storage module; 86. and a sending module.

Detailed Description

Preferred embodiments of the present application are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only for explaining the technical principles of the present application, and are not intended to limit the scope of protection of the present application.

In the description of the present invention, a "module" or "processor" may include hardware, software, or a combination of both. A module may comprise hardware circuitry, various suitable sensors, communication ports, memory, may comprise software components such as program code, or may be a combination of software and hardware. The processor may be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and/or signal processing functionality. The processor may be implemented in software, hardware, or a combination thereof. Non-transitory computer readable storage media include any suitable medium that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random-access memory, and the like. The term "a and/or B" denotes all possible combinations of a and B, such as a alone, B alone or a and B. The term "at least one A or B" or "at least one of A and B" means similar to "A and/or B" and may include only A, only B, or both A and B. The singular forms "a", "an" and "the" may include the plural forms as well.

It should be noted that in the description of the present application, the terms "in", "up", "down", "vertical", "horizontal", "inner", etc. indicate directions or positional relationships based on those shown in the drawings, which are merely for convenience of description, and do not indicate or imply that the device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and thus, should not be construed as limiting the present application. Furthermore, the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.

Furthermore, it should be noted that, in the description of the present application, unless otherwise explicitly specified or limited, the terms "mounted" and "connected" are to be interpreted broadly, e.g., as being either fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the present application can be understood by those skilled in the art as the case may be.

First, some terms in the following embodiments are explained:

the container number: the identification is the most important identification in the container transportation process, the container adopts the ISO6346(1995) standard, the box number is composed of 11-bit codes, and the identification comprises three parts: the first part is composed of 4-digit English letters, the second part is composed of 6-digit numbers, and the third part is a check code which is obtained by the operation of the first 4-digit letters and the second 6-digit numbers through a check rule.

And (3) sealing the container: is a device like a latch that is applied by a particular person after the cargo is loaded into the container and the door is properly closed. The seals can be classified into customs seals, business checks seals, and business seals according to the personnel applying the seals. Once the seal is properly locked, it cannot be opened unless violently broken, and the broken seal cannot be reused.

The seal number: each seal is provided with a unique number mark, namely a seal number. The identification numbers are composed of at least numbers, some of which also include combinations of letters and numbers.

OCR: optical Character Recognition, refers to a process in which an electronic device (e.g., a scanner or a digital camera) checks a Character printed on paper, determines its shape by detecting dark and light patterns, and then translates the shape into a computer text by a Character Recognition method; the method is characterized in that characters in a paper document are converted into an image file with a black-white dot matrix in an optical mode aiming at print characters, and the characters in the image are converted into a text format through recognition software for further editing and processing by word processing software.

Docker: docker is an open source application container engine, so that developers can pack their applications and dependency packages into a portable image, and then distribute the image to any popular Linux or Windows machine, and also realize virtualization. The containers are fully sandboxed without any interface between each other.

As described in the background art, the prior art center mainly depends on manpower to complete the inspection work of the container, and this inspection method is difficult to adapt to the multiplied inspection requirement of the container, resulting in low inspection efficiency and manpower and material resource consumption. In order to solve the problems, the inspection of the container to be inspected is realized by means of the robot, the automation of the inspection process is realized, the manpower and material resources are saved, the inspection efficiency is also improved, and the operation mode is more in line with the modern, scientific and intelligent customs inspection requirements.

Next, a method of inspecting a container using a robot according to the present application will be described.

First, a brief description will be given of a robot in the present application with reference to fig. 1, where fig. 1 is a structural diagram of a robot in one possible embodiment of the present application.

As shown in fig. 1, in one possible embodiment, the robot in the present application mainly includes a main body 1, a moving part 2, an image capturing part 3 and a pan-tilt 4, wherein the moving part 2 is disposed at the bottom of the main body 1 and is mainly used for realizing the movement of the main body 1; the pan-tilt 4 is arranged on the main body 1 and is mainly used for supporting the image acquisition part 3 and adjusting the orientation of the image acquisition part 3, and the image acquisition part 3 is arranged on the pan-tilt 4 and is mainly used for acquiring images. In addition, the robot also comprises a controller which is connected with the components so as to control the logic action of the components.

Specifically, the main body 1 is in a volute shape, the bottom of the volute is approximately rectangular, a mounting cavity is formed inside the main body 1, and a controller is mounted in the mounting cavity. Removal portion 2 is installed to the bottom of main part 1, and removal portion 2 includes action wheel 21 and universal wheel 22, and action wheel 21 is equipped with two, sets up respectively in the relative both sides of main part 1, and universal wheel 22 is equipped with four, lies in the four corners of main part 1 respectively. The robot mainly depends on the action wheel 21 to travel, and the universal wheel 2 is more nimble for the action wheel, can assist action wheel 21 drive robot and travel towards more directions, helps improving the flexibility and the accuracy nature of robot travel in-process.

The cradle head 4 is connected with the main body 1 through a connecting part 7, the connecting part 7 is columnar, one end of the connecting part is connected with the main body 1, and the other end of the connecting part is connected with the cradle head 4. The pan/tilt head 4 comprises a body 41, a first rotating shaft (not shown in the figure) and a second rotating shaft 42, wherein the lower end of the body 41 is connected with the connecting part 7, the lower end of the second rotating shaft 42 is connected with the body 41, the upper end of the second rotating shaft 42 is connected with the first rotating shaft, the second rotating shaft 42 extends along the vertical direction, and the rotating shaft thereof is in the vertical direction; the first axis of rotation extends in a horizontal direction, with its axis of rotation being in the horizontal direction. Image acquisition portion 3 includes an infrared camera 31, a high definition digtal camera 32 and four driving cameras 33, infrared camera 31 and high definition digtal camera 32 are connected respectively at the both ends of first pivot, so, infrared camera 31 and high definition digtal camera 32 can be rotatory around the horizontal direction through first pivot, and rotate around vertical direction through second pivot 42, in this application, infrared camera 31 and high definition digtal camera 32 can wind vertical direction at 0 to 360 within range rotations, wind horizontal direction at 0 to 180 within range rotations, thereby acquire the image information of more angle ranges. Four driving cameras 33 are arranged on the body 41, so that the robot can acquire information in more directions when moving. In the embodiment, the high definition camera 32 with pixels of 300w-600w is adopted, and the high definition camera 32 can be aimed at a target scene and magnify, capture or store a picture. In addition, the infrared camera 31 enables the robot to obtain surrounding environment information at night or in a dark environment, and the efficiency and the quality of the robot are guaranteed to be improved.

For realizing accurate location, the robot still includes the GPS module (not shown in the figure) of being connected with the controller, and the GPS module includes the GPS antenna, and in this application, the GPS antenna setting is inside cloud platform 4, specifically, sets up the top at first pivot to guarantee effectively that the GPS antenna is good and receive radio signal steadily.

In order to realize the communication connection with a server of the customs system (hereinafter, the customs system or the server for short), the robot further comprises a wireless communication module, the wireless communication module is connected with the controller, the robot is in communication connection with the server through the wireless communication module, and the wireless communication module can be a WiFi module, a 3G/4G/5G module, a Bluetooth module, a Zigbee module and the like. The server can be a cloud server or an acquired server.

The robot further comprises a single line laser radar (not shown in the figure), a multi-line laser radar 61 and an ultrasonic radar 62 which are connected with the controller, wherein the single line laser radar and the ultrasonic radar 62 are arranged on the main body 1, preferably, the single line laser radar and the multi-line laser radar 61 are arranged on one side of the forward moving direction of the robot, the single line laser radar is arranged on the main body 1, and the surrounding environment is scanned through the released laser lines so as to obtain the environment information. The multi-line laser radar 61 is located at the joint between the main body and the connecting portion 7, and the multi-line laser radar 61 scans the surrounding environment by releasing a plurality of laser lines to acquire surrounding environment information, thereby establishing a three-dimensional point cloud map related to the surrounding environment. The single-line laser radar is faster in response to angular frequency and sensitivity, high in scanning speed, high in resolution and high in reliability, and the multi-line laser radar 61 has a very large detection range and a very high detection precision, so that the scanning range can be effectively enlarged, and scanning blind areas are reduced. The robot in this embodiment is equipped with twelve way ultrasonic radar 62, and twelve way ultrasonic radar 62 sets gradually around main part 1 to enlarge the scope of patrolling and examining. So, can establish the scanogram in customs place through single line laser radar, multi-line laser radar 61 and ultrasonic radar 62 to the scanogram is more comprehensive, makes direction, distance etc. that the robot traveles more accurate, thereby improves the autonomous navigation ability of robot.

In addition, the robot further comprises a sound box 5, the sound box 5 is arranged on the side face of the main body 1 and connected with the controller, and the sound box 5 is mainly used for playing audio so as to finish interaction with workers.

It should be noted that the specific embodiment of the robot is only used for illustrating the principle of the present application, and is not intended to limit the protection scope of the present application, and those skilled in the art can adjust the specific form and composition of the robot without departing from the principle of the present application, as long as the adjusted robot includes at least the necessary functional components such as the main body 1, the moving part 2, the image capturing part 3, and the pan/tilt head 4.

Referring next to fig. 2 to 5, a method for inspecting a container using a robot according to the present invention will be described. Fig. 2 is a flowchart of a method for inspecting a container by using a robot according to the present application; FIG. 3 is a graph of the identification of container numbers in one possible embodiment of the present application; FIG. 4 is a diagram of a sealing position result in one possible embodiment of the present application;

fig. 5 is a diagram of identification results of a seal number in one possible embodiment of the present application.

As shown in fig. 2, the method for checking a container by using a robot of the present application mainly includes the following steps:

s101, when the robot reaches the checking position of the current container to be checked, a first image containing the current container to be checked and shot by an image acquisition part of the robot is acquired.

In one possible embodiment, for each checkpoint, a corresponding scan may be created by the robot for that site in advance and each container location numbered. Before the inspection task is executed, the customs system issues the position number and the inspection sequence of the container to be inspected to the robot as a task instruction, and after the robot receives the task instruction, an automatic navigation map is established according to the position number and the inspection sequence, wherein the automatic navigation map comprises a cruising route and an inspection position, and the inspection position is the distance between the robot and the container, and the distance can be positioned and determined based on a GPS antenna. After being ready, the robot starts performing the auto-cruise task, starting from the first ping location until the end of the last ping location.

After the robot reaches the inspection position of the current container to be inspected, a first image containing the current container to be inspected is shot through the high-definition camera. In one possible embodiment, a first image of the robot taken by the high definition camera is shown in fig. 3.

The robot can automatically establish a navigation chart by issuing the position number and the checking sequence to the robot, and automatic checking is realized based on the navigation chart, so that automatic cruising operation of the robot is realized.

S103, determining the container number and the actual identification number of the container to be detected currently based on the first image.

In a possible implementation manner, the step S103 specifically includes: identifying the first image by using a container number identification model to obtain a container number; detecting the first image by using a position detection model to obtain position information of the seal in the first image; based on the position information, controlling the direction of a holder on the robot to adjust, controlling the image acquisition part to adjust the focal length and focus so as to enable the seal to be positioned at the center of a viewing frame of the image acquisition part; acquiring a second image containing the seal, which is shot by an image acquisition part; and identifying the second image by using the identification number identification model to obtain an actual identification number.

After the robot acquires the first image, the first image can be uploaded to a server of a customs system, and the customs system performs recognition analysis on the first image. A container number identification model, a position detection model and an identification number identification model are deployed in a server of a customs system by means of Docker, and the identification number identification model, the identification number inquiry and the identification number comparison service are reported.

Preferably, the models are trained by a deep learning algorithm. The deep learning algorithm has made great progress in the fields of target detection/text detection and text content recognition, and the invention can adopt various deep learning algorithms to carry out OCR region detection and OCR content recognition. For example: target detection/text detection can be performed by using algorithms such as EAST, CTPN, YOLO, Fast-RCNN, textBox + +, SSD and the like, and text content recognition can be performed by using algorithms such as CRNN, FOTS, CTC and the like.

Specifically, in the embodiment, the container number recognition model includes three main functions of container detection, container text detection and container number text recognition, wherein the container detection is trained by using an SSD algorithm, the container text detection is trained by using a TextBoxes + + algorithm, and the container number text recognition is trained by using a CRNN + CTC algorithm. And after receiving the first image, the server inputs a container number identification model, the model determines the position of the container through a container detection function, determines the position of the container number through a container text detection function, and identifies the container number through the container number identification model. The specific recognition result can be seen in fig. 3.

The position detection model adopts an SSD algorithm to train, and because the seal is smaller than the target of the container, the application adapts the anchor-size aiming at the small target, so that the detection precision after adaptation is higher.

While identifying the container number, the server determines position information, such as three-dimensional coordinate information, etc., which is enclosed in the first image, through the position detection model. And after the three-dimensional coordinate information is obtained, the information is sent to the robot, and the robot calls the direction of the 3D positioning adjusting holder based on the three-dimensional coordinate information and controls the high-definition camera to adjust the focal length and focus so as to enable the seal to be positioned at the center of the view-finding frame. Then, a second image containing the seal is shot through the high-definition camera. The result of detecting the sealing position and the second image can be seen in fig. 4 and 5.

The identification number recognition model comprises three main functions of identification number text detection, identification text direction recognition and identification text recognition, wherein the identification number text detection is trained by TextBox + + and the identification text direction recognition is trained by resnet50, and the identification text recognition is trained by a CRNN + CTC algorithm. And after receiving the second image, the server inputs the second image into an identification number recognition model, the model determines the position of the identification number through an identification number text detection function, determines the text direction of the identification number through an identification text direction function, and recognizes the actual identification number of the container to be detected through the identification number recognition function. The specific recognition result can be seen in fig. 5. Because the direction of putting the seals is inconsistent, the identification of the direction of the text of the seals is increased in the application, so that the text is firstly rotated to the positive direction and then the subsequent text identification function is called to identify when each identification is carried out, and the identification accuracy is improved.

It should be noted that the principle and the training process of the deep learning algorithm are commonly used in the present application, and therefore, are not described in detail.

The position of the seal in the first image is obtained by detecting the first image, the direction of the holder and the focal length of the image acquisition part are adjusted and focused to obtain a second image containing the seal, and finally the actual seal number is identified based on the second image. Through training container number recognition model, position detection model and identification number recognition model based on the deep learning algorithm for this application can improve recognition efficiency and precision when discerning container number and actual identification number, especially to the recognition accuracy of actual identification number, has very big improvement.

Through repeated tests, observation, comparison and analysis of the applicant, the identification of the seals by adopting the method has the advantage that the accuracy of the whole process (including the positioning of a robot, the accuracy of navigation points, the identification of container numbers, the identification of seal positions, the identification of seal numbers and the like) can reach more than 90% for some common seal locks. For some less common locks, the model may be continuously updated by continuing training after a certain number of samples are collected.

Of course, the specific manner of identifying the container number and the seal number is not exclusive, and those skilled in the art can modify the manner without changing the principle of the present application. For example, the algorithms adopted by the models can be replaced, and the number of the algorithms can be increased or decreased based on the actual situation; for another example, when the pixel of the high definition camera is high enough, the identification number can be recognized only by amplifying the first image without shooting the second image.

And S105, inquiring the reported identification number of the current container to be detected according to the container number.

In a possible embodiment, after the server identifies the container number and the actual identification number, the server may query a customs declaration form corresponding to the container number in the customs system based on the container number, and obtain the reported identification number based on the customs declaration form information.

And S107, based on the query result, matching and verifying the actual identification number and the reported identification number.

In one possible embodiment, after the reported identification number is obtained, the actual identification number and the reported identification number are subjected to matching verification to determine whether the actual identification number and the reported identification number are the same. If the actual identification number is successfully matched with the reported identification number, controlling the robot to send out unpacking prompt information and shooting an unpacking image after preset time; and if the actual identification number is not matched with the reported identification number, controlling the robot to send out alarm prompt information and move to the inspection position of the next container to be inspected.

For example, if the actual identification number is the same as the reported identification number, an inspection passing instruction is issued to the robot, the robot receives the instruction and then sends a container number identification matching through a sound box, please execute the unpacking operation, and shoot the unpacking photo after 30 seconds to prompt a worker to carry out the unpacking operation, and then after waiting for 30 seconds, the unpacking photo is shot through a high-definition camera. If the actual identification number is different from the reported identification number, an inspection failure instruction is issued to the robot, the robot sends out a prompt sound of 'container number identification number is not matched and opening of the container' through a sound after receiving the instruction, and then the robot skips over the current container to the inspection position of the next container to be inspected to continue inspection operation. Meanwhile, the server can also inform relevant personnel of the information of the failure of opening the box in the forms of short messages, mails, WeChat and the like so as to prompt the relevant personnel to carry out secondary inspection.

The robot is controlled to send out unpacking information and shoot unpacking images after preset time when the actual identification number is successfully matched with the reported identification number, so that a worker can be prompted to carry out unpacking operation in time when the matching is successful, and the unpacking images are automatically shot after the unpacking. On the other hand, when the matching between the actual identification number and the reported identification number fails, the controller sends alarm information and moves to the checking position of the next container to be checked, so that the alarm information can be reported in time when the matching fails and moves to the next checking position, and the checking efficiency is improved.

In a possible implementation, after step S107, the method further includes: if the actual identification number is successfully matched with the reported identification number, receiving and storing a first image, a second image and an unpacking image uploaded by the robot; and if the matching of the actual identification number and the reported identification number fails, receiving and storing the first image and the second image uploaded by the robot.

Specifically, if matching is successful, at the moment, the robot takes three images, namely a first image, a second image and an opening box image, the robot uploads the three images to the customs system, and the customs system receives and stores the three images as image evidence of the checking process to be filed. If the matching fails, the robot shoots two images at the moment, namely the first image and the second image respectively, the robot uploads the two images to a customs system, and the customs system receives and stores the two images as an image evidence of the checking process to be filed.

By archiving the images acquired by the robot when the matching is successful and failed, the files can be kept in the checking process, and the subsequent evidence can be conveniently provided and the updating and the improvement of the checking method can be conveniently read.

Referring now to fig. 6, the container inspection system of the present application will be described. Fig. 6 is a block diagram illustrating a structure of the container inspection system according to the present invention.

As shown in fig. 6, the container inspection system 8 of the present application mainly includes an acquisition module 81, a determination module 82, an inquiry module 83, a verification module 84, a receiving storage module 85, and an issuing module 86. The acquisition module 81 is configured to acquire a first image containing the current container to be inspected, which is captured by an image acquisition part on the robot, after the robot reaches the inspection position of the current container to be inspected; the determination module 82 is configured to determine a container number and an actual identification number of the current container to be inspected based on the first image; the query module 83 is configured to query the reported identification number of the current container to be inspected according to the container number; the verification module 84 is configured to perform matching verification on the actual identification number and the reported identification number based on the query result; the receiving storage module 85 is configured to: if the actual identification number is successfully matched with the reported identification number, receiving and storing a first image, a second image and an unpacking image uploaded by the robot; if the matching of the actual identification number and the reported identification number fails, receiving and storing a first image and a second image uploaded by the robot; the issue module 86 is configured to: and issuing an inspection instruction comprising the position number and the inspection sequence of the container to be inspected to the robot so that the robot establishes a navigation map based on the position number and the inspection sequence and inspects the container based on the navigation map. In one embodiment, the detailed implementation function may be described in reference to steps S101 to S107.

In one possible embodiment, the determination module 82 is further configured to determine the container number and the actual identification number of the container currently under inspection based on the first image by: identifying the first image by using a container number identification model to obtain a container number; detecting the first image by using a position detection model to obtain position information of the seal in the first image; based on the position information, controlling the direction of a holder on the robot to adjust, controlling the image acquisition part to adjust the focal length and focus so as to enable the seal to be positioned at the center of a viewing frame of the image acquisition part; acquiring a second image containing the seal, which is shot by an image acquisition part; and identifying the second image by using the identification number identification model to obtain an actual identification number. The specific implementation function can be described in step S103.

In one possible implementation, one or more of the container number identification model, the location detection model, and the identification number identification model are trained based on a deep learning algorithm. The specific implementation function can be described in step S103.

In one possible embodiment, the verification module 84 is further configured to verify the matching of the actual identification number and the reported identification number based on the query result by: if the actual identification number is successfully matched with the reported identification number, controlling the robot to send out unpacking prompt information and shooting an unpacking image after preset time; and if the actual identification number is not matched with the reported identification number, controlling the robot to send out alarm prompt information and move to the inspection position of the next container to be inspected. The specific implementation function can be described in step S107.

In the embodiment of the method for checking a container by using a robot, the technical principles, the technical problems to be solved, and the technical effects of the two embodiments are similar, and it can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working process and related description of the system for checking a container by using a robot may refer to the content described in the embodiment of the method for checking a container by using a robot, and the description thereof is not repeated here.

It should be understood that, since the configuration of each module is only for explaining the functional units of the apparatus of the present invention, the corresponding physical devices of the modules may be the processor itself, or a part of software, a part of hardware, or a part of a combination of software and hardware in the processor. Thus, the number of individual modules in the figures is merely illustrative.

Those skilled in the art will appreciate that the various modules in the apparatus may be adaptively split or combined. Such splitting or combining of specific modules does not cause the technical solutions to deviate from the principle of the present invention, and therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.

Referring now to fig. 7, a process for performing a container inspection using a robot in one possible embodiment of the present application will be described.

As shown in fig. 7, in one possible implementation:

(1) when the readiness, the staff presses the button that the robot began the operation, and the robot is automatic to the initial position, treats all adjustments and finishes the back, reports: the robot starts the unpacking inspection operation and please leave a channel, and then starts the unpacking inspection operation.

(2) The robot stops at the checking position of the first container to be checked, the container photo is shot and uploaded to the server, the server calls the container number identification model, the sealing position detection model and the sealing number identification model to identify and obtain the container number and the actual sealing number corresponding to the container, then the reporting identification number corresponding to the container number in the system declaration form is inquired, and the actual sealing number is compared with the reporting identification number. If the two match then the robot broadcasts: the container number seals are matched, the container opening operation is executed, the container opening picture is shot after 30s, and the staff can perform the container opening operation at the moment; if the two do not match, the robot broadcasts: the container number identification numbers are not matched, the container opening is refused, at the moment, the robot skips the container to the checking position of the next container to be checked to continue the operation, and the server sends the information of the failure of the container opening to related personnel.

(3) When the robot finishes broadcasting and can open the container, the robot waits for 30s, and after the robot finishes opening the container, the robot can shoot the container opening light of a container and then go to the next point to continue the operation.

(4) After the box is opened successfully, the robot uploads three pictures to a customs server for storage, and the three pictures are respectively: a photo of the container to be detected, a focused seal photo and a box opening photo; the phone robot which fails to open the container only needs to upload two photos, namely the photo of the container to be detected and the focused seal photo.

(5) And after the robot finishes the inspection of all containers, the robot can automatically return to the initial position, and if the electric quantity is lower than the set threshold value, the robot can automatically charge.

It should be noted that although the foregoing embodiments describe each step as being in the foregoing sequence, those skilled in the art will understand that, in order to achieve the effect of the present embodiments, different steps need not be executed in such sequence, and they may be executed simultaneously (in parallel) or in reverse sequence, and these simple changes are all within the scope of the present invention.

It will be understood by those skilled in the art that all or part of the flow of the method according to the above-described embodiment may be implemented by a computer program, which is stored in a computer-readable storage medium and used for instructing related hardware, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying computer program code, media, U-disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier wave signals, telecommunications signals, software distribution media, and the like. It should be noted that the computer readable medium may contain other components which may be suitably increased or decreased as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, in accordance with legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunications signals.

The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in a server, client, or the like, according to embodiments of the present invention. The present invention may also be embodied as an apparatus or device program (e.g., PC program and PC program product) for carrying out a portion or all of the methods described herein. Such a program implementing the invention may be stored on a PC readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.

Further, the invention also provides a computer readable storage medium. In one embodiment of a computer-readable storage medium according to the present invention, the computer-readable storage medium may be configured to store a method for performing inspection of a container using a robot as described above, which may be loaded and executed by a processor to implement the method for performing inspection of a container using a robot as described above. For convenience of explanation, only the parts related to the embodiments of the present invention are shown, and details of the specific techniques are not disclosed. The computer readable storage medium may be a storage device formed by including various electronic devices, and optionally, the computer readable storage medium is a non-transitory computer readable storage medium in the embodiment of the present invention.

Furthermore, the invention also provides a computer device. In an embodiment of the computer apparatus according to the present invention, the computer apparatus includes a processor and a memory, the memory may be configured to store a program for executing the method for container inspection using a robot of the above-mentioned method embodiment, and the processor may be configured to execute the program in the memory, the program including but not limited to the program for executing the method for container inspection using a robot of the above-mentioned method embodiment. For convenience of explanation, only the parts related to the embodiments of the present invention are shown, and details of the specific techniques are not disclosed. The computer apparatus may be an apparatus device formed including various electronic devices.

So far, the technical solutions of the present application have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present application is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the present application, and the technical scheme after the changes or substitutions will fall into the protection scope of the present application.