1. The mass AIS track data storage method is characterized by comprising the following steps:

receiving mass AIS track data to be stored, and encrypting the AIS track data to obtain encrypted data;

analyzing the type of the AIS track data, sending the AIS track data to a flash memory area or a magnetic hard disk area of the hybrid hard disk for storage based on the type of the AIS track data needing to be stored, so as to realize optimized storage of the data, and sending the encrypted data to a server;

step three, if the type of the AIS track data needing to be stored is static information, the AIS track data is sent to a magnetic hard disk area of the hybrid hard disk for storage;

step four, if the type of the AIS track data needing to be stored is dynamic information, audio and video files and the like, the data are sent to a flash memory area of the hybrid hard disk for storage;

and step five, if the storage space required by the AIS track data does not exceed the residual storage space of the flash memory area of the hybrid hard disk, storing the AIS track data into the flash memory area of the hybrid hard disk.

2. The mass AIS trajectory data storage method according to claim 1, wherein the first step comprises an automatic monitoring module and a decision module, and the decision module is used for judgment in a ship database.

3. The mass AIS trajectory data storage method according to claim 1, wherein the two steps include a data storage planning module, data in the data storage planning module are uniformly distributed among the clusters, load balancing is maintained, and data of failed nodes are normal data.

4. The mass AIS trajectory data storage method according to claim 3, wherein the normal state data includes associated data, the associated data adopts a standard Hadoop layout scheme, continuous operation on the data needs to be completed within a predetermined time, during the process, a large amount of useless data is transmitted, if the data retains a part of device attributes during uploading, all data on the device can be stored at the same node, and a self-mapping function is utilized to realize a stored continuous operation behavior, thereby greatly reducing the stage data communication time and improving the overall storage efficiency.

5. The mass AIS trajectory data storage method according to claim 1, wherein the AIS data static information in step three is inherent attribute information of a ship system, and the AIS data dynamic information in step four includes real-time data of a ship.

6. The mass AIS trajectory data storage method of claim 1, wherein step five includes a data storage optimization module, and clustering technology is provided in the data storage optimization module.

7. The mass AIS trajectory data storage method of claim 6, wherein the clustering technique is to connect mutually independent machines into a group through a high-speed network and to use a unified control system for regulation and management, so that the interaction between the client and the cluster and the interaction between the client and the independent server are the same.

8. The mass AIS trajectory data storage method according to claim 3, wherein the data storage planning module is such that nodes in the network are distributed in a tree structure, so that nodes of different sub-numbers need to be exchanged according to a computer connection mode, and the distance between different nodes is the number of hops that a node needs to pass from one node to another node.

9. The mass AIS trajectory data storage method according to claim 2, wherein the automatic monitoring module is used for extracting a monitored target in a ship database, further forming a monitoring queue, calculating priorities in the queue, storing all data, and facilitating tracking and monitoring of the target.

Background

The automatic identification system AIS of the ship can monitor the motion state of the ship in real time, and for ship operators, navigation dynamic information of the ship and other target ships can be known by using AIS data, so that actions such as ship navigation, collision avoidance and the like are facilitated; for the future of the maritime affairs administration department, the AIS data information can be used for managing and controlling the ship; at present, as an important navigation aid for ship collision avoidance and automatic information exchange, more and more ships are provided with AIS equipment, the deficiency of information acquired by radar is supplemented, the ships can automatically exchange dynamic information such as position, navigational speed, course and the like with other ships equipped with the same equipment in real time through the equipment, the sea damage accidents such as ship collision and the like can be effectively avoided, the ships can also send static information such as ship name, identification code, destination, whether dangerous goods are loaded or not to an AIS base station through the AIS, and meanwhile, the information such as navigation aid, alarm and the like sent by a management authority through the base station can be received.

Therefore, a large amount of data exists in the mass AIS tracks, which results in that the time for storing the data is prolonged, and the storage speed is reduced.

Disclosure of Invention

The invention provides a mass AIS track data storage method, which solves the problems that the time for storing data is prolonged and the storage speed is reduced due to the existence of a large amount of data in a mass AIS track, and meanwhile, a large amount of time is consumed for reading and retrieving the data due to the existence of a large amount of data in the AIS track, and the efficiency for inquiring the data is reduced.

In order to solve the technical problem, the mass AIS track data storage method provided by the invention comprises the following steps:

receiving mass AIS track data to be stored, and encrypting the AIS track data to obtain encrypted data;

analyzing the type of the AIS track data, sending the AIS track data to a flash memory area or a magnetic hard disk area of the hybrid hard disk for storage based on the type of the AIS track data needing to be stored, so as to realize optimized storage of the data, and sending the encrypted data to a server;

step three, if the type of the AIS track data needing to be stored is static information, the AIS track data is sent to a magnetic hard disk area of the hybrid hard disk for storage;

step four, if the type of the AIS track data needing to be stored is dynamic information, audio and video files and the like, the data are sent to a flash memory area of the hybrid hard disk for storage;

and step five, if the storage space required by the AIS track data does not exceed the residual storage space of the flash memory area of the hybrid hard disk, storing the AIS track data into the flash memory area of the hybrid hard disk.

Preferably, the first step includes an automatic monitoring module and a decision module, and the decision module is used for judgment in a ship database.

Preferably, the second step includes a data storage planning module, data in the data storage planning module is uniformly distributed in each cluster, load balance is maintained, and data of a failed node is normal data.

Preferably, the normal state data comprises associated data, the associated data adopts a standard Hadoop layout scheme, continuous operation on the data needs to be completed within appointed time, in the process, a large amount of useless data is transmitted, if the data retains part of equipment attributes during uploading, all the data on the equipment can be stored in the position of the same node, the self-mapping function is utilized, the stored continuous operation behavior is realized, the phase data communication time is greatly reduced, and the overall storage efficiency is improved.

Preferably, the AIS data static information in step three is attribute information inherent to the ship system, and the AIS data dynamic information in step four includes real-time data of the ship.

Preferably, the fifth step includes a data storage optimization module, and a cluster technology is arranged in the data storage optimization module.

Preferably, the cluster technology is that mutually independent machines are connected into a group through a high-speed network and are regulated and managed by using a uniform control system, so that the interaction between a client and a cluster is not different from the interaction between independent servers.

Preferably, the data storage planning module is that nodes in the network are distributed in a tree structure, so nodes with different sub-numbers need to be exchanged according to a computer connection mode, and the distance between different nodes is the hop number that one node needs to pass through to another node.

Preferably, the automatic monitoring module is used for extracting a monitoring target from a ship database, so as to form a monitoring queue, calculate the priority in the queue, store all data and facilitate the tracking and monitoring of the target.

Compared with the prior art, the invention has the beneficial effects that:

1. the intelligent data storage optimization method in the ship system is improved through the cooperation of the automatic monitoring and decision-making module, the data storage planning module and the data storage optimization module, and the hash value storage optimization method with multiple consistent data relativity is used, so that the data related to the main attribute of the system can be gathered in cluster nodes, and the data storage optimization speed is further accelerated.

2. Based on the cluster scheme, the method can be used for storing and retrieving mass AIS data, an index mechanism facing mass space-time data is established, an index structure comprises basic static, dynamic and track information indexes, and partial static information is introduced into the dynamic and track information indexes, so that the query efficiency is effectively improved, the query efficiency of the AIS track data is improved, and meanwhile, a foundation is laid for realizing efficient AIS data mining and analyzing.

Drawings

FIG. 1 is a schematic step diagram of an embodiment of a mass AIS trajectory data storage method provided by the present invention;

fig. 2 is a flow frame diagram of a data cluster according to an embodiment of the massive AIS trajectory data storage method provided in the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.

Please refer to fig. 1-2. The mass AIS track data storage method comprises the following steps:

receiving mass AIS track data to be stored, and encrypting the AIS track data to obtain encrypted data;

analyzing the type of the AIS track data, sending the AIS track data to a flash memory area or a magnetic hard disk area of the hybrid hard disk for storage based on the type of the AIS track data needing to be stored, so as to realize optimized storage of the data, and sending the encrypted data to a server;

step three, if the type of the AIS track data needing to be stored is static information, the AIS track data is sent to a magnetic hard disk area of the hybrid hard disk for storage;

step four, if the types of the AIS track data needing to be stored are dynamic information, audio and video files and the like, sending the data to a flash memory area of the hybrid hard disk for storage;

and fifthly, if the storage space required by the AIS track data does not exceed the residual storage space of the flash memory area of the hybrid hard disk, storing the AIS track data into the flash memory area of the hybrid hard disk, and if the storage space required by the AIS track data exceeds the residual storage space of the flash memory area of the hybrid hard disk, transferring the data with low storage level in the flash memory area of the hybrid hard disk into the magnetic hard disk area of the hybrid hard disk according to a preset data storage rule of the flash memory area of the hybrid hard disk so as to preferentially store the data with high storage level into the flash memory area of the hybrid hard disk.

The first step comprises an automatic monitoring module and a decision module, the decision module is used for judging in a ship database, the intelligent data storage optimization method in a ship system is improved through the cooperation of the automatic monitoring and decision module, a data storage planning module and a data storage optimization module, a Hash value storage optimization method with multiple consistent data relativity is used, the method can be gathered in cluster nodes according to data related to main attributes of the system, and further the data storage optimization speed is accelerated, the second step comprises the data storage planning module, the data in the data storage planning module are uniformly distributed in each cluster, the load balance is kept, the data of a fault node is normal state data, the normal state data comprises related data, the related data adopts a standard Hadoop layout scheme, and the continuous operation of the data is required to be completed within appointed time, in the process, a large amount of useless data is transmitted, if the data is uploaded and part of the equipment attribute is reserved, all the data on the equipment can be stored to the position of the same node, the self-mapping function is utilized to realize the continuous operation behavior of storage, so that the stage data communication time is greatly reduced, the overall storage efficiency is improved, the AIS data static information in the third step is the inherent attribute information of a ship system, the AIS data dynamic information in the fourth step comprises the real-time data of a ship, the fifth step comprises a data storage optimization module, a cluster technology is arranged in the data storage optimization module, the cluster technology is that mutually independent machines are connected into a group through a high-speed network and are regulated and managed by utilizing a unified control system, so that a client interacts with a cluster and an independent server, and the cluster technology allows a user to obtain higher performance only by paying lower cost, the reliability, expandability, flexibility and other aspects of the system are improved, the data storage planning module is characterized in that nodes in the network are distributed in a tree structure, so that nodes with different numbers need to be exchanged according to a computer connection mode, the distance between different nodes is the hop number which needs to be passed from one node to another node, the automatic monitoring module is used for extracting a monitoring target in a ship database, further forming a monitoring queue, calculating the priority in the queue, storing all data, facilitating the tracking and monitoring of the target, based on a cluster scheme, being used for storing and retrieving mass AIS data, establishing an indexing mechanism facing mass space-time data, the indexing mechanism comprises basic static, dynamic and track information indexes, and partial static information is introduced into the dynamic and track information indexes, thereby effectively improving the query efficiency, the query efficiency of the AIS track data is improved, and meanwhile, a foundation is laid for realizing efficient AIS data mining analysis.

The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be able to cover the technical scope of the present invention and the equivalent alternatives or modifications according to the technical solution and the inventive concept of the present invention within the technical scope of the present invention.