1. A cascading board, wherein the cascading board is respectively connected with a Solid State Disk (SSD) and a controller, the cascading board comprises an access processing device and a processor connected with the access processing device, wherein,

the processor is used for completing initialization of the access processing device and the SSD;

the access processing device is used for receiving an access command of the controller; and realizing the access to the SSD according to the access command.

2. The cascade plate of claim 1, wherein the cascade plate and the controller communicate via direct memory access RDMA.

3. The cascade plate according to claim 1 or 2,

the access processing device is further configured to notify the SSD to execute the operation of the access command; and returning data or status information over the RDMA network after receiving the interrupt from the SSD.

4. The cascade board of any one of claims 1 to 3, wherein the cascade board comprises a peripheral component interface express (PCIe) switch chip, the PCIe switch chip being connected to the CPU and the access processing device, respectively, and being connected to the SSD.

5. The cascade plate of claim 4, wherein the access processing means comprises: a PCIe interface, a RDMA interface, and a memory;

the PCIe interface is used for connecting to the PCIe exchange chip;

the RDMA interface is to connect to an RDMA network;

the memory is used for storing an access queue.

6. A cascade plate according to any of claims 1 to 3, characterized in that the access processing means comprises: an RDMA interface and a memory, wherein:

the RDMA interface is to connect to an RDMA network;

the memory is used for storing an access queue.

7. The cascade plate according to any one of claims 1 to 6, wherein the access processing means is a field programmable gate array FPGA or an application specific integrated circuit ASIC.

8. A system for SSD remote access of a solid state disk, the system comprising at least one SSD, at least one controller and a cascade board according to any one of claims 1 to 7, wherein:

the first controller is connected with the cascade board through an RDMA network and sends an access command to the access processing device through the RDMA network, and the first controller is any one of the at least one controller.

9. The system of claim 8, further comprising an RDMA switch to connect the at least one controller and the cascade plate.

10. The system of claim 8, wherein the at least one SSD is arranged in at least one SSD disk frame, one said cascading plate being included in each SSD disk frame of the at least one SSD disk frame.

Background

Solid State Drive (SSD) is a hard disk made with an array of Solid State electronic memory chips. The SSD has been popular with more and more people because of its advantages of fast read/write speed, low power consumption, portability, etc.

Currently, with the increasing capacity and performance of SSDs, there is an increasing interest in sharing a single SSD or a group of SSDs to other various devices connected via a network to fully utilize the characteristics of the SSDs.

In the process of implementing remote shared access to the SSD, the access to the SSD by the remote device is processed by a processor (CPU). That is, the processor receives an SSD Access command from a Remote Direct Memory Access (RDMA) network and forwards the command to a corresponding SSD. However, this access method introduces software processing time, resulting in increased access latency.

Disclosure of Invention

The embodiment of the invention provides a cascade plate, a system and a method for SSD remote shared access, which are used for reducing the time delay of the SSD remote shared access.

In a first aspect, there is provided a cascade board having one end connected to an RDMA network and the other end connected to an SSD, the cascade board comprising an access processing means and a CPU coupled to the access processing means, wherein:

the CPU is used for completing initialization of the access processing device and the SSD;

the access processing device is used for realizing the RDMA access to the SSD when receiving an access command from the RDMA network.

Wherein the access processing means is means for processing remote shared access, and the CPU does not process remote shared access any more but for initialization of the access processing means and the SSD. Therefore, software processing time introduced by the CPU for realizing the remote shared access to the SSD can be avoided, so that the time delay of the remote shared access to the SSD can be reduced, and the remote shared access to the SSD is realized without increasing access time.

With reference to the first aspect, in a first possible implementation manner, the access processing apparatus includes: an RDMA interface and a memory, wherein:

the RDMA interface is to connect to an RDMA network;

the memory is used for storing an access queue.

In the embodiment of the invention, RDMA to the SSD is completed by the access processing device, and the access processing device can be provided with a memory to store an access queue. The access processing device is provided with an RDMA interface for connecting with an RDMA network.

With reference to the first aspect, in a second possible implementation manner, the cascade board further includes a PCIe (Peripheral Component Interface Express) switch chip, where the PCIe switch chip is connected to the CPU and the access processing apparatus, and is connected to the SSD.

In the embodiment of the invention, when the PCIe exchange chip does not exist, the CPU is directly connected with the SSD, and the number of the SSDs connected by the CPU is limited and is not large by adopting the connection mode. Thus, when there are many SSDs, the CPU cannot meet the requirement of direct SSD. By introducing the PCIe switch chip in the embodiment of the present invention, on one hand, the cascade board can be connected to more SSDs through the PCIe switch chip 103, and on the other hand, the function of connecting the access processing apparatus 102 and the CPU 101 can be better performed, and the access processing apparatus 102 and the CPU 101 serve as an adaptor, and efficient forwarding and transmission of data are realized.

With reference to the second implementation manner of the first aspect, in a third possible implementation manner, the access processing apparatus includes: a PCIe interface, a RDMA interface, and a memory;

the PCIe interface is used for connecting to the PCIe exchange chip;

the RDMA interface is to connect to an RDMA network;

the memory is used for storing an access queue.

In the case where the PCIe switch chip is present in the cascade board, the access processing apparatus may be provided with a PCIe interface to connect to the PCIe switch chip, adaptively, in addition to the RDMA interface and the memory.

With reference to any one of the foregoing implementation manners of the first aspect, in a fourth possible implementation manner, the access processing device is a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC). FPGAs and ASICs are particularly suitable as access processing devices because of their faster processing speed.

With reference to the fourth implementation manner of the first aspect, in a fifth possible implementation manner, the CPU, the access processing apparatus, and the PCIe switch chip are integrated together. In an embodiment of the present invention, the CPU, the access processing device, and the PCIe interaction chip may be integrated together, for example, on one chip. In this way, space for the cascade plates can be saved.

In a second aspect, there is provided a system for SSD remote shared access, the system comprising at least one SSD, at least one controller and any of the cascade plates of the first aspect above; wherein:

the controller is connected with the cascade board through an RDMA network and sends an access command to the access processing device through the RDMA network.

The controller may store the controller.

According to the system for SSD remote shared access provided by the embodiment of the invention, the CPU completes the initialization of the access processing device and the SSD, and the access processing device completes the RDMA to the SSD, so that the initialization process and the access process can be divided into two mutually independent processes, the RDMA access to the SSD is completed by introducing the hardware of the access processing device (such as an FPGA or an ASIC), the software processing time introduced by the realization of the RDMA access to the SSD by the CPU can be avoided, the time delay of the remote shared access to the SSD can be reduced, and the access time is not increased while the remote shared access to the SSD is realized.

With reference to the second aspect, in one possible implementation manner, the system further includes an RDMA switch for connecting the controller and the cascade board.

When the number of the cascade boards is large, the system for SSD shared access according to the embodiment of the present invention may further include a plurality of RDMA switches, and the RDMA switches are used to implement connection to the plurality of cascade boards. Of course, within the scope of the invention, only one RDMA switch may be used for connection when the number of cascaded boards is small.

In a third aspect, a method for SSD remote shared access is provided, the method comprising:

the CPU initializes the access processing device and the SSD;

and the access processing device realizes the RDMA access to the SSD when receiving the access command from the RDMA network.

In the method for accessing the hard disk in the embodiment of the invention, the CPU completes the initialization of the access processing device and the SSD, and the access processing device completes the RDMA to the SSD, so that the initialization process and the access process can be divided into two mutually independent processes, the RDMA access to the SSD is completed by introducing the hardware of the access processing device (such as an FPGA or an ASIC), the software processing time introduced by the realization of the RDMA access to the SSD by the CPU can be avoided, the time delay of the remote shared access to the SSD can be reduced, and the remote shared access to the SSD is realized without increasing the access time.

With reference to the third aspect, in a first possible implementation manner, the initializing, by the CPU, the access processing apparatus includes:

the CPU allocates a storage space for storing the access queue on the access processing device according to the configuration information of the access queue and initializes the access queue;

the CPU designates an interrupt receiver as the access processing means.

In the embodiment of the present invention, the CPU completes initialization of the access processing apparatus, and during the initialization, the interrupt receiving end is designated as the access processing apparatus, so that the subsequent SSD feeds back an interrupt to the access processing apparatus instead of the CPU after the remote shared access command is executed. The access processing device can know that the SSD has executed the command according to the terminal fed back by the SSD.

With reference to the third aspect or the first implementation manner of the third aspect, in a second possible implementation manner, when the access processing apparatus receives an access command, implementing RDMA access to the SSD includes:

when the access processing device receives an access command, prompting an SSD corresponding to the access command to perform access operation;

and the access processing device returns data or state information through the RDMA network after receiving the interrupt from the SSD.

After the CPU completes initialization of the access processing apparatus, the subsequent access processing apparatus can process the SSD according to the new job flow when receiving the access command to the SSD.

With reference to any implementation manner of the third aspect, in a third possible implementation manner, the access processing device is an FPGA or an ASIC. FPGAs and ASICs are particularly suitable as access processing devices because of their faster processing speed.

Optionally, a PCIe interaction chip exists between the CPU and the access processing apparatus, and the interaction between the CPU and the access processing apparatus may be transmitted through the PCIe interaction chip.

In an embodiment of the present invention, the CPU, the access processing device, and the PCIe interaction chip may be integrated together, for example, on one chip. In this way, space for the cascade plates can be saved.

In the system and the method for remote shared access of the cascade board and the SSD, the CPU completes initialization of the access processing device and the SSD, and the access processing device completes RDMA (remote direct memory Access) to the SSD, so that the initialization process and the access process can be divided into two mutually independent processes, the RDMA access to the SSD is completed by introducing the hardware of the access processing device (such as an FPGA (field programmable gate array) or an ASIC (application specific integrated Circuit)) to avoid software processing time introduced by the CPU for realizing the RDMA access to the SSD, further, the time delay of the remote shared access to the SSD can be reduced, and the remote shared access to the SSD is realized without increasing the access time.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a basic framework diagram of a system for implementing remote shared access to an SSD according to an embodiment of the present invention;

FIG. 2 is a basic framework diagram of another system for implementing remote shared access to SSDs provided by an embodiment of the present invention;

fig. 3 is an internal structural diagram of an access processing apparatus provided in an embodiment of the present invention;

FIG. 4 is a basic framework diagram of another system for implementing remote shared access to SSDs provided by an embodiment of the present invention;

fig. 5 is a flowchart of a method for accessing a hard disk according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Fig. 1 is a basic framework diagram of a system for implementing remote shared access to an SSD according to an embodiment of the present invention.

Referring to fig. 1, a system for remote shared access to SSDs provided by an embodiment of the present invention may include a cascade board 10, a hard disk 104 connected to the cascade board, and a controller 106 connected to the cascade board through a network 110. The controller 106 may be a stand-alone device or a controller in a network device connected to the network. The controller 106 is configured to receive RDMA access requests and control the operation of the cascade board 10; the RDMA access request may come from any network device on the network. The hard disk 104 connected to the cascade board may be a solid state disk such as PCIe (Peripheral Component Interface Express) SSD, which may be NVMe SSD and non-NVMe SSD, wherein NVMe (NVM Express) is an SSD access protocol defined by Intel corporation. When the PCIe SSD is a non-NVMe SSD, an access protocol of the non-NVMe SSD is open. The network 110 may be an RDMA network. In the embodiment of the present invention, the storage controller 106 may be located in the same lan as the cascade board 10, but the storage controller 106 may also be located in a different lan from the cascade board 10.

The cascade plate 10 may include an access processing device 102 and a processor (CPU)101 coupled to the access processing device 102. The access processing means 102 in the cascade board may be connected to a network 110 and the processor 101 may be connected to a hard disk 104. The access processing device 102 may be an FPGA or an ASIC. Wherein:

the CPU 101 is configured to complete initialization of the access processing apparatus and the SSD when powering on.

The access processing device 102 is configured to implement an RDMA access to the SSD104 when an access command is received from the RDMA network 110.

Wherein an access command may be sent by the storage controller 106 to the access processing device 102 over the network 110.

The cascade board in the embodiment of the invention completes the initialization of the access processing device and the SSD by the CPU, and completes the RDMA to the SSD by the access processing device, thereby being capable of dividing the initialization process and the access process into two mutually independent processes, and completing the RDMA access to the SSD by introducing the hardware of the access processing device (such as an FPGA or an ASIC), thereby avoiding the software processing time introduced by the realization of the RDMA access to the SSD by the CPU, further being capable of reducing the time delay of the remote shared access to the SSD, and realizing the remote shared access to the SSD without increasing the access time.

Fig. 2 is a basic framework diagram of another system for implementing remote shared access to an SSD according to an embodiment of the present invention. In the following description, the access processing device 102 in the system shown in fig. 2 is an FPGA or an ASIC, the hard disk 104 is an NVMe SSD, and the network 110 is an RDMA network.

In addition to the embodiment shown in fig. 1, a PCIe switch chip 103 is introduced into the cascade board 10 in the system shown in fig. 2. That is, the cascade board 10 may include a PCIe switch chip 103 in addition to the CPU 101 and the FPGA or ASIC 102. The PCIe switch chip 103 is connected to the CPU 101 and the access processing apparatus 102, and is connected to the SSD 104.

Also included in the system of FIG. 2 is an RDMA network connection 105 located in the RDMA network 110, which acts as an interface to the RDMA network to connect the memory controller 106 and the FPGA or ASIC 102.

In the embodiment of the present invention, the RDMA network connection device 105 may be an FPGA or an ASIC, or may be a common network chip supporting an RDMA network.

NVMe SSD104 is a standard PCIe interface SSD. The NVMe SSD104 and the FPGA/ASIC102 are connected to the processor 101 through the PCIe switch chip 103, and the processor 101 serves as a host (host) of the NVMe SSD104 and the FPGA/ASIC102, so as to complete initialization configuration and management of the NVMe SSD104 and the FPGA/ASIC102, but the processor 101 does not participate in specific hard disk access services.

In this embodiment, the PCIe switch chip 103 is introduced into the cascade board 10, so that on one hand, the cascade board can connect more SSDs through the PCIe switch chip 103 (because the number of SSDs connected by the CPU is often limited in the manner shown in fig. 1), on the other hand, the cascade board can better function to connect the access processing apparatus 102 and the CPU 101, act as an adaptor between the access processing apparatus 102 and the CPU 101, and implement efficient forwarding and transmission of data.

In an embodiment of the present invention, the FPGA/ASIC102 may be a customized network chip, which is characterized by integrating a certain storage space (e.g., Random Access Memory (RAM)) inside the network chip and allowing the NVMe Access queue to be created and managed.

Fig. 3 is an internal configuration diagram of an access processing apparatus according to an embodiment of the present invention. Referring to fig. 3, the access processing device shown in fig. 3 is an FPGA or an ASIC. The FPGA or ASIC102 can include: PCIe interface 102-1, RDMA interface 102-2, and memory 102-3. Wherein:

the PCIe interface 102-1 is used for connecting to the PCIe switch chip 103;

the RDMA interface 102-2 is used to connect to an external network 110, such as an RDMA network;

the memory 102-3 is used to hold an access queue.

The Memory 102-3 may be a Random Access Memory (RAM), which allows the CPU 101 to create an NVMe Access queue in the storage space, and simultaneously triggers a doorbell controller (doorbell) of the corresponding NVMe SSD after the storage controller 106 operates a Completion Queue (CQ)/commit queue (SQ).

It should be noted that, in the case shown in fig. 1, that is, when the PCIe switch chip 103 does not exist, the access processing apparatus 102 may include only: RDMA interface 102-2 and memory 102-3, but not PCIe interface 102-1.

The system for remote shared access to the SSD, provided by the embodiment, can be suitable for the situation that a cascade board is connected with a plurality of SSDs, and can reduce the time delay of the remote shared access to the SSDs, so that the storage controller can realize efficient access to the plurality of SSDs.

Fig. 4 is a basic framework diagram of another system for implementing remote shared access to an SSD according to an embodiment of the present invention.

Referring to fig. 4, the system for remote shared access to SSDs in this embodiment includes a plurality of storage controllers 106 connected to one or more NVMe SSD disk frames 100 through a plurality of, for example, two RDMA switches 107, where the disk frame 100 is a dual-disk frame and includes two disk frame management units 10, and the disk frame management units 10 may be cascaded boards. Each disk frame management unit 10 contains a management processor (CPU)101, a PCIe switch chip 103, and an FPGA/ASIC102, and two disk frame management units 109 are connected to a number of dual port standard NVMe SSDs 104.

It should be noted that the system shown in fig. 4 is merely an example. The number of storage controllers, the number of switches, the number of disk frame management units 10, the number of NVMe SSDs 104 shown in fig. 4 are all non-limiting, and any other number of individual devices may be employed to implement the present invention within the scope of the present invention. For example, in the embodiment of the present invention, the disk frame 100 may also be a single control device, that is, only includes one disk frame management unit (i.e., cascade board) 10; as another example, the NVMe SSD may be a single port standard NVMe SSD.

In each of the above embodiments of the system for remote shared access to an SSD, the initialization of the access processing device and the SSD may be done by the CPU, and the RDMA access to the SSD is performed by the access processing device upon receiving an access command from the RDMA network, so that the remote direct access to the NVMe SSD can be achieved with little latency.

The embodiment of the invention also provides a method for accessing the hard disk. The method for accessing a hard disk according to the embodiment of the present invention is described below with reference to fig. 5. Referring to fig. 5, a method for accessing a hard disk according to an embodiment of the present invention may include:

51. the CPU initializes the access processing device and the SSD;

52. and the access processing device realizes the RDMA access to the SSD when receiving the access command from the RDMA network.

Wherein, the access processing device can be FPGA or ASIC.

Optionally, in an embodiment, the step 51 of initializing, by the CPU, the access processing device may include:

the CPU allocates a storage space for storing the access queue on the access processing device according to the configuration information of the access queue and initializes the access queue;

the CPU designates an interrupt receiver as the access processing means.

Optionally, in another embodiment, when the access processing apparatus receives the access command in step 52, implementing RDMA access to the SSD may include:

when the access processing device receives an access command, prompting an SSD corresponding to the access command to perform access operation;

and the access processing device returns data or state information through the RDMA network after receiving the interrupt from the SSD.

In the method for accessing the hard disk in the embodiment of the invention, the CPU completes the initialization of the access processing device and the SSD, and the access processing device completes the RDMA to the SSD, so that the initialization process and the access process can be divided into two mutually independent processes, the RDMA access to the SSD is completed by introducing the hardware of the access processing device (such as an FPGA or an ASIC), the software processing time introduced by the realization of the RDMA access to the SSD by the CPU can be avoided, the time delay of the remote shared access to the SSD can be reduced, and the remote shared access to the SSD is realized without increasing the access time.

To facilitate a further understanding of the specific implementation of the present invention. The initialization and access processes are explained further below.

Specifically, after the cascade board 10 is powered on, the process of initialization by the CPU 101 may be as follows:

1) the cascade plate 10 is powered on and started;

2) the CPU 101 performs PCIe peripheral initialization, for example, completes initialization of some relevant registers;

3) the CPU 101 initializes the access processing apparatus 102 and establishes communication of the access processing apparatus 102 with the memory controller 106;

4) CPU 101 performs device initialization for SSD 104;

5) the CPU 101 obtains queue configuration information from a local area according to a preset policy or obtains the queue configuration information from the storage controller 106, allocates a storage space (e.g., RAM) on the access processing device 102 and initializes the NVMe access queues, obtains a doorbell address and an interrupt number corresponding to each access queue according to NVMe protocol requirements, and writes the doorbell address and the interrupt number corresponding to each access queue into corresponding address spaces on the access processing device 102 and the SSD104 according to NVMe rules.

6) The CPU 101 configures a PCIe switch chip 103 to designate an interrupt receiving end as an access processing apparatus 102;

7) the CPU 101 sends the head and tail pointers of the access queue to the memory controller 106.

Wherein, the CPU 101 shown can obtain a head pointer and a tail pointer of the access queue when initializing the NVMe access queue in 5). Further, the head and tail pointers of the access queue may be sent to the memory controller 106 in 7).

It should be noted that the initialization process is described by taking the case where the cascade board 10 includes a PCIe switch chip as an example. When the cascade board 10 does not include the PCIe switch chip, the initialization process is similar to the above, and only the initialization content related to the PCIe switch chip 103 needs to be removed, which is not described herein again.

Taking the system shown in fig. 4 as an example, after the initialization is performed according to the above initialization procedure, the disk frame 108 may send the NVMe queues that have been initialized and their head-to-tail pointers to all the connected storage controllers 106 through the network. Some or all of the resources in the disk frame 108 can be obtained by negotiation among the multiple storage controllers 106, so as to avoid the risk of data consistency caused by access conflict.

Thus, after the CPU 101 completes initialization through the above process, when receiving a hard disk access request from the network 110 (e.g., RDMA network), the cascade board 10 in the subsequent disk box 108 will deliver the hard disk access request to the access processing apparatus 102 for processing, and the access processing apparatus 102 implements RDMA access to the SSD 104.

A typical read/write access flow will be described as an example. The flow of remote read-write access operations to the SSD may be as follows:

the memory controller 106 selects a required access queue, and finds a head address and a tail address of the access queue;

the memory controller 106 writes a read/write command into the SQ queue on the memory 102-3 in the access processing apparatus 102;

the access processing device 102 knocks a corresponding hard disk doorbell according to the corresponding relationship between the access queue and the doorbell number;

the hard disk (e.g., PCIe SSD)104 obtains and executes the commands in the SQ queue stored on memory 102-3;

after the hard disk 104 executes the command, writing an execution result into a CQ queue and initiating an interrupt to the access control device 102;

after receiving the interrupt, the access control device 102 returns data or status information to the storage controller 106;

after receiving the data or the state information, the memory controller 106 maintains a tail pointer of the access queue and updates a head pointer of the access queue;

and finishing the read-write operation.

As can be seen from the above, the SSD remote shared access method, the cascade board, and the SSD remote shared access system provided in the embodiments of the present invention may implement direct access to the NVMe SSD104 by the access processing device 102 (e.g., FPGA/ASIC), share the NVMe SSD104 with the standard PCIe interface to the plurality of storage controllers 106 through the RDMA network, and implement remote shared access to the NVMe SSD104 without increasing the read-write access time.

It should be noted that: the SSD remote shared access apparatus (e.g., the cascade board 10) provided in the above embodiment and the SSD remote shared access method embodiment belong to the same concept, and therefore, corresponding parts therebetween may be referred to each other, and for repeated contents, details are not described herein again.

It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that are within the spirit of the present invention are intended to be included therein.