WO2015139470A1

WO2015139470A1 - Data write/read method and apparatus in rdma and network system

Info

Publication number: WO2015139470A1
Application number: PCT/CN2014/091263
Authority: WO
Inventors: 孙群虎; 袁学文
Original assignee: 华为技术有限公司
Priority date: 2014-03-21
Filing date: 2014-11-17
Publication date: 2015-09-24
Also published as: CN103929415A; CN103929415B

Abstract

The present invention relates to the field of computers. Provided is a data write/read method in remote direct memory access (RDMA), so that in a case in which a client reads discrete memory addresses from or writes discrete memory addresses into a serving end, RDMA read operations and RDMA write operations are supported. The data read method in RDMA comprises: a first end sending an RDMA read request message to a second end, the RDMA read request message carrying specified addresses of data that needs to be read, and the specified addresses being discrete multiple addresses inside a memory area range of the second end; and the first end receiving data that is sent by the second end and corresponds to the specified addresses. The present invention further provides a corresponding apparatus.

Description

Method, device and network system for reading and writing data under RDMA

Technical field

The present invention relates to the field of communications, and in particular, to a method, apparatus, and network system for reading and writing data under remote direct memory access (RDMA).

Background technique

In computer computing, RDMA refers to direct memory access from the memory of one computer to the memory of another computer, without involving the operating system of any computer. RDMA allows for high-throughput, low-latency network connections, which is especially useful in large-scale parallel computer clusters.

Currently RDMA technology supports three data read and write modes: send/receive, RDMA read, and RDMA write. Among them, in the case where the client reads or writes the distributed memory address to the server, the RDMA read and the RDMA write operation may not be performed normally.

Summary of the invention

The embodiment of the invention provides a method, a device and a network system for reading and writing RDMA write data, which can support RDMA read operation and RDMA write operation when the client reads or writes a distributed memory address to the server. .

In a first aspect, a method for remote data access to read data under RDMA is provided, the method comprising:

The first end sends an RDMA read request message to the second end, where the RDMA read request message carries a specified address that needs to be read data, and the specified address is a plurality of addresses dispersed in the memory area of the second end ;

The first end receives data corresponding to the specified address sent by the second end.

In conjunction with the first aspect, in a first implementation manner of the first aspect, before the sending, by the first end, the RDMA read request message to the second end, the method further includes:

The first end acquires memory information of the memory area of the second end, where the memory information includes: address information and/or length information.

With reference to the first implementation of the first aspect, in a second implementation manner of the first aspect, the memory area of the second end includes a plurality of distributed memory areas, and the first end acquires a memory area of the second end The memory information includes:

The first end acquires memory information of each of the dispersed memory areas in the memory area of the second end.

With reference to the second implementation manner of the first aspect, in a third implementation manner of the first aspect, the specified address includes multiple addresses that are distributed in the multiple distributed memory regions,

The receiving, by the first end, the data corresponding to the specified address sent by the second end includes:

The first end receives data corresponding to each address in the specified address sent by the second end.

In conjunction with the first aspect, in a fourth implementation manner of the first aspect, before the sending, by the first end, the RDMA read request message to the second end, the method further includes:

The first end registers the network card with its own memory area.

In a second aspect, a method for remote data access to read data under RDMA is provided, the method comprising:

The second end receives the RDMA read request message sent by the first end, where the RDMA read request message carries a specified address that needs to be read data, and the specified address is a plurality of distributed areas within the memory area of the second end address;

The second end reads data corresponding to the specified address;

The second end sends data corresponding to the specified address to the first end.

With reference to the second aspect, in a first implementation manner of the second aspect, the reading, by the second end, the data corresponding to the specified address includes: the second end reading, corresponding to each address in the specified address data;

The sending, by the second end, the data corresponding to the specified address to the first end, the second end sends, to the first end, data corresponding to each address in the specified address.

With reference to the second aspect, in a second implementation manner of the second aspect, a plurality of distributed memory regions exist in the second end, where the specified address includes multiple addresses distributed in the plurality of distributed memory regions, then

The reading, by the second end, the data corresponding to the specified address, is: the second end reads data corresponding to each address in the specified address;

The sending, by the second end, the data corresponding to the specified address to the first end includes: the second The terminal sends data corresponding to each address in the specified address to the first end.

With reference to the second aspect, in a third implementation manner of the second aspect, before the receiving, by the second end, the RDMA read request message sent by the first end, the method further includes:

The second end registers the network card with its own memory area.

In a third aspect, a method for remote data access to data writes under RDMA is provided, the method comprising:

The first end encapsulates the data corresponding to the specified address and the specified address in the RDMA write data packet, where the specified address is a plurality of addresses dispersed within the memory area of the second end;

The first end sends the RDMA write data message to the second end.

With reference to the third aspect, in a first implementation manner of the third aspect, before the first end end, the data corresponding to the specified address to be written data and the specified address is encapsulated in the RDMA write data message, The method further includes:

The first end acquires memory information of the memory area of the second end, where the memory information of the memory area includes: address information and/or length information of the memory area.

With reference to the first implementation manner of the third aspect, in a second implementation manner of the third aspect, the memory area of the second end includes a plurality of distributed memory areas, and the first end acquires the memory area of the second end The memory information includes:

In conjunction with the second implementation of the third aspect, in a third implementation of the third aspect, the specified address includes a plurality of addresses distributed in the plurality of scatter memory regions.

With the third aspect, in a fourth implementation manner of the third aspect, after the first end sends the RDMA write data packet to the second end, the method further includes:

The first end receives an acknowledgement message sent by the second end indicating successful writing.

In a fourth aspect, a method for remote data access to data writes under RDMA is provided, the method comprising:

The second end receives the RDMA write data message sent by the first end, where the RDMA write data message carries a specified address that needs to be written data and data corresponding to the specified address, where the designated address is the Multiple addresses dispersed within the memory area of the second end;

The second end writes data corresponding to the specified address at the specified address.

With reference to the fourth aspect, in a first implementation manner of the fourth aspect, a plurality of distributed memory regions exist in the second end, where the specified address includes multiple addresses distributed in the plurality of distributed memory regions,

Writing, by the second end, the corresponding data at the specified address includes: the second end writing data corresponding to the address in each of the specified addresses.

In a fifth aspect, an apparatus for data reading under RDMA is provided, the apparatus comprising:

a sending module, configured to send an RDMA read request message to the second end, where the RDMA read request message carries a specified address that needs to be read data, where the specified address is dispersed in the memory area of the second end Address

The receiving module is configured to receive data corresponding to the specified address sent by the second end.

With reference to the fifth aspect, in a first implementation manner of the fifth aspect, the device further includes:

And an obtaining module, configured to acquire, after the sending module sends an RDMA read request message to the second end, the memory information of the memory area of the second end, where the memory information includes: address information and/or length information.

With reference to the first implementation manner of the fifth aspect, in a second implementation manner of the fifth aspect, the memory area of the second end includes a plurality of scatter memory areas, and the acquiring module is specifically configured to:

Obtaining memory information of each of the scattered memory areas in the memory area of the second end.

With reference to the second implementation manner of the fifth aspect, in a third implementation manner of the fifth aspect, the specified address includes a plurality of addresses distributed in the plurality of distributed memory regions,

The receiving module is specifically configured to:

Receiving data corresponding to each of the specified addresses sent by the second end.

With reference to the fifth aspect, in a fourth implementation manner of the fifth aspect, the device further includes:

The registration module is configured to register the network area of the memory module with the VR card before sending the RDMA read request message to the second end.

In a sixth aspect, an apparatus for data reading under RDMA is provided, wherein the apparatus comprises:

a receiving module, configured to receive an RDMA read request message sent by the first end, where the RDMA read request message carries a specified address that needs to be read data, where the specified address is dispersed within a memory area of the second end Multiple addresses;

a processing module, configured to read data corresponding to the specified address;

And a sending module, configured to send data corresponding to the specified address to the first end.

With reference to the sixth aspect, in a first implementation manner of the sixth aspect, the processing module is specifically configured to: read data corresponding to each address in the specified address;

The sending module is specifically configured to: send data corresponding to each address in the specified address to the first end.

With reference to the sixth aspect, in a second implementation manner of the sixth aspect, a plurality of distributed memory regions exist in the second end, and the specified address includes multiple addresses distributed in the plurality of distributed memory regions, then

The processing module is specifically configured to: read data corresponding to each address in the specified address;

In conjunction with the sixth aspect, in a third implementation manner of the sixth aspect, the device further includes:

The registration module is configured to register the memory area of the first end with the network card before receiving the RDMA read request message sent by the first end.

In a seventh aspect, an apparatus for data writing under RDMA is provided, the apparatus comprising:

a processing module, configured to encapsulate data corresponding to the specified address and the specified address in the RDMA write data packet, where the specified address is a plurality of addresses dispersed within the memory area of the second end ;

And a sending module, configured to send the RDMA write data message to the second end.

In conjunction with the seventh aspect, in a first implementation manner of the seventh aspect, the device further includes:

An acquiring module, configured to acquire, after the processing module encapsulates the data corresponding to the data to be written and the data corresponding to the specified address, in the RDMA write data packet, the memory information of the memory area of the second end, where The memory information of the memory area includes: address information and/or length information of the memory area.

With reference to the first implementation manner of the seventh aspect, in a second implementation manner of the seventh aspect, the memory area of the second end includes a plurality of scatter memory areas, and the acquiring module is specifically configured to:

With reference to the second implementation manner of the seventh aspect, in a third implementation manner of the seventh aspect, the specified address includes a plurality of addresses distributed in the plurality of distributed memory regions.

With reference to the seventh aspect, in a fourth implementation manner of the seventh aspect, the device further includes:

The receiving module is configured to: after the sending module sends the RDMA write data message to the second end, receive an acknowledgement message sent by the second end indicating successful writing.

In an eighth aspect, an apparatus for data writing under RDMA is provided, the apparatus comprising:

a receiving module, configured to receive an RDMA write data message sent by the first end, where the RDMA write data message carries a specified address that needs to be written data and data corresponding to the specified address, where the designated address is a plurality of addresses dispersed within a memory area of the second end;

And a processing module, configured to write data corresponding to the specified address at the specified address.

With reference to the eighth aspect, in a first implementation manner of the eighth aspect, a plurality of distributed memory regions exist in the second end, and the specified address includes multiple addresses distributed in the plurality of distributed memory regions,

The processing module is specifically configured to: write data corresponding to the address in each of the specified addresses.

According to a ninth aspect, a network system is provided, the network system comprising any one of the apparatus for reading RDMA data mentioned in the above fifth aspect, and the apparatus for reading any of the RDMA data mentioned in the sixth aspect above .

According to a tenth aspect, a network system is provided, the network system comprising any one of the apparatus for writing RDMA data mentioned in the seventh aspect, and any apparatus for writing RDMA data mentioned in the above eighth aspect .

After the foregoing technical solution, the method, device and network system for reading and writing data under RDMA provided by the embodiment of the present invention, when the first end performs RDMA read and write operations, can read data in the memory area of the second end. A plurality of distributed addresses are sent to the second end, and the second end performs reading and writing of data of the distributed address to implement data read and write operations under RDMA.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. Other drawings may also be obtained from those of ordinary skill in the art in light of the inventive work.

1 is a flowchart of a method for reading data under RDMA according to an embodiment of the present invention;

2 is a flowchart of another method for data reading under RDMA according to an embodiment of the present invention;

3 is a flowchart of a method for writing data under RDMA according to an embodiment of the present invention;

4 is a flowchart of another method for data writing under RDMA according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a method for reading data under RDMA according to an embodiment of the present invention; FIG.

6 is a schematic diagram of a method for writing data under RDMA according to an embodiment of the present invention;

7a-7c are structural block diagrams of an apparatus for reading data under RDMA according to an embodiment of the present invention;

8a-8b are structural block diagrams of an apparatus for reading data under an RD MA according to an embodiment of the present invention;

9a-9c are block diagrams showing the structure of an apparatus for writing data under RDMA according to an embodiment of the present invention;

FIG. 10 is a structural block diagram of an apparatus for writing data under RDMA according to an embodiment of the present invention.

detailed description

The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

1 is a flowchart of a method for reading data under RDMA according to an embodiment of the present invention. Referring to FIG. 1, a method for reading data under RDMA according to an embodiment of the present invention includes:

The first end sends an RDMA read request message to the second end, where the RDMA read request message carries a specified address that needs to be read data, and the specified address is dispersed in the memory area of the second end. Addresses.

It should be noted that the first end and the second end of the embodiment of the present invention are equal to each other, and they are distinguished only for convenience of description. In the embodiment of the present invention, the first end may be a client, and the second end may be a server; otherwise, the first end may also be a server, and the second end may be a client.

Optionally, before the sending, by the first end, the RDMA read request message to the second end, the method for reading the data in the RDMA according to the embodiment of the present invention may further include: the first end acquiring the second Memory information of the memory area of the end. The memory information of the memory area may include: address information and/or length information of the memory area.

Further, when the memory area of the second end includes a plurality of distributed memory areas, the acquiring, by the first end, the memory information of the memory area of the second end may include:

The method for obtaining the memory information of the memory area of the second end is not limited, and any manner of obtaining the memory information of the memory area of the second end by using the interaction between the first end and the second end may be adopted.

At the same time, before the first end sends the RDMA read request message to the second end, the first end may also register the network card with its own memory area.

12. The first end receives data corresponding to the specified address sent by the second end.

The receiving, by the first end, the data corresponding to the specified address sent by the second end may include: the first end receiving each of the specified addresses sent by the second end The data corresponding to the address.

When a plurality of distributed memory areas exist in the second end, each of the specified addresses may exist in different dispersed memory areas, and at this time, data corresponding to each address in the specified address comes from Data in different disparate memory regions.

In addition, the RDMA read request message sent by the first end to the second end may carry the memory information of the dispersed memory area corresponding to each address in the specified address, in addition to the specified address. In this way, after the second end knows the memory information of the dispersed memory area corresponding to each address in the specified address, the data corresponding to the address can be read in the corresponding distributed memory area.

In the RDMA data reading method provided by the embodiment of the present invention, when the first end performs the RDMA read operation, the first end can send a plurality of distributed addresses in the memory area of the second end that need to read data to the second end. The data of each distributed address is read by the second end to implement the data reading operation under RDMA.

2 is a flowchart of another method for data reading under RDMA according to an embodiment of the present invention. Referring to FIG. 2, a method for writing data under RDMA according to an embodiment of the present invention includes:

The second end receives the RDMA read request message sent by the first end, where the RDMA read request message carries a specified address that needs to be read data, and the specified address is dispersed within the memory area of the second end. Multiple addresses.

Optionally, the RDMA read request message further carries the memory information of the dispersed memory area corresponding to each address in the specified address. The memory information of the dispersed memory area may include address information and/or length information of the dispersed memory area.

At the same time, before the second end receives the RDMA read request message sent by the first end, the first end may also register the network card with its own memory area.

22. The second end reads data corresponding to the specified address.

The reading, by the second end, the data corresponding to the specified address, when the plurality of addresses are in the same memory area of the second end, may include: the second end reads the Designation The data corresponding to each address in the address.

In the second end, when there are a plurality of distributed memory areas, where the specified address includes a plurality of addresses distributed in the plurality of distributed memory areas, step 22 may be specifically: the second end reads the The data corresponding to each address in the specified address, wherein the data corresponding to each address may be located in a different distributed memory area.

When the RDMA read request message further carries the memory information of the scatter memory area corresponding to each address in the specified address, the second end may directly read the address corresponding to the address in the scatter memory area corresponding to each address. data.

23. The second end sends data corresponding to the specified address to the first end.

The sending, by the second end, the data corresponding to the specified address to the first end by the second end may include: the second end sending, to the first end, data corresponding to each address in the specified address.

In the second end, when there are a plurality of singular memory areas, where the specified address includes a plurality of addresses distributed in the plurality of scattered memory areas, the step 23 may be specifically: the second end is The first end sends data corresponding to each address in the specified address. The data corresponding to each address may be located in different dispersed memory areas.

The method for reading data in the RDMA provided by the embodiment of the present invention, after the first end acquires the memory information of the dispersed memory area at the second end, the RDMA read operation can be performed in the range of the dispersed memory area of the second end. The address of the read data is sent to the second end, and the data of the address is read by the second end to implement the data reading operation under the RDMA.

FIG. 3 is a flowchart of a method for writing data under RDMA according to an embodiment of the present invention. Referring to FIG. 3, a method for writing data under RDMA according to an embodiment of the present invention includes:

The first end encapsulates the data corresponding to the data to be written and the data corresponding to the specified address in an RDMA write data packet, where the specified address is a plurality of addresses dispersed in the memory area of the second end. .

The first end may acquire the data of the second end of the first end before the first end of the data is encapsulated in the RDMA write data packet. Memory information in the memory area. The memory information of the memory area may include: address information and/or length information of the memory area.

Further, the memory area of the second end may include a plurality of distributed memory areas. In this case, the acquiring, by the first end, the memory information of the memory area of the second end may include:

The present invention is not limited to how to obtain the memory information of the memory area of the second end, and any manner of obtaining the memory information of the memory area of the second end by using the interaction between the first end and the second end may be adopted.

Optionally, when there are multiple dispersed memory regions in the second end, the specified address may include multiple addresses distributed in the plurality of distributed memory regions.

In the embodiment of the present invention, the RDMA write data packet may further carry the memory information of the distributed memory area corresponding to each address in the specified address.

At the same time, before the step 31, the first end can also register the network card with its own memory area.

32. The first end sends the RDMA write data packet to the second end.

After the first end sends the RDMA write data message to the second end, the first end may receive an acknowledgement message sent by the second end indicating successful writing.

In the RDMA data writing method provided by the embodiment of the present invention, when the first end performs the RDMA write operation, the first end can send a plurality of distributed addresses and corresponding data to be written in the memory area of the second end. To the second end, the second end writes the corresponding data of the address at each address to implement the data writing operation under the RDMA.

4 is a flowchart of a method for writing data under RDMA according to an embodiment of the present invention. Referring to FIG. 4, a method for writing data under RDMA according to an embodiment of the present invention includes:

The second end receives the RDMA write data message sent by the first end, and the RDMA write data message carries a specified address that needs to be written data and data corresponding to the specified address, where the designated address is A plurality of addresses dispersed within the memory area of the second end.

The second end may register its own memory area before the second end receives the RDMA write data message sent by the first end.

42. The second end writes data corresponding to the specified address at the specified address.

Wherein the plurality of distributed memory areas exist in the second end, and the specified address includes a plurality of addresses distributed in the plurality of distributed memory areas, and the second end is in the specified address in step 42 Writing the corresponding data may include: the second end writing data corresponding to the address in each of the specified addresses. The data corresponding to each address may be located in different dispersed memory areas.

In the embodiment of the present invention, the RDMA write data packet may further carry the memory information of the distributed memory area corresponding to each address in the specified address. The memory information of the distributed memory area may include: address information and/or length information of the dispersed memory area.

In order to better understand the technical solution of the present invention, the present invention is further illustrated by the specific embodiment by taking the client to read or write the distributed memory address to the server.

FIG. 5 is a schematic diagram of a method for reading data under RDMA according to an embodiment of the present invention. Referring to FIG. 5, a method for reading data under RDMA according to an embodiment of the present invention includes:

51. The client uses the ibv_reg_mr interface to specify the addr parameter, and registers the memory area to the network card; the server registers the memory area to the network card in the same manner.

52. The client acquires memory information of a memory area of the server. The memory area of the server includes a distributed memory area and a contiguous memory area, and the memory information of the memory area includes: address information and/or length information of the memory area. In the embodiment of the present invention, the client may obtain the memory information of the memory area of the server in various different manners, including but not limited to, in the process of establishing an RDMA communication connection between the client and the server, the client The memory information of the memory area in the server is obtained through a private channel.

The client sends an RDMA read request message to the server, where the RDMA read request message carries a specified address that needs to be read data, where the specified address is a plurality of scattered within the memory area of the server. address. Optionally, if the server includes a plurality of distributed memory regions, the specified address may include multiple addresses distributed in the plurality of distributed memory regions. The RDMA read request message may further include memory information of the dispersed memory area corresponding to the specified address.

The server receives the RDMA read request message sent by the client, reads the data corresponding to the specified address, and sends the data corresponding to the specified address to the client. Specifically, the server may read data corresponding to each address in the specified address, and send data corresponding to each address in the specified address to the client.

In the embodiment of the present invention, if there are multiple distributed memory areas in the server, The specified address may include a plurality of addresses distributed in the plurality of discrete memory regions.

If the server has a plurality of distributed memory areas, and the specified address includes a plurality of addresses distributed in the plurality of distributed memory areas, the data that the server reads the specified address may be specifically The server reads data corresponding to each address in the specified address. At the same time, the sending, by the server, the data corresponding to the specified address to the client may be: the server sends the data corresponding to each address in the specified address to the client.

Taking the client to read three scattered memory addresses to the server as an example, the protocol header of the RDMA read request message can be as follows:

That is, each SrcTO corresponds to a scatter memory address. There are three SrcTOs for the three scattered memory addresses.

Among them, the meaning of each field is as follows:

SinkSTag: the memory space identifier of the data receiving end;

SinkTO: the offset of the starting address when the data is stored in the memory space of the receiving end;

RDMARDSZ: RDMA read message size;

SrcSTag: data source memory space identifier;

SrcTO: The offset of the starting address when the data is read from the source memory space.

It can be seen from the above that any two servers connected through the network can modify the RDMA read protocol header field in the RDMA protocol and solidify the capability to the RDMA-capable network card. The client can read the server once by RDMA. Data of the decentralized memory address is read to the local memory space.

FIG. 6 is a schematic diagram of a method for data writing under RDMA according to an embodiment of the present invention. Referring to FIG. 6, a method for reading data under RDMA according to an embodiment of the present invention includes:

61. The client uses the ibv_reg_mr interface to specify the addr parameter, and registers the memory area to the network card; the server registers the memory area to the network card in the same manner.

62. The client acquires memory information of a memory area of the server. The memory area of the server includes a distributed memory area and a contiguous memory area, and the memory information of the memory area includes: address information and/or length information of the memory area. In the embodiment of the present invention, the client may obtain the memory information of the memory area of the server in various different manners, including but not limited to, in the process of establishing an RDMA communication connection between the client and the server, the client The memory information of the dispersed memory area in the server may be obtained through a private channel.

63. The client encapsulates the data corresponding to the specified address and the specified address in an RDMA write data packet, and sends the RDMA write data packet to the server. The designated address is a plurality of distributed addresses in which a data needs to be written in a range of a memory area of the server. Optionally, if the server includes a plurality of distributed memory regions, the specified address may include multiple addresses distributed in the plurality of distributed memory regions. The RDMA write data message may further include memory information of the dispersed memory area corresponding to the specified address.

64. The server receives an RDMA write data message sent by the client, and writes data corresponding to the specified address at the specified address.

In the embodiment of the present invention, if there are multiple distributed memory areas in the server, the specified address may include multiple addresses distributed in the plurality of distributed memory areas.

The server may write the corresponding data at the specified address, where the server may write data corresponding to the address in each of the specified addresses.

For example, if the client writes three decentralized memory addresses to the server, the protocol header of the RDMA write data packet can be as follows:

That is, each Data Sink Tagged offset corresponds to a scatter memory address. There are three Data Sink Tagged offsets for the three scattered memory addresses.

Among them, the meaning of each field is as follows:

Data Sink STag: data space identifier of the data receiving end;

Data Sink Tagged offset: The offset of the starting address when the data is stored in the memory space of the receiving end;

RDMA write ULP Payload: Data payload.

It can be seen from the above that any two servers connected through the network can modify the RDMA protocol protocol header field in the RDMA protocol and solidify the capability to the RDMA-capable network card. The client can write the client once by RDMA write. The data of the decentralized memory address is written to the server memory space.

As can be seen from the description of the above embodiments, the method for reading and writing data under RDMA provided by the present invention has the following advantages:

First, the conventional RDMA read operation does not support reading the server-side distributed memory address data into the client's continuous or decentralized memory address, and only supports reading the server-side contiguous memory address into the client memory address. The present invention supports the read server. Distributes memory addresses and is compatible with traditional read contiguous memory addresses.

Second, the traditional RDMA write operation does not support writing client memory data to the server. The memory address only supports writing to the server contiguous memory address. The present invention supports the function of writing the server-side distributed memory address and being compatible with the traditional write server contiguous memory address.

Moreover, it should be noted that the described scenario of the present invention supports any platform and operating system such as a virtual machine environment, a physical machine environment, a Linux system, and a Windows system.

It should be noted that the “server” and “client” in the embodiments of the present invention are relatively speaking, and the distinction between them in the text is only for convenience of description, and is not intended to be limiting, for example, in the present invention. In the embodiment, they can be used interchangeably, that is, the server can be a client, and the client can also be a server.

Correspondingly, an embodiment of the present invention further provides an apparatus for reading data under RDMA. Referring to FIG. 7a, the device 70 for data reading under RDMA provided by the embodiment of the present invention includes a sending module 71 and a receiving module 72. among them:

The sending module 71 is configured to send an RDMA read request message to the second end, where the RDMA read request message carries a specified address that needs to be read data, where the specified address is dispersed within a range of the memory area of the second end Multiple addresses;

The receiving module 72 is configured to receive data corresponding to the specified address sent by the second end.

Optionally, referring to FIG. 7b, in one embodiment, the device 70 further includes an acquisition module 73. among them:

The obtaining module 73 is configured to acquire, after the sending module 71 sends an RDMA read request message to the second end, the memory information of the memory area of the second end, where the memory information includes: address information and/or length information.

Further, when the memory area of the second end includes a plurality of distributed memory areas, the obtaining module 73 may be specifically configured to:

Further, when the specified address includes a plurality of addresses distributed in the plurality of distributed memory areas, the receiving module 72 may be specifically configured to:

Optionally, referring to FIG. 7c, in an embodiment, the apparatus further includes: a registration module 74, configured to: forward the memory area to the network card before the sending module sends the RDMA read request message to the second end Sign up.

The apparatus for reading data under RDMA provided by the embodiment of the present invention, the first end is performing RDMA During the read operation, a plurality of distributed addresses in the memory area of the second end that need to be read data can be sent to the second end, and the data of each distributed address is read by the second end to implement data reading under RDMA. operating.

FIG. 8 is a structural block diagram of another apparatus for reading RDMA data according to an embodiment of the present invention. Referring to FIG. 8a, the device 80 for data reading under RDMA provided by the embodiment of the present invention includes a receiving module 81, a processing module 82, and a sending module 83. among them:

The receiving module 81 is configured to receive an RDMA read request message sent by the first end, where the RDMA read request message carries a specified address that needs to be read data, and the specified address is dispersed within a range of the memory area of the second end Multiple addresses;

The processing module 82 is configured to read data corresponding to the specified address;

The sending module 83 is configured to send data corresponding to the specified address to the first end.

Optionally, in an embodiment, the processing module 82 is specifically configured to: read data corresponding to each address in the specified address. The sending module 83 is specifically configured to: send, to the first end, data corresponding to each address in the specified address.

Optionally, when there are multiple dispersed memory regions in the second end, the specified address includes multiple addresses distributed in the plurality of distributed memory regions,

The processing module 82 is specifically configured to: read data corresponding to each address in the specified address;

The sending module 83 is specifically configured to: send, to the first end, data corresponding to each address in the specified address.

Optionally, referring to FIG. 8b, in an embodiment, the apparatus 80 further includes:

The registration module 84 is configured to register the memory area of the user with the network card before the receiving module 81 receives the RDMA read request message sent by the first end.

The apparatus for reading data under the RDMA provided by the embodiment of the present invention, after the first end acquires the memory information of the dispersed memory area of the second end, and performs an RDMA read operation, the second end of the dispersed memory area can be required. The address of the read data is sent to the second end, and the data of the address is read by the second end to implement the data reading operation under the RDMA.

FIG. 9 is a structural block diagram of another apparatus for writing RDMA data according to an embodiment of the present invention. Referring to FIG. 9a, the apparatus 90 for writing data under RDMA according to an embodiment of the present invention includes a processing module 91 and a sending module 92. among them:

The processing module 91 is configured to encapsulate, in the RDMA write data packet, data corresponding to the specified address that needs to be written into the data, and the specified address is a plurality of distributed in the memory area of the second end address;

The sending module 92 is configured to send the RDMA write data message to the second end.

Optionally, referring to FIG. 9b, in an embodiment, the device 90 further includes:

The obtaining module 93 is configured to acquire the memory information of the memory area of the second end before the processing module 91 encapsulates the data corresponding to the specified address and the specified address to be written in the RDMA write data packet. The memory information of the memory area includes: address information and/or length information of the memory area.

Further, when the memory area of the second end includes a plurality of distributed memory areas, the obtaining module 93 may be specifically configured to: acquire memory information of each of the dispersed memory areas in the memory area of the second end.

Optionally, the specified address in the embodiment of the present invention may include multiple addresses distributed in the plurality of distributed memory regions.

Optionally, referring to FIG. 9c, in an embodiment, the device 90 further includes:

The receiving module 94 is configured to receive, after the sending module sends the RDMA write data message to the second end, an acknowledgement message sent by the second end indicating successful writing.

The device for writing data under RDMA according to the embodiment of the present invention, when the first end performs an RDMA write operation, can send a plurality of distributed addresses and corresponding data to be written in the memory area of the second end. To the second end, the second end writes the corresponding data of the address at each address to implement the data writing operation under the RDMA.

FIG. 10 is a structural block diagram of another apparatus for writing RDMA data according to an embodiment of the present invention. Referring to FIG. 10, an apparatus 100 for data writing under RDMA according to an embodiment of the present invention includes a receiving module 101 and a processing module 102. among them:

The receiving module 101 is configured to receive an RDMA write data message sent by the first end, where the RDMA write data message carries a specified address that needs to be written data and data corresponding to the specified address, where the designated address is a plurality of addresses dispersed within a memory area of the second end;

The processing module 102 is configured to write data corresponding to the specified address at the specified address.

Optionally, when there are multiple dispersed memory regions in the second end, the specified address may include multiple addresses distributed in the plurality of distributed memory regions. At this time, the processing module 102 has The body is configured to: write data corresponding to the address in each of the specified addresses.

Furthermore, embodiments of the present invention also provide a network system that can include the RDMA reading device shown in any of Figures 7a-7c and the RDMA reading device shown in Figure 8a or Figure 8b.

In addition, the embodiment of the present invention further provides another network system, which may include the RDMA writing device shown in any of FIGS. 9a to 9c and the RDMA writing device shown in FIG.

It should be noted that the apparatus for reading data under RDMA and the apparatus for writing data under RDMA provided by the above embodiments are merely exemplified by the division of the above functional modules. In actual applications, the functions may be allocated according to needs. It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device for reading and writing data under the RDMA provided by the above embodiment is the same as the method for reading and writing the RDMA data. For the specific implementation process, refer to the method embodiment, and details are not described herein again.

A person skilled in the art may understand that all or part of the steps of implementing the above embodiments may be completed by hardware, or may be instructed by a program to execute related hardware, and the program may be stored in a computer readable storage medium. The storage medium mentioned may be a read only memory, a magnetic disk or an optical disk or the like.

The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalents, improvements, etc., which are within the spirit and scope of the present invention, should be included in the protection of the present invention. Within the scope.

Claims

A remote direct memory access method for reading data under RDMA, characterized in that the method comprises:

The first end sends an RDMA read request message to the second end, where the RDMA read request message carries a specified address that needs to be read data, and the specified address is a plurality of addresses dispersed in the memory area of the second end ;

The first end receives data corresponding to the specified address sent by the second end.
The method of claim 1, wherein before the sending, by the first end, the RDMA read request message to the second end, the method further comprises:

The first end acquires memory information of the memory area of the second end, where the memory information includes: address information and/or length information.
The method according to claim 2, wherein the memory area of the second end comprises a plurality of discrete memory areas, and the first end obtains the memory information of the memory area of the second end, including:

The first end acquires memory information of each of the dispersed memory areas in the memory area of the second end.
The method according to claim 3, wherein said specified address comprises a plurality of addresses distributed in said plurality of discrete memory regions,

The receiving, by the first end, the data corresponding to the specified address sent by the second end includes:

The first end receives data corresponding to each address in the specified address sent by the second end.
The method of claim 1, wherein before the sending, by the first end, the RDMA read request message to the second end, the method further comprises:

The first end registers the network card with its own memory area.
A remote direct memory access method for reading data under RDMA, characterized in that the method comprises:

The second end receives the RDMA read request message sent by the first end, where the RDMA read request message carries a specified address that needs to be read data, and the specified address is a plurality of distributed areas within the memory area of the second end Address

The second end reads data corresponding to the specified address;

The second end sends data corresponding to the specified address to the first end.
The method according to claim 6, wherein the reading, by the second end, the data corresponding to the specified address comprises: the second end reading data corresponding to each address in the specified address;

The sending, by the second end, the data corresponding to the specified address to the first end, the second end sends, to the first end, data corresponding to each address in the specified address.
The method according to claim 6, wherein a plurality of dispersed memory areas exist in the second end, and the specified address includes a plurality of addresses distributed in the plurality of distributed memory areas,

The reading, by the second end, the data corresponding to the specified address, is: the second end reads data corresponding to each address in the specified address;

The sending, by the second end, the data corresponding to the specified address to the first end, the second end sends, to the first end, data corresponding to each address in the specified address.
The method according to claim 6, wherein before the second end receives the RDMA read request message sent by the first end, the method further includes:

The second end registers the network card with its own memory area.
A remote direct memory access method for data writing under RDMA, characterized in that the method comprises:

The first end encapsulates the data corresponding to the specified address and the specified address in the RDMA write data packet, where the specified address is a plurality of addresses dispersed within the memory area of the second end;

The first end sends the RDMA write data message to the second end.
The method according to claim 10, wherein before the first end encapsulates the data corresponding to the specified address to be written and the specified address in the RDMA write data message, the method further include:

The first end acquires memory information of the memory area of the second end, where the memory information of the memory area includes: address information and/or length information of the memory area.
The method according to claim 11, wherein the memory area of the second end comprises a plurality of distributed memory areas, and the first end obtains the memory information of the memory area of the second end, including:

The first end acquires memory information of each of the dispersed memory areas in the memory area of the second end.
The method of claim 12 wherein said specified address comprises a plurality of addresses distributed in said plurality of discrete memory regions.
The method according to claim 10, wherein after the first end sends the RDMA write data message to the second end, the method further includes:

The first end receives an acknowledgement message sent by the second end indicating successful writing.
A remote direct memory access method for data writing under RDMA, characterized in that the method comprises:

The second end receives the RDMA write data message sent by the first end, where the RDMA write data message carries a specified address that needs to be written data and data corresponding to the specified address, where the designated address is the Multiple addresses dispersed within the memory area of the second end;

The second end writes data corresponding to the specified address at the specified address.
The method according to claim 15, wherein a plurality of distributed memory regions exist in the second end, and the specified address includes a plurality of addresses distributed in the plurality of distributed memory regions.

Writing, by the second end, the corresponding data at the specified address includes: the second end writing data corresponding to the address in each of the specified addresses.
A remote direct memory access device for reading data under RDMA, characterized in that the device comprises:

a sending module, configured to send an RDMA read request message to the second end, where the RDMA read request message carries a specified address that needs to be read data, where the specified address is dispersed in the memory area of the second end Address

The receiving module is configured to receive data corresponding to the specified address sent by the second end.
The device according to claim 17, wherein the device further comprises:

And an obtaining module, configured to acquire, after the sending module sends an RDMA read request message to the second end, the memory information of the memory area of the second end, where the memory information includes: address information and/or length information.
The device according to claim 18, wherein the memory area of the second end comprises a plurality of discrete memory areas, and the acquiring module is specifically configured to:

Obtaining memory information of each of the scattered memory areas in the memory area of the second end.
The apparatus according to claim 19, wherein said specified address comprises a plurality of addresses distributed in said plurality of dispersed memory areas,

The receiving module is specifically configured to:

Receiving data corresponding to each of the specified addresses sent by the second end.
The device according to claim 17, wherein the device further comprises:

The registration module is configured to register the network area of the memory module with the VR card before sending the RDMA read request message to the second end.
A remote direct memory access device for reading data under RDMA, characterized in that the device comprises:

a receiving module, configured to receive an RDMA read request message sent by the first end, where the RDMA read request message carries a specified address that needs to be read data, where the specified address is dispersed within a memory area of the second end Multiple addresses;

a processing module, configured to read data corresponding to the specified address;

And a sending module, configured to send data corresponding to the specified address to the first end.
The device according to claim 22, wherein the processing module is specifically configured to: read data corresponding to each address in the specified address;

The sending module is specifically configured to: send data corresponding to each address in the specified address to the first end.
The apparatus according to claim 22, wherein a plurality of distributed memory areas exist in the second end, and the specified address includes a plurality of addresses distributed in the plurality of distributed memory areas,

The processing module is specifically configured to: read data corresponding to each address in the specified address;

The sending module is specifically configured to: send data corresponding to each address in the specified address to the first end.
The device of claim 22, wherein the device further comprises:

The registration module is configured to register the memory area of the first end with the network card before receiving the RDMA read request message sent by the first end.
A remote direct memory access device for data writing under RDMA, characterized in that the device comprises:

a processing module, configured to: use a specified address that needs to write data and data corresponding to the specified address Encapsulating in an RDMA write data message, the specified address is a plurality of addresses dispersed within a range of the memory area of the second end;

And a sending module, configured to send the RDMA write data message to the second end.
The device of claim 26, wherein the device further comprises:

An acquiring module, configured to acquire, after the processing module encapsulates the data corresponding to the data to be written and the data corresponding to the specified address, in the RDMA write data packet, the memory information of the memory area of the second end, where The memory information of the memory area includes: address information and/or length information of the memory area.
The device according to claim 27, wherein the memory area of the second end comprises a plurality of discrete memory areas, and the obtaining module is specifically configured to:

Obtaining memory information of each of the scattered memory areas in the memory area of the second end.
The apparatus of claim 28, wherein the designated address comprises a plurality of addresses distributed in the plurality of discrete memory regions.
The device of claim 26, wherein the device further comprises:

The receiving module is configured to: after the sending module sends the RDMA write data message to the second end, receive an acknowledgement message sent by the second end indicating successful writing.
A remote direct memory access device for data writing under RDMA, characterized in that the device comprises:

a receiving module, configured to receive an RDMA write data message sent by the first end, where the RDMA write data message carries a specified address that needs to be written data and data corresponding to the specified address, where the designated address is a plurality of addresses dispersed within a memory area of the second end;

And a processing module, configured to write data corresponding to the specified address at the specified address.
The apparatus according to claim 31, wherein a plurality of distributed memory areas exist in the second end, and the specified address includes a plurality of addresses distributed in the plurality of distributed memory areas,

The processing module is specifically configured to: write data corresponding to the address in each of the specified addresses.
A network system, comprising the apparatus for remote data access under RDMA according to any one of claims 17-21, and the data reading under RDMA according to any one of claims 22-25 Take the device.
A network system, comprising the method of any one of claims 26-30 The apparatus for remote data access to data writing under RDMA and the apparatus for data writing under RDMA according to any one of claims 31-32.