1. A method for determining spectral reflectance, comprising:

acquiring a registration depth map and a target spectrum image;

acquiring depth information of a target point from the registered depth map, and acquiring a first gray value of the target point from the target spectral image;

calculating to obtain three-dimensional coordinate information of the target point in a coordinate system corresponding to the imaging spectrum device according to the depth information of the target point, a first internal parameter of the imaging spectrum device and first coordinate information of the target point in the target spectrum image;

determining normal vector information corresponding to the target point according to the neighborhood point of the target point;

inputting the three-dimensional coordinate information and the normal vector information into a preset diffuse reflection plate full illumination model to obtain a second gray value;

and obtaining the reflectivity of the diffuse reflection plate, and calculating the reflectivity of the target point according to the reflectivity of the diffuse reflection plate, the first gray value and the second gray value.

2. The method for determining spectral reflectance according to claim 1, wherein said obtaining a registered depth map comprises:

acquiring an initial depth map output by a 3D measurement device;

converting the initial depth map into a registered depth map according to the out-of-target parameters, the first internal parameters of the imaging spectroscopy device and the second internal parameters of the 3D measurement device.

3. The method for determining spectral reflectance according to claim 1, further comprising, prior to said acquiring a registered depth map and target spectral image:

acquiring an initial spectral image and a dark field image;

and obtaining the target spectral image according to the initial spectral image and the dark field image.

4. The method for determining spectral reflectance according to claim 1, wherein before said inputting said three-dimensional coordinate information and said normal vector information into a predetermined diffuse reflector plenoptic model to obtain a second gray scale value, further comprising:

acquiring sample data of each preset position posture under the target wavelength; the sample data comprises a sample gray value of a sample point, sample normal vector information and sample three-dimensional coordinate information;

and fitting the sample data to obtain a preset diffuse reflection plate full illumination model.

5. The method for determining spectral reflectance according to claim 1, wherein after the obtaining the reflectance of the diffusive reflective plate and calculating the reflectance of the target point from the reflectance of the diffusive reflective plate, the first gray scale value, and the second gray scale value, further comprising:

and determining the spectral reflectivity curve of the object to be detected according to the reflectivity of all the target points in the registration depth map.

6. The method for determining spectral reflectance according to claim 1, wherein the determining normal vector information corresponding to the target point according to the neighborhood point of the target point includes:

and determining a space curved surface formed by the neighborhood points according to the neighborhood points of the target point, and acquiring the normal vector information corresponding to the target point according to the space curved surface.

7. An apparatus for determining spectral reflectance, comprising:

the first acquisition unit is used for acquiring a registration depth map and a target spectrum image;

the second acquisition unit is used for acquiring the depth information of a target point from the registered depth map and acquiring a first gray value of the target point from the target spectral image;

the first calculating unit is used for calculating to obtain three-dimensional coordinate information of the target point in a coordinate system corresponding to the imaging spectrum device according to the depth information of the target point, a first internal parameter of the imaging spectrum device and first coordinate information of the target point in the target spectrum image;

the determining unit is used for determining the normal vector information corresponding to the target point according to the neighborhood point of the target point;

the first processing unit is used for inputting the three-dimensional coordinate information and the normal vector information into a preset diffuse reflection plate full illumination model to obtain a second gray value;

and the second calculating unit is used for acquiring the reflectivity of the diffuse reflection plate and calculating the reflectivity of the target point according to the reflectivity of the diffuse reflection plate, the first gray value and the second gray value.

8. The apparatus for determining spectral reflectance according to claim 7, wherein the first obtaining unit is specifically configured to:

acquiring an initial depth map output by a 3D measurement device;

converting the initial depth map into a registered depth map according to the out-of-target parameters, the first internal parameters of the imaging spectroscopy device and the second internal parameters of the 3D measurement device.

9. Apparatus for determining spectral reflectance, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 6.

Background

When the spectral reflectivity is measured, an indirect measurement method can be adopted, and the indirect measurement method directly utilizes the ratio of the target gray value detected by the spectral detection equipment to the gray value of the reference diffuse reflection plate to calculate the spectral reflectivity. When using indirect measurement, an important prerequisite is that the irradiance received by the object to be measured is equal to the irradiance received by the diffuse reflection plate, i.e. the diffuse reflection plate is required to be as close as possible to the object to be measured, so that the final measurement result can be obtained.

However, in a short-distance observation scene, the structural change of the surface of the object to be measured cannot be ignored, and the distance change between different parts of the object to be measured and the light source cannot be ignored, and at this time, the object to be measured cannot be regarded as a uniform plane, so that irradiance received by different parts of the object to be measured cannot be obtained by using a diffuse reflection plate with fixed posture and position. That is, in actual measurement, the irradiance received by the object to be measured and the irradiance received by the diffuse reflection plate are different greatly in some scenes, and the difference between the irradiance at the surface of the diffuse reflection plate and the surface of the object can cause errors in calculating the spectral reflectivity.

Disclosure of Invention

The embodiment of the application provides a method, a device and equipment for determining spectral reflectivity, which can solve the problems.

In a first aspect, an embodiment of the present application provides a method for determining spectral reflectivity, including:

acquiring a registration depth map and a target spectrum image;

acquiring depth information of a target point from the registered depth map, and acquiring a first gray value of the target point from the target spectral image;

calculating to obtain three-dimensional coordinate information of the target point in a coordinate system corresponding to the imaging spectrum device according to the depth information of the target point, a first internal parameter of the imaging spectrum device and first coordinate information of the target point in the target spectrum image;

determining normal vector information corresponding to the target point according to the neighborhood point of the target point;

inputting the three-dimensional coordinate information and the normal vector information into a preset diffuse reflection plate full illumination model to obtain a second gray value;

and obtaining the reflectivity of the diffuse reflection plate, and calculating the reflectivity of the target point according to the reflectivity of the diffuse reflection plate, the first gray value and the second gray value.

Further, the acquiring a registered depth map includes:

acquiring an initial depth map output by a 3D measurement device;

converting the initial depth map into a registered depth map according to the out-of-target parameters, the first internal parameters of the imaging spectroscopy device and the second internal parameters of the 3D measurement device.

Further, before the acquiring the registered depth map and the target spectrum image, the method further includes:

acquiring an initial spectral image and a dark field image;

and obtaining a target spectral image according to the initial spectral image and the dark field image.

Further, before the inputting the three-dimensional coordinate information and the normal vector information into a preset diffuse reflection plate full illumination model to obtain a second gray value, the method further includes:

acquiring sample data of each preset position posture under the target wavelength; the sample data comprises a sample gray value of a sample point, sample normal vector information and sample three-dimensional coordinate information;

and fitting the sample data to obtain a preset diffuse reflection plate full illumination model.

Further, after the obtaining the reflectivity of the diffuse reflection plate and calculating the reflectivity of the target point according to the reflectivity of the diffuse reflection plate, the first gray scale value and the second gray scale value, the method further includes:

and determining the spectral reflectivity curve of the object to be detected according to the reflectivity of all the target points in the registration depth map.

Further, the determining, according to the neighborhood point of the target point, the normal vector information corresponding to the target point includes:

and determining a space curved surface formed by the neighborhood points according to the neighborhood points of the target point, and acquiring the normal vector information corresponding to the target point according to the space curved surface.

In a second aspect, an embodiment of the present application provides an apparatus for determining spectral reflectance, including:

the first acquisition unit is used for acquiring a registration depth map and a target spectrum image;

the second acquisition unit is used for acquiring the depth information of a target point from the registered depth map and acquiring a first gray value of the target point from the target spectral image;

the first calculating unit is used for calculating to obtain three-dimensional coordinate information of the target point in a coordinate system corresponding to the imaging spectrum device according to the depth information of the target point, a first internal parameter of the imaging spectrum device and first coordinate information of the target point in the target spectrum image;

the determining unit is used for determining the normal vector information corresponding to the target point according to the neighborhood point of the target point;

the first processing unit is used for inputting the three-dimensional coordinate information and the normal vector information into a preset diffuse reflection plate full illumination model to obtain a second gray value;

and the second calculating unit is used for acquiring the reflectivity of the diffuse reflection plate and calculating the reflectivity of the target point according to the reflectivity of the diffuse reflection plate, the first gray value and the second gray value.

Further, the first obtaining unit is specifically configured to:

acquiring an initial depth map output by a 3D measurement device;

converting the initial depth map into a registered depth map according to the out-of-target parameters, the first internal parameters of the imaging spectroscopy device and the second internal parameters of the 3D measurement device.

Further, the apparatus for determining spectral reflectance further comprises:

a third acquisition unit for acquiring an initial spectral image and a dark field image;

and the second processing unit is used for obtaining a target spectral image according to the initial spectral image and the dark field image.

Further, the apparatus for determining spectral reflectance further comprises:

the third acquisition unit is used for acquiring sample data of each preset position posture under the target wavelength; the sample data comprises a sample gray value of a sample point, sample normal vector information and sample three-dimensional coordinate information;

and the third processing unit is used for fitting the sample data to obtain a preset diffuse reflection plate full illumination model.

Further, the apparatus for determining spectral reflectance further comprises:

and the fourth processing unit is used for determining the spectral reflectivity curve of the object to be detected according to the reflectivity of all the target points in the registration depth map.

Further, the determining unit is specifically configured to:

and determining a space curved surface formed by the neighborhood points according to the neighborhood points of the target point, and acquiring the normal vector information corresponding to the target point according to the space curved surface.

In a third aspect, an embodiment of the present application provides a device for determining spectral reflectance, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method for determining spectral reflectance according to the first aspect.

In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the method for determining spectral reflectance according to the first aspect.

In the embodiment of the application, a registration depth map and a target spectrum image are obtained; acquiring depth information of a target point from the registered depth map, and acquiring a first gray value of the target point; calculating to obtain three-dimensional coordinate information of the target point according to the depth information of the target point, a first internal parameter of the imaging spectrum device and first coordinate information of the target point in the target spectrum image; determining normal vector information corresponding to the target point according to the neighborhood point of the target point; inputting the three-dimensional coordinate information and the normal vector information into a preset diffuse reflection plate full-illumination model to obtain a second gray value; and obtaining the reflectivity of the diffuse reflection plate, and calculating the reflectivity of the target point according to the reflectivity of the diffuse reflection plate, the first gray value and the second gray value. According to the method, when the spectral reflectivity is measured according to a surface light source or other complex illumination conditions, a diffuse reflection plate with a determined position and posture is corrected in gray scale value to obtain a diffuse reflection body which is completely the same as the three-dimensional shape of an object to be measured, the diffuse reflection body is placed at the position which is completely the same as the object to be measured, at the moment, the irradiance received by each part of the diffuse reflection body is completely the same as the irradiance received by the same part of the object to be measured, and then the spectral reflectivity is calculated. Therefore, the phenomenon that the difference between the irradiance received by the object to be measured and the irradiance received by the diffuse reflection plate is large in certain scenes is avoided, the possible error of the spectral reflectivity in calculation is further avoided, and a more accurate data source can be provided for near-distance spectrum application.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

FIG. 1 is a schematic view of a scene of a human face prosthesis provided herein;

FIG. 2 is a schematic flow chart of a method for determining spectral reflectance according to a first embodiment of the present application;

fig. 3 is a schematic flowchart of a refinement of S104 in a method for determining spectral reflectance provided in the first embodiment of the present application;

FIG. 4 is a schematic diagram of a first line segment and a first included angle in a method for determining spectral reflectance according to a first embodiment of the present application;

FIG. 5 is a schematic flow chart diagram of another method for determining spectral reflectance as provided in the second embodiment of the present application;

FIG. 6 is a schematic diagram of a spectral reflectance determination apparatus provided in a third embodiment of the present application;

fig. 7 is a schematic diagram of a device for determining spectral reflectance according to a fourth embodiment of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.

Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.

In the measurement of the spectral reflectance, there can be adopted methods of: direct measurement and indirect measurement.

Direct measurements are generally only used in laboratories. The experimental environment and experimental conditions are accurately controlled in measurement, the reflected light power and the received light power of the target surface under each wavelength are measured by a precise optical instrument, and the spectral reflectivity is calculated by the ratio of the reflected light power and the received light power. The direct measurement method has strict measurement requirements and is not widely applied.

The indirect measurement method directly calculates the spectral reflectance using the ratio of the target gray value (DN value) detected by the spectral detection device to the gray value of the reference diffuse reflection plate. The indirect measurement method is easy and convenient to operate, and has low requirements on measurement environment and equipment, so that the indirect measurement method is quite wide in application. For example, fig. 1 is a scene (700nm image) of a prosthetic face, a spectral image of each wavelength band of the scene is captured by using an imaging spectrometer, and besides an object to be detected, a diffuse reflection plate is disposed near the object to be detected in the scene. The diffuse reflection plate has lambertian emission characteristics, the reflectivity of which is known and the brightness of the reflection in all directions is the same. In order to calculate the reflectivity curve of a point A on the forehead of the artificial face, the following operations are carried out:

1) shooting scene data by using an imaging spectrometer to obtain spectral images under various wave bands;

2) shielding a lens of an imaging spectrometer, shooting dark field data, and obtaining dark field images under various wave bands;

3) taking 700nm as an example, reading the gray value of the point A in the 700nm spectral image, assuming to be 112, and reading the gray value of the point A in the 700nm dark field image, assuming to be 12;

7) calculating the average gray value of the diffuse reflection plate area in the 700nm spectral image, assuming to be 210, and calculating the average gray value of the diffuse reflection plate area in the 700nm dark field image, assuming to be 10;

5) the reflectance of the diffuse reflection plate at 700nm is 98%, and the reflectance of the point a at 700nm is 79% (112-12)/(210-10) × 98%;

repeating the steps 3-5 on the spectral images with the wavelengths to obtain the reflectivity data of the point A at the wavelengths.

The above process can be expressed by the following formula:

rho is reflectance

0: target to be measured

1 reference diffuse reflection plate

DN₀And DN₁For gray values with dark background subtracted

The indirect measurement method calculates the reflectivity by using the ratio of gray values, and the theoretical derivation is as follows:

E_sirradiance at pixels of the sensor surface

L object radiance

M degree of radiation emergence of object

E, irradiance on the surface of the object

As can be seen from the above formula, in indirect measurement, there is an important assumption: the irradiance E0 received by the object to be measured is equal to the irradiance E1 received by the diffuse reflector, which is also why the diffuse reflector is as close to the object as possible in indirect measurement.

However, in actual measurements the above assumption does not always hold, i.e. the irradiance received by the object to be measured and the irradiance received by the diffuse reflector differ significantly in some scenarios.

In a large-scale observation scene, such as satellite observation, unmanned aerial vehicle observation and the like in a remote measurement scene or a remote sensing scene, the structural change of the surface of the object to be measured is omitted by scale, only macroscopic observation characteristics are shown, and the surface of the object to be measured can be regarded as a uniform plane. Meanwhile, because the distance between the object to be measured and the light source (such as the sun) is long, the illumination in the scene is more uniform. At this time, the diffuse reflection plate is placed near the object to be measured, and the irradiance received by the diffuse reflection plate and the object to be measured can be considered to be the same.

However, in a short-distance observation scene, the structural change of the surface of the object to be measured cannot be ignored, and the distance change between different parts of the object to be measured and the light source cannot be ignored, and at this time, the object to be measured cannot be regarded as a uniform plane, so that irradiance received by different parts of the object to be measured cannot be obtained by using a diffuse reflection plate with fixed posture and position.

That is, in actual measurement, the irradiance received by the object to be measured and the irradiance received by the diffuse reflection plate are different greatly in some scenes, and the difference between the irradiance at the surface of the diffuse reflection plate and the surface of the object can cause errors in calculating the spectral reflectivity.

To solve this problem, the present embodiment proposes a method of determining the spectral reflectance. Referring to fig. 2, fig. 2 is a schematic flow chart of a method for determining spectral reflectance according to a first embodiment of the present application. An execution subject of the method for determining spectral reflectance in the present embodiment is an apparatus having a function of determining spectral reflectance.

Before describing the method for determining the spectral reflectivity in detail, a hardware device used for implementing the method is described, which comprises: imaging spectroscopy equipment, 3D measuring equipment, lighting equipment.

The imaging spectrum device can be a common imaging spectrometer at present, such as a grating type, a rotary filter type, a picture type and the like; and the system can also be a novel imaging spectrometer such as an optical filter array. The method is characterized in that the spectral images of all wave bands/channels can be output.

The 3D measurement device may be a structured light sensor, a TOF sensor, a binocular sensor, or the like. The method is characterized in that the depth image can be output.

The lighting device can be a point light source with small geometric dimension, can uniformly emit light in all directions of space, and can also be a surface light source or a light source with a complex form, or even a combination of various complex light sources. The light source needs to maintain a stable luminous intensity during the measurement or the light source combination cannot vary greatly. In this embodiment, the illumination device is a point light source.

Before the method of this embodiment is implemented, hardware needs to be calibrated, an internal reference of the 3D measurement device, an internal reference of the imaging spectrum device, and an external reference between the two are calibrated by using a camera calibration method, and distance information between the point light source and the imaging spectrum device, that is, a translation vector from the light source to the measurement coordinate system of the imaging spectrum device, is accurately measured. And after the calibration is finished, keeping the relative pose among the three constant in the whole determination period of the spectral reflectivity.

The method for determining the spectral reflectance as shown in fig. 2 may include:

s101: and acquiring a registered depth map and a target spectrum image.

The device acquires a registered depth map and a target spectral image. Wherein, the registered depth map is a depth map observed by the imaging spectrum equipment. The target spectral image may be a spectral image with dark field data subtracted.

In particular, in acquiring the registered depth map, the device acquires an initial depth map output by the 3D measurement device; the initial depth map is converted into a registered depth map according to the target external parameters, the first internal parameters of the imaging spectroscopy device and the second internal parameters of the 3D measurement device. The target external parameter, the first internal parameter of the imaging spectrum device and the second internal parameter of the 3D measuring device are obtained by calibration when hardware is calibrated. The target external parameter is distance information between the point light source and the imaging spectrum device, namely a translation vector of a measurement coordinate system from the light source to the imaging spectrum device. The device converts the initial depth map output by the 3D measuring device into a registration depth map observed by the imaging spectrum device, and actually realizes pixel-by-pixel registration of the spectrum data and the 3D data.

When converting, the following formula may be adopted to convert the depth value corresponding to each point in the initial depth map, where the specific formula is as follows:

wherein, K_dA second internal parameter of the 3D measurement device; k_sFor imaging the first internal parameters of the spectroscopic device, the external parameters of the target include the rotation matrix and translation vector from the 3D measurement device to the spectroscopic measurement device, denoted R respectively_d2sAnd T_d2s；d_dOne point [ u ] in initial depth map output for 3D measuring equipment_d,v_d]Corresponding depth values, readable from the initial depth map; d_sRegistration of corresponding points [ u ] in depth map for post-conversion imaging spectroscopy apparatus_s,v_s]The depth value of (2). According to the above formula traversal initial depth map, a registration depth map corresponding to the target spectral image output by the imaging spectrum device pixel by pixel can be obtained, and therefore registration of the depth image data and the spectral image data is achieved.

When the target spectral image is obtained, the device can obtain an initial spectral image and a dark field image; and obtaining a target spectral image according to the initial spectral image and the dark field image. Specifically, after the light source emits light stably, a spectral image of a scene is collected by an imaging spectrometer; shielding a lens of an imaging spectrometer, and shooting dark field data to obtain a dark field image; and subtracting the dark field data of the dark field image of the corresponding wave band from the image data of the initial spectral image of each wave band to obtain the target spectral image with the dark field subtracted.

S102: and acquiring the depth information of the target point and the depth information of the diffuse reflection plate from the registered depth map.

The device acquires the depth information of the target point and the depth information of the diffuse reflection plate from the registered depth map. In particular, the depth information of the target point and the depth information of the diffusely reflecting plate may be read directly from the registered depth map. For example, assuming that the coordinate of the point a in the target spectral image is (m, n), the depth value of the point with the coordinate of (m, n) in the registered depth map is directly read, that is, the depth information of the point a is obtained.

The target point is any point on the object to be detected in the target spectrum image of a certain waveband.

S103: and acquiring a first gray value of the target point and a second gray value of the diffuse reflection plate from the target spectrum image.

The device acquires a first gray value of the target point and a second gray value of the diffuse reflection plate from the target spectrum image, and the first gray value of the target point and the second gray value of the diffuse reflection plate can be directly read from the target spectrum image.

S104: and correcting the second gray value according to the depth information of the target point and the depth information of the diffuse reflection plate to obtain a third gray value.

And the equipment corrects the second gray value according to the depth information of the target point and the depth information of the diffuse reflection plate to obtain a third gray value. The equipment can calculate the three-dimensional coordinates of the target point and the three-dimensional coordinates of the diffuse reflection plate according to the depth information of the target point and the depth information of the diffuse reflection plate, acquire the three-dimensional coordinates of points in the neighborhood of the target point, and correct the second gray value according to the three-dimensional coordinates of the points to obtain a corrected third gray value.

The principle of correcting the gray value of the diffuse reflection plate by using the three-dimensional coordinate data is the distance square inverse law of the irradiance of the point light source, and the principle is described as follows:

if the intensity of the point light source radiating to the space is known as I, the irradiance received by the object surface is inversely proportional to the distance from the object to the point light source and directly proportional to the cosine of the included angle between the illumination direction and the normal. Therefore, because the distances between the point C and the point A from the light source are different, and the included angles between the illumination directions of the two points and the normal are different, the irradiance received by the point C is different from the irradiance received by the point A, and therefore, an error exists in the calculation of the reflectivity by using the gray value obtained by observing the point C. The gray scale value of the point C can be corrected by using the three-dimensional coordinate data.

This problem can be described as: and knowing the gray value observed by the diffuse reflection plate under the known position posture, and solving the gray value observed by the diffuse reflection plate under the specified position posture (the same as the posture of the target A point). By utilizing the distance square inverse law of the point light source irradiance, the correction coefficient of the diffuse reflection plate irradiance after the position and the attitude are changed can be obtained, and the correction coefficient is the correction coefficient for observing the gray value due to the direct proportional relation between the irradiance and the gray value.

Specifically, S104 may include S1041 to S1043, and as shown in fig. 3, S1041 to S1043 are specifically as follows:

s1041: and calculating a first distance of a first line segment between the point light source and the target point according to the depth information of the target point, and calculating a first angle of a first included angle between the first line segment and a normal corresponding to the target point.

The device calculates a first distance of a first line segment between the point light source and the target point according to the depth information of the target point, and calculates a first angle of a first included angle between the first line segment and a normal corresponding to the target point. Specifically, as shown in fig. 4, a point a is a target point, a point B is a point light source, and a point C is a central point of the diffuse reflection plate. The line segment AB is a first line segment between the point light source and the target point, and a first distance d for marking the first line segment AB₀The first included angle between the first line segment AB and the corresponding normal of the target point A is theta₀。

Specifically, the apparatus calculates a first distance of a first line segment between the point light source and the target point according to the depth information of the target point. The device can calculate the coordinates of the point light source and the target point and obtain the first distance according to the coordinate information of the point light source and the target point. The equipment calculates and obtains a first three-dimensional coordinate of the target point in a coordinate system corresponding to the imaging spectrum equipment according to the depth information of the target point, a first internal parameter of the imaging spectrum equipment and first coordinate information of the target point in the target spectrum image; specifically, the following formula can be adopted:

wherein, (X, Y, Z) is a first three-dimensional coordinate of the target point in a coordinate system corresponding to the imaging spectrum device, (u)_s,v_s) Is the image coordinate of the point in the target spectral image, K_sFor imaging a first internal parameter of the spectroscopic apparatus, d_sIs the depth value of the point.

Similarly, the device acquires the depth information of the point light source from the registration depth map, and calculates a second three-dimensional coordinate of the point light source in a coordinate system corresponding to the imaging spectrum device according to the depth information of the point light source, the first internal parameter of the imaging spectrum device and the second coordinate information of the point light source in the target spectrum image; the calculation method of the second three-dimensional coordinate may refer to the related description in the calculation method of the first three-dimensional coordinate, and is not described herein again.

After the first three-dimensional coordinate and the second three-dimensional coordinate are obtained through calculation, the equipment calculates a first distance of a first line segment between the point light source and the target point according to the first three-dimensional coordinate and the second three-dimensional coordinate. For example, the coordinates of point A of the target point are known (X, Y, Z), and the coordinates of point B of the target point are also known (X)₁,Y₁,Z₁) Then a first distance d₀＝|AB|＝[(X-X₁)²+(Y-Y₁)²+(Z-Z₁)²]^1/2。

When a first angle of a first included angle between the first line segment and a normal corresponding to the target point is calculated, the equipment acquires a space curved surface formed by neighborhood points of the target point, and acquires a normal vector corresponding to the target point according to the space curved surface; calculating a target vector between the point light source and the target point according to the first three-dimensional coordinate and the second three-dimensional coordinate; and determining a first angle of a first included angle between the first line segment and the normal corresponding to the target point according to the normal vector and the target vector.

Specifically, the device needs to obtain a neighborhood point of the target point, and the neighborhood of the target point is not specifically limited, and for example, a 21 × 21 pixel window, or 31 × 31 pixels, or 71 × 71 pixels centered on the target point may be selected as the neighborhood of the target point. Wherein, when determining the neighborhood, a window with an odd width is generally selected.

The space curved surface formed by the neighborhood points of the target point is the surface contour formed by the neighborhood points, and may be specifically determined by determining the three-dimensional coordinates of the neighborhood points, where the method for calculating the three-dimensional coordinates of the neighborhood points may refer to the above calculation method of the first three-dimensional coordinates, and details are not repeated here. And reading the depth value of the neighborhood from the registration depth map according to the pixel coordinate of the neighborhood. Assuming that a point a is selected as a center in the target spectral image, and a window of 21 × 21 is used as a neighborhood of the point a, each neighborhood point in the neighborhood is traversed to solve the three-dimensional coordinates thereof, and the finally obtained three-dimensional coordinates of 441 points form point cloud data of the neighborhood of the point a, that is, a space curved surface formed by the neighborhood points of the target point.

Then, the device can analyze and calculate the space curved surface according to a principal component analysis algorithm to obtain a normal vector corresponding to the target point. The apparatus calculates a target vector between the point light source and the target point based on the first three-dimensional coordinates and the second three-dimensional coordinates, the target vector AB ═ X₁-X,Y₁-Y,Z₁-Z)。

The equipment determines a first angle of a first included angle between the first line segment and a normal corresponding to the target point according to the normal vector and the target vector, wherein the normal vector is n, the target vector is m, and then the first angle theta of the first included angle is₀＝arccos(m*n/|m||n|)。

S1042: and calculating a second distance of a second line segment between the point light source and the central point of the diffuse reflection plate according to the depth information of the diffuse reflection plate, and calculating a second angle of a second included angle between the second line segment and the normal corresponding to the diffuse reflection plate.

In S1042, the specific details of the method for calculating the second distance and the second angle may refer to the related description in S1071, which is not described herein again.

Obtaining a second angle theta of the second line segment and the second included angle by the similar method in S1041₁。

S1043: and correcting the second gray value according to the first distance, the second distance, the first angle and the second angle to obtain a third gray value.

The device corrects the second gray value according to the first distance, the second distance, the first angle and the second angle to obtain a third gray value, and specifically, the correction can be performed according to the following formula:

wherein DN₂Is the third gray value, DN₁Is a second gray scale value, θ₀Is a first angle, theta₁At a second angle, d₀Is a first distance, d₁Is the second distance.

S105: and acquiring the reflectivity of the diffuse reflection plate, and calculating the reflectivity of the target point according to the reflectivity of the diffuse reflection plate, the first gray value and the third gray value.

The equipment obtains the reflectivity of the diffuse reflection plate and obtains the reflectivity of the diffuse reflection plate according to the reflectivity of the diffuse reflection plate and the first gray value DN₀And the third gray value calculates the reflectivity of the target point. The reflectivity of a known diffuse reflector is ρ_rUsing the corrected second gray value DN₂Calculating target point reflectivity rho_AThe following were used:

the equipment can perform the calculation on each target point on the object to be detected to obtain the reflectivity of all the target points, the operation is repeated on all the wave bands, and then the equipment determines the spectral reflectivity curve of all the points of the object to be detected according to the reflectivity of all the target points.

In the embodiment of the application, a registration depth map and a target spectrum image are obtained; acquiring depth information of a target point and depth information of a diffuse reflection plate from the registered depth map; acquiring a first gray value of a target point and a second gray value of the diffuse reflection plate from the target spectrum image; correcting the second gray value according to the depth information of the target point and the depth information of the diffuse reflection plate to obtain a third gray value; and obtaining the reflectivity of the diffuse reflection plate, and calculating the reflectivity of the target point according to the reflectivity of the diffuse reflection plate, the first gray value and the third gray value. According to the method, when the lighting equipment is a point light source and the spectral reflectivity is measured, the gray value of a diffuse reflection plate with a determined position and posture is corrected to obtain a diffuse reflector which is completely the same as the three-dimensional shape of the object to be measured and is placed at the position completely the same as the object to be measured, the irradiance received by each part of the diffuse reflector is completely the same as the irradiance received by the same part of the object to be measured, and then the spectral reflectivity is calculated. Therefore, the phenomenon that the difference between the irradiance received by the object to be measured and the irradiance received by the diffuse reflection plate is large in certain scenes is avoided, the possible error of the spectral reflectivity in calculation is further avoided, and a more accurate data source can be provided for near-distance spectrum application.

In the above embodiments, a method is provided in which the lighting device is a point light source, and for a surface light source or other complex lighting conditions, the reflectivity of the target point may be determined in the following manner. Referring to fig. 5, fig. 5 is a schematic flow chart of another method for determining spectral reflectance according to a second embodiment of the present application. An execution subject of the method for determining spectral reflectance in the present embodiment is an apparatus having a function of determining spectral reflectance.

In this embodiment, the hardware devices and the hardware calibration method used in this embodiment are completely the same as those in the first embodiment, and are not described here again.

The method for determining the spectral reflectance as shown in fig. 5 may include:

s201: and acquiring a registered depth map and a target spectrum image.

The device acquires a registered depth map and a target spectral image. Wherein, the registered depth map is a depth map observed by the imaging spectrum equipment. The target spectral image may be a spectral image with dark field data subtracted.

Specifically, when acquiring the registered depth map, acquiring an initial depth map output by the 3D measurement device; and converting the initial depth map into a registered depth map according to the target external parameters, the first internal parameters of the imaging spectrum device and the second internal parameters of the 3D measuring device.

When the target spectral image is obtained, the equipment obtains an initial spectral image and a dark field image; and obtaining a target spectral image according to the initial spectral image and the dark field image.

The manner and the related details of acquiring the registered depth map and the target spectral image in S201 are the same as S101 in the first embodiment, and refer to the related description in S101, which is not described herein again.

S202: and acquiring the depth information of a target point from the registered depth map, and acquiring a first gray value of the target point from the target spectral image.

The equipment selects a target point, wherein the target point is any point on an object to be detected in a target spectral image of a certain wave band. The device obtains depth information of the target point from the registered depth map. In particular, the depth information of the target points may be read directly from the registered depth map. For example, assuming that the coordinate of the point a in the target spectral image is (m, n), the depth value of the point with the coordinate of (m, n) in the registered depth map is directly read, that is, the depth information of the point a is obtained.

The device acquires a first gray value of the target point from the target spectral image, and the first gray value of the target point can be directly read from the target spectral image.

S203: and calculating to obtain three-dimensional coordinate information of the target point in a coordinate system corresponding to the imaging spectrum device according to the depth information of the target point, the first internal parameter of the imaging spectrum device and the first coordinate information of the target point in the target spectrum image.

In S203, the method for calculating the three-dimensional coordinate information of the target point in the coordinate system corresponding to the imaging spectrum device according to the depth information of the target point, the first internal parameter of the imaging spectrum device, and the first coordinate information of the target point in the target spectrum image in S1071 in the first embodiment is completely consistent with the method for calculating the first three-dimensional coordinate of the target point in the coordinate system corresponding to the imaging spectrum device, which may refer to the related description in S1071, and is not described herein again.

S204: and determining normal vector information corresponding to the target point according to the neighborhood point of the target point.

The device determines the normal vector information corresponding to the target point according to the neighborhood points of the target point, and the device first determines the neighborhood points of the target point and the neighborhood of the target point, without specific limitation, for example, the device may select 21 × 21 pixels, 31 × 31 pixels, or 71 × 71 pixels, centered on the target point, as the neighborhood of the target point. Wherein, when determining the neighborhood, a window with an odd width is generally selected.

The device may determine normal vector information corresponding to the target point according to the neighborhood points of the target point, and may calculate coordinates of the neighborhood points by using a preset algorithm to obtain the normal vector information corresponding to the target point.

Specifically, the device determines a spatial curved surface formed by neighborhood points according to the neighborhood points of the target point, and obtains normal vector information corresponding to the target point according to the spatial curved surface. The space curved surface formed by the neighborhood points of the target point is the surface contour formed by the neighborhood points, and may be specifically determined by determining the three-dimensional coordinates of the neighborhood points, where the method for calculating the three-dimensional coordinates of the neighborhood points may be the method for calculating the first three-dimensional coordinates in the first embodiment, and details are not repeated here. The equipment can analyze and calculate the space curved surface according to a principal component analysis algorithm to obtain the corresponding normal vector information of the target point.

S205: and inputting the three-dimensional coordinate information and the normal vector information into a preset diffuse reflection plate full illumination model to obtain a second gray value.

The device stores a preset diffuse reflection plate full illumination model, wherein the input of the preset diffuse reflection plate full illumination model is three-dimensional coordinate information and normal vector information, and the output of the three-dimensional coordinate information and the normal vector information is a corrected second gray value. The preset diffuse reflection plate full-illumination model can be directly trained by equipment in advance, or can be transplanted to local-end equipment after training is completed by other equipment in advance.

The equipment inputs the three-dimensional coordinate information and the normal vector quantity information into a preset diffuse reflection plate full-illumination model to obtain a second gray scale value. The second gray value is the observation gray value of the diffuse reflection plate with the same distance and posture as the point A of the target point under the current complex illumination condition.

In one embodiment, the device may perform modeling in advance, and obtain a preset diffuse reflection plate full illumination model by fitting the obtained data. In the modeling stage, a certain wavelength is fixed and set as a target wavelength, and data of a certain position posture is acquired. The equipment obtains sample data of each preset position and posture under the target wavelength, and the sample data is subjected to fitting processing to obtain a preset diffuse reflection plate full illumination model.

The sample data comprises a sample gray value of a sample point, sample normal vector information and sample three-dimensional coordinate information. Taking a central point C of the diffuse reflection plate as an example, extracting a sample gray value DN of the central point C of the diffuse reflection plate, sample three-dimensional coordinate information (X, Y, Z) of the central point C of the diffuse reflection plate, and sample normal vector information [ m, n, k ] of the diffuse reflection plate to obtain a group of data [ DN, X, Y, Z, m, n, k ]; and extracting data of each preset position and posture at the wavelength. Using all these data, the diffuse reflector sample gray values DN versus (X, Y, Z, m, n, k) are fitted. The specific form of the fitting function is not limited, and can be uniformly expressed as follows:

DN＝F(X,Y,Z,m,n,k)

when the illumination condition is fixed, the model can output the observation gray value of the imaging spectrometer to the diffuse reflection plate at the position and the posture by giving any distance and posture of the diffuse reflection plate. And performing the above operation on all the working wave bands of the imaging spectrometer to obtain the diffuse reflection plate full illumination model, namely, a preset diffuse reflection plate full illumination model.

S206: and obtaining the reflectivity of the diffuse reflection plate, and calculating the reflectivity of the target point according to the reflectivity of the diffuse reflection plate, the first gray value and the second gray value.

The equipment obtains the reflectivity of the diffuse reflection plate and obtains the reflectivity of the diffuse reflection plate according to the reflectivity of the diffuse reflection plate and the first gray value DN₀And a second gray scaleValue DN₂The reflectivity of the target point is calculated. The reflectivity of a known diffuse reflector is ρ_rUsing the corrected second gray value DN₂Calculating target point reflectivity rho_AThe following were used:

the equipment can perform the calculation on each target point on the object to be detected to obtain the reflectivity of all the target points, the operation is repeated on all the wave bands, and then the equipment determines the spectral reflectivity curve of all the points of the object to be detected according to the reflectivity of all the target points.

In the embodiment of the application, a registration depth map and a target spectrum image are obtained; acquiring depth information of a target point from the registered depth map, and acquiring a first gray value of the target point; calculating to obtain three-dimensional coordinate information of the target point according to the depth information of the target point, a first internal parameter of the imaging spectrum device and first coordinate information of the target point in the target spectrum image; determining normal vector information corresponding to the target point according to the neighborhood point of the target point; inputting the three-dimensional coordinate information and the normal vector information into a preset diffuse reflection plate full-illumination model to obtain a second gray value; and obtaining the reflectivity of the diffuse reflection plate, and calculating the reflectivity of the target point according to the reflectivity of the diffuse reflection plate, the first gray value and the second gray value. According to the method, when the spectral reflectivity is measured according to a surface light source or other complex illumination conditions, a diffuse reflection plate with a determined position and posture is corrected in gray scale value to obtain a diffuse reflection body which is completely the same as the three-dimensional shape of an object to be measured, the diffuse reflection body is placed at the position which is completely the same as the object to be measured, at the moment, the irradiance received by each part of the diffuse reflection body is completely the same as the irradiance received by the same part of the object to be measured, and then the spectral reflectivity is calculated. Therefore, the phenomenon that the difference between the irradiance received by the object to be measured and the irradiance received by the diffuse reflection plate is large in certain scenes is avoided, the possible error of the spectral reflectivity in calculation is further avoided, and a more accurate data source can be provided for near-distance spectrum application.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

Referring to fig. 6, fig. 6 is a schematic diagram of an apparatus for determining spectral reflectance according to a third embodiment of the present application. The units are included for performing the steps in the corresponding embodiment of fig. 5. Please refer to fig. 5 for a related description of the embodiment. For convenience of explanation, only the portions related to the present embodiment are shown.

Referring to fig. 6, the spectral reflectance determining apparatus 6 includes:

a first obtaining unit 610, configured to obtain a registered depth map and a target spectral image;

a second obtaining unit 620, configured to obtain depth information of a target point from the registered depth map, and obtain a first gray value of the target point from the target spectral image;

a first calculating unit 630, configured to calculate, according to the depth information of the target point, a first internal parameter of an imaging spectrum device, and first coordinate information of the target point in the target spectrum image, to obtain three-dimensional coordinate information of the target point in a coordinate system corresponding to the imaging spectrum device;

a determining unit 640, configured to determine, according to a neighborhood point of the target point, normal vector information corresponding to the target point;

the first processing unit 650 is configured to input the three-dimensional coordinate information and the normal vector information into a preset diffuse reflection plate full illumination model to obtain a second gray scale value;

a second calculating unit 660, configured to obtain a reflectivity of the diffuse reflection plate, and calculate a reflectivity of the target point according to the reflectivity of the diffuse reflection plate, the first gray scale value, and the second gray scale value.

Further, the first obtaining unit 610 is specifically configured to:

acquiring an initial depth map output by a 3D measurement device;

converting the initial depth map into a registered depth map according to the out-of-target parameters, the first internal parameters of the imaging spectroscopy device and the second internal parameters of the 3D measurement device.

Further, the determination device 6 for spectral reflectance further includes:

a third acquisition unit for acquiring an initial spectral image and a dark field image;

and the second processing unit is used for obtaining a target spectral image according to the initial spectral image and the dark field image.

Further, the determination device 6 for spectral reflectance further includes:

the third acquisition unit is used for acquiring sample data of each preset position posture under the target wavelength; the sample data comprises a sample gray value of a sample point, sample normal vector information and sample three-dimensional coordinate information;

and the third processing unit is used for fitting the sample data to obtain a preset diffuse reflection plate full illumination model.

Further, the determination device 6 for spectral reflectance further includes:

and the fourth processing unit is used for determining the spectral reflectivity curve of the object to be detected according to the reflectivity of all the target points in the registration depth map.

Further, the determining unit 640 is specifically configured to:

and determining a space curved surface formed by the neighborhood points according to the neighborhood points of the target point, and acquiring the normal vector information corresponding to the target point according to the space curved surface.

Fig. 7 is a schematic diagram of a device for determining spectral reflectance according to a fourth embodiment of the present application. As shown in fig. 7, the spectral reflectance determining apparatus 7 of this embodiment includes: a processor 70, a memory 71 and a computer program 72, such as a determination program of spectral reflectance, stored in said memory 71 and executable on said processor 70. The processor 70, when executing the computer program 72, implements the steps in the various spectral reflectance determination method embodiments described above, such as steps 201 through 206 shown in fig. 5. Alternatively, the processor 70, when executing the computer program 72, implements the functions of the modules/units in the above-mentioned device embodiments, such as the functions of the modules 610 to 660 shown in fig. 6.

Illustratively, the computer program 72 may be partitioned into one or more modules/units that are stored in the memory 71 and executed by the processor 70 to accomplish the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions for describing the execution of the computer program 72 in the determination device 7 of the spectral reflectance. For example, the computer program 72 may be divided into a first acquiring unit, a second acquiring unit, a first calculating unit, a determining unit, a first processing unit, and a second calculating unit, and each unit has the following specific functions:

the first acquisition unit is used for acquiring a registration depth map and a target spectrum image;

the second acquisition unit is used for acquiring the depth information of a target point from the registered depth map and acquiring a first gray value of the target point from the target spectral image;

the first calculating unit is used for calculating to obtain three-dimensional coordinate information of the target point in a coordinate system corresponding to the imaging spectrum device according to the depth information of the target point, a first internal parameter of the imaging spectrum device and first coordinate information of the target point in the target spectrum image;

the determining unit is used for determining the normal vector information corresponding to the target point according to the neighborhood point of the target point;

the first processing unit is used for inputting the three-dimensional coordinate information and the normal vector information into a preset diffuse reflection plate full illumination model to obtain a second gray value;

and the second calculating unit is used for acquiring the reflectivity of the diffuse reflection plate and calculating the reflectivity of the target point according to the reflectivity of the diffuse reflection plate, the first gray value and the second gray value.

The spectral reflectance determination device may include, but is not limited to, a processor 70, a memory 71. It will be appreciated by those skilled in the art that fig. 7 is merely an example of the determination device 7 of spectral reflectivity and does not constitute a limitation of the determination device 7 of spectral reflectivity, and may comprise more or less components than those shown, or some components in combination, or different components, for example the determination device of spectral reflectivity may also comprise an input-output device, a network access device, a bus, etc.

The Processor 70 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 71 may be an internal storage unit of the determination device 7 of the spectral reflectivity, such as a hard disk or a memory of the determination device 7 of the spectral reflectivity. The memory 71 may also be an external storage device of the apparatus for determining spectral reflectance 7, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the apparatus for determining spectral reflectance 7. Further, the determination device 7 of the spectral reflectance may also comprise both an internal storage unit and an external storage device of the determination device 7 of the spectral reflectance. The memory 71 is used to store the computer program and other programs and data required by the apparatus for determination of the spectral reflectance. The memory 71 may also be used to temporarily store data that has been output or is to be output.

It should be noted that, for the information interaction, execution process, and other contents between the above-mentioned devices/units, the specific functions and technical effects thereof are based on the same concept as those of the embodiment of the method of the present application, and specific reference may be made to the part of the embodiment of the method, which is not described herein again.

An embodiment of the present application further provides a network device, where the network device includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, the processor implementing the steps of any of the various method embodiments described above when executing the computer program.

The embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps in the above-mentioned method embodiments.

The embodiments of the present application provide a computer program product, which when running on a mobile terminal, enables the mobile terminal to implement the steps in the above method embodiments when executed.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium and can implement the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing apparatus/terminal apparatus, a recording medium, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc. In certain jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and patent practice.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus/network device and method may be implemented in other ways. For example, the above-described apparatus/network device embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.