Main MRPT website > C++ reference for MRPT 1.4.0
COpenNI2Sensor.h
Go to the documentation of this file.
1/* +---------------------------------------------------------------------------+
2 | Mobile Robot Programming Toolkit (MRPT) |
3 | http://www.mrpt.org/ |
4 | |
5 | Copyright (c) 2005-2016, Individual contributors, see AUTHORS file |
6 | See: http://www.mrpt.org/Authors - All rights reserved. |
7 | Released under BSD License. See details in http://www.mrpt.org/License |
8 +---------------------------------------------------------------------------+ */
9#ifndef mrpt_COpenNI2Sensor_H
10#define mrpt_COpenNI2Sensor_H
11
16
18
19
20namespace mrpt
21{
22 namespace hwdrivers
23 {
24 /** A class for grabing "range images", intensity images (either RGB or IR) and other information from an OpenNI2 sensor.
25 * This class permits to access several sensors simultaneously. The same options (resolution, fps, etc.) are used for every sensor.
26 *
27 * <h2>Configuration and usage:</h2> <hr>
28 * Data is returned as observations of type mrpt::obs::CObservation3DRangeScan.
29 * See those classes for documentation on their fields.
30 *
31 * As with any other CGenericSensor class, the normal sequence of methods to be called is:
32 * - CGenericSensor::loadConfig() - Or calls to the individual setXXX() to configure the sensor parameters.
33 * - COpenNI2Sensor::initialize() - to start the communication with the sensor.
34 * - call COpenNI2Sensor::getNextObservation() for getting the data.
35 *
36 * <h2>Calibration parameters</h2><hr>
37 * In this class we employ the OpenNI2 method to return depth images refered to the RGB camera. Otherwise we could specify
38 * an accurate transformation of depth images to 3D points, you'll have to calibrate your RGBD sensor for that, and supply
39 * the following <b>threee pieces of information</b> (default calibration data will be used otherwise, but they'll be not optimal for all sensors!):
40 * - Camera parameters for the RGB camera. See COpenNI2Sensor::setCameraParamsIntensity()
41 * - Camera parameters for the depth camera. See COpenNI2Sensor::setCameraParamsDepth()
42 * - The 3D relative pose of the two cameras. See COpenNI2Sensor::setRelativePoseIntensityWrtDepth()
43 *
44 * See http://www.mrpt.org/Kinect_calibration for a procedure to calibrate RGBD sensors with an interactive GUI program.
45 *
46 * <h2>Coordinates convention</h2><hr>
47 * The origin of coordinates is the focal point of the depth camera, with the axes oriented as in the
48 * diagram shown in mrpt::obs::CObservation3DRangeScan. Notice in that picture that the RGB camera is
49 * assumed to have axes as usual in computer vision, which differ from those for the depth camera.
50 *
51 * The X,Y,Z axes used to report the data from accelerometers coincide with those of the depth camera
52 * (e.g. the camera standing on a table would have an ACC_Z=-9.8m/s2).
53 *
54 * Notice however that, for consistency with stereo cameras, when loading the calibration parameters from
55 * a configuration file, the left-to-right pose increment is expected as if both RGB & IR cameras had
56 * their +Z axes pointing forward, +X to the right, +Y downwards (just like it's the standard in stereo cameras
57 * and in computer vision literature). In other words: the pose stored in this class uses a different
58 * axes convention for the depth camera than in a stereo camera, so when a pose L2R is loaded from a calibration file
59 * it's actually converted like:
60 *
61 * L2R(this class convention) = CPose3D(0,0,0,-90deg,0deg,-90deg) (+) L2R(in the config file)
62 *
63 *
64 * <h2>Some general comments</h2><hr>
65 * - Depth is grabbed in millimeters
66 * - This sensor can be also used from within rawlog-grabber to grab datasets within a robot with more sensors.
67 * - There is no built-in threading support, so if you use this class manually (not with-in rawlog-grabber),
68 * the ideal would be to create a thread and continuously request data from that thread (see mrpt::system::createThread ).
69 * - The intensity channel default to the RGB images, but it can be changed with setVideoChannel() to read the IR camera images (useful for calibrating).
70 * - There is a built-in support for an optional preview of the data on a window, so you don't need to even worry on creating a window to show them.
71 * - This class relies on an embedded version of libfreenect (you do NOT need to install it in your system). Thanks guys for the great job!
72 *
73 * <h2>Converting to 3D point cloud </h2><hr>
74 * You can convert the 3D observation into a 3D point cloud with this piece of code:
75 *
76 * \code
77 * mrpt::obs::CObservation3DRangeScan obs3D;
78 * mrpt::maps::CColouredPointsMap pntsMap;
79 * pntsMap.colorScheme.scheme = CColouredPointsMap::cmFromIntensityImage;
80 * pntsMap.loadFromRangeScan(obs3D);
81 * \endcode
82 *
83 * Then the point cloud mrpt::maps::CColouredPointsMap can be converted into an OpenGL object for
84 * rendering with mrpt::maps::CMetricMap::getAs3DObject() or alternatively with:
85 *
86 * \code
87 * mrpt::opengl::CPointCloudColouredPtr gl_points = mrpt::opengl::CPointCloudColoured::Create();
88 * gl_points->loadFromPointsMap(&pntsMap);
89 * \endcode
90 *
91 *
92 * <h2>Platform-specific comments</h2><hr>
93 * For more details, refer to <a href="http://openkinect.org/wiki/Main_Page" >libfreenect</a> documentation:
94 * - Linux: You'll need root privileges to access Kinect. Or, install <code> MRPT/scripts/51-kinect.rules </code> in <code>/etc/udev/rules.d/</code> to allow access to all users.
95 * - Windows:
96 * - Since MRPT 0.9.4 you'll only need to install <a href="http://sourceforge.net/projects/libusb-win32/files/libusb-win32-releases/" >libusb-win32</a>: download and extract the latest libusb-win32-bin-x.x.x.x.zip
97 * - To install the drivers, read this: http://openkinect.org/wiki/Getting_Started#Windows
98 * - MacOS: (write me!)
99 *
100 *
101 * <h2>Format of parameters for loading from a .ini file</h2><hr>
102 *
103 * \code
104 * PARAMETERS IN THE ".INI"-LIKE CONFIGURATION STRINGS:
105 * -------------------------------------------------------
106 * [supplied_section_name]
107 * sensorLabel = OPENNI2 // A text description
108 * preview_window = false // Show a window with a preview of the grabbed data in real-time
109 *
110 * device_number = 0 // Device index to open (0:first Kinect, 1:second Kinect,...)
111 *
112 * grab_image = true // Grab the RGB image channel? (Default=true)
113 * grab_depth = true // Grab the depth channel? (Default=true)
114 * grab_3D_points = true // Grab the 3D point cloud? (Default=true) If disabled, points can be generated later on.
115 *
116 * video_channel = VIDEO_CHANNEL_RGB // Optional. Can be: VIDEO_CHANNEL_RGB (default) or VIDEO_CHANNEL_IR
117 *
118 * pose_x=0 // Camera position in the robot (meters)
119 * pose_y=0
120 * pose_z=0
121 * pose_yaw=0 // Angles in degrees
122 * pose_pitch=0
123 * pose_roll=0
124 *
125 *
126 * // Kinect sensor calibration:
127 * // See http://www.mrpt.org/Kinect_and_MRPT
128 *
129 * // Left/Depth camera
130 * [supplied_section_name_LEFT]
131 * rawlog-grabber-ignore = true // Instructs rawlog-grabber to ignore this section (it is not a separate device!)
132 *
133 * resolution = [640 488]
134 * cx = 314.649173
135 * cy = 240.160459
136 * fx = 572.882768
137 * fy = 542.739980
138 * dist = [-4.747169e-03 -4.357976e-03 0.000000e+00 0.000000e+00 0.000000e+00] // The order is: [K1 K2 T1 T2 K3]
139 *
140 * // Right/RGB camera
141 * [supplied_section_name_RIGHT]
142 * rawlog-grabber-ignore = true // Instructs rawlog-grabber to ignore this section (it is not a separate device!)
143 *
144 * resolution = [640 480]
145 * cx = 322.515987
146 * cy = 259.055966
147 * fx = 521.179233
148 * fy = 493.033034
149 * dist = [5.858325e-02 3.856792e-02 0.000000e+00 0.000000e+00 0.000000e+00] // The order is: [K1 K2 T1 T2 K3]
150 *
151 * // Relative pose of the right camera wrt to the left camera:
152 * // This assumes that both camera frames are such that +Z points
153 * // forwards, and +X and +Y to the right and downwards.
154 * // For the actual coordinates employed in 3D observations, see figure in mrpt::obs::CObservation3DRangeScan
155 * [supplied_section_name_LEFT2RIGHT_POSE]
156 * rawlog-grabber-ignore = true // Instructs rawlog-grabber to ignore this section (it is not a separate device!)
157 *
158 * pose_quaternion = [0.025575 -0.000609 -0.001462 0.999987 0.002038 0.004335 -0.001693]
159 *
160 * \endcode
161 *
162 * More references to read:IMPEXP mrpt
163 * - http://http://www.openni.org/
164 * \ingroup mrpt_hwdrivers_grp
165 */
167 {
169
170 public:
171
172 COpenNI2Sensor(); //!< Default ctor
173 ~COpenNI2Sensor(); //!< Default ctor
174
175 /** Set the serial number of the device to open.
176 * \exception This method must throw an exception when such serial number is not found among the connected devices.
177 */
178 inline void setSerialToOpen(const unsigned serial) { m_serial_number = serial; }
179
180 /** Set the sensor_id of the device to open.
181 * \exception This method must throw an exception when such serial number is not found among the connected devices.
182 */
183 inline void setSensorIDToOpen(const unsigned sensor_id) { m_user_device_number = sensor_id; }
184
185 /** Initializes the 3D camera - should be invoked after calling loadConfig() or setting the different parameters with the set*() methods.
186 * \exception This method must throw an exception with a descriptive message if some critical error is found.
187 */
188 virtual void initialize();
189
190 /** To be called at a high rate (>XX Hz), this method populates the internal buffer of received observations.
191 * This method is mainly intended for usage within rawlog-grabber or similar programs.
192 * For an alternative, see getNextObservation()
193 * \exception This method must throw an exception with a descriptive message if some critical error is found.
194 * \sa getNextObservation
195 */
196 virtual void doProcess();
197
198 /** The main data retrieving function, to be called after calling loadConfig() and initialize().
199 * \param out_obs The output retrieved observation (only if there_is_obs=true).
200 * \param there_is_obs If set to false, there was no new observation.
201 * \param hardware_error True on hardware/comms error.
202 *
203 * \sa doProcess
204 */
207 bool &there_is_obs,
208 bool &hardware_error);
209
210 /** Set the path where to save off-rawlog image files (this class DOES take into account this path).
211 * An empty string (the default value at construction) means to save images embedded in the rawlog, instead of on separate files.
212 * \exception std::exception If the directory doesn't exists and cannot be created.
213 */
214 virtual void setPathForExternalImages( const std::string &directory );
215
216
217 /** @name Sensor parameters (alternative to \a loadConfig ) and manual control
218 @{ */
219
220 /** Get the maximum range (meters) that can be read in the observation field "rangeImage" */
221 inline double getMaxRange() const { return m_maxRange; }
222
223 /** Get the row count in the camera images, loaded automatically upon camera open(). */
224 inline size_t getRowCount() const { return m_cameraParamsRGB.nrows; }
225 /** Get the col count in the camera images, loaded automatically upon camera open(). */
226 inline size_t getColCount() const { return m_cameraParamsRGB.ncols; }
227
228 /** Get a const reference to the depth camera calibration parameters */
229 inline const mrpt::utils::TCamera & getCameraParamsIntensity() const { return m_cameraParamsRGB; }
230 inline void setCameraParamsIntensity(const mrpt::utils::TCamera &p) { m_cameraParamsRGB=p; }
231
232 /** Get a const reference to the depth camera calibration parameters */
233 inline const mrpt::utils::TCamera & getCameraParamsDepth() const { return m_cameraParamsDepth; }
234 inline void setCameraParamsDepth(const mrpt::utils::TCamera &p) { m_cameraParamsDepth=p; }
235
236 /** Set the pose of the intensity camera wrt the depth camera \sa See mrpt::obs::CObservation3DRangeScan for a 3D diagram of this pose */
237 inline void setRelativePoseIntensityWrtDepth(const mrpt::poses::CPose3D &p) { m_relativePoseIntensityWRTDepth=p; }
238 inline const mrpt::poses::CPose3D &getRelativePoseIntensityWrtDepth() const { return m_relativePoseIntensityWRTDepth; }
239
240 /** Enable/disable the grabbing of the RGB channel */
241 inline void enableGrabRGB(bool enable=true) { m_grab_image=enable; }
242 inline bool isGrabRGBEnabled() const { return m_grab_image; }
243
244 /** Enable/disable the grabbing of the depth channel */
245 inline void enableGrabDepth(bool enable=true) { m_grab_depth=enable; }
246 inline bool isGrabDepthEnabled() const { return m_grab_depth; }
247
248 /** Enable/disable the grabbing of the 3D point clouds */
249 inline void enableGrab3DPoints(bool enable=true) { m_grab_3D_points=enable; }
250 inline bool isGrab3DPointsEnabled() const { return m_grab_3D_points; }
251
252 /** @} */
253
254 protected:
255
257 const mrpt::utils::CConfigFileBase &configSource,
258 const std::string &section );
259
261
262 bool m_preview_window; //!< Show preview window while grabbing
263 size_t m_preview_window_decimation; //!< If preview is enabled, only show 1 out of N images.
264 size_t m_preview_decim_counter_range, m_preview_decim_counter_rgb;
266
267 mrpt::utils::TCamera m_cameraParamsRGB; //!< Params for the RGB camera
268 mrpt::utils::TCamera m_cameraParamsDepth; //!< Params for the Depth camera
269 mrpt::poses::CPose3D m_relativePoseIntensityWRTDepth; //!< See mrpt::obs::CObservation3DRangeScan for a diagram of this pose
270
271 double m_maxRange; //!< Sensor max range (meters)
272
273 int m_user_device_number; //!< Number of device to open (0:first,...)
274 int m_serial_number; //!< Serial number of device to open
275
276 }; // End of class
277 } // End of NS
278
279} // End of NS
280
281
282
283#endif
#define DEFINE_GENERIC_SENSOR(class_name)
This declaration must be inserted in all CGenericSensor classes definition, within the class declarat...
A generic interface for a wide-variety of sensors designed to be used in the application RawLogGrabbe...
An abstract class for accessing OpenNI2 compatible sensors.
A class for grabing "range images", intensity images (either RGB or IR) and other information from an...
void setCameraParamsIntensity(const mrpt::utils::TCamera &p)
mrpt::gui::CDisplayWindowPtr m_win_int
size_t m_preview_window_decimation
If preview is enabled, only show 1 out of N images.
mrpt::poses::CPose3D m_relativePoseIntensityWRTDepth
See mrpt::obs::CObservation3DRangeScan for a diagram of this pose.
void setSerialToOpen(const unsigned serial)
Set the serial number of the device to open.
void setCameraParamsDepth(const mrpt::utils::TCamera &p)
virtual void loadConfig_sensorSpecific(const mrpt::utils::CConfigFileBase &configSource, const std::string &section)
Loads specific configuration for the device from a given source of configuration parameters,...
double getMaxRange() const
Get the maximum range (meters) that can be read in the observation field "rangeImage".
virtual void initialize()
Initializes the 3D camera - should be invoked after calling loadConfig() or setting the different par...
int m_serial_number
Serial number of device to open.
const mrpt::poses::CPose3D & getRelativePoseIntensityWrtDepth() const
double m_maxRange
Sensor max range (meters)
void enableGrab3DPoints(bool enable=true)
Enable/disable the grabbing of the 3D point clouds.
mrpt::utils::TCamera m_cameraParamsDepth
Params for the Depth camera.
const mrpt::utils::TCamera & getCameraParamsDepth() const
Get a const reference to the depth camera calibration parameters.
mrpt::utils::TCamera m_cameraParamsRGB
Params for the RGB camera.
int m_user_device_number
Number of device to open (0:first,...)
size_t getRowCount() const
Get the row count in the camera images, loaded automatically upon camera open().
void setRelativePoseIntensityWrtDepth(const mrpt::poses::CPose3D &p)
Set the pose of the intensity camera wrt the depth camera.
void setSensorIDToOpen(const unsigned sensor_id)
Set the sensor_id of the device to open.
size_t getColCount() const
Get the col count in the camera images, loaded automatically upon camera open().
void getNextObservation(mrpt::obs::CObservation3DRangeScan &out_obs, bool &there_is_obs, bool &hardware_error)
The main data retrieving function, to be called after calling loadConfig() and initialize().
void enableGrabDepth(bool enable=true)
Enable/disable the grabbing of the depth channel.
virtual void setPathForExternalImages(const std::string &directory)
Set the path where to save off-rawlog image files (this class DOES take into account this path).
bool m_preview_window
Show preview window while grabbing.
void enableGrabRGB(bool enable=true)
Enable/disable the grabbing of the RGB channel.
mrpt::poses::CPose3D m_sensorPoseOnRobot
virtual void doProcess()
To be called at a high rate (>XX Hz), this method populates the internal buffer of received observati...
const mrpt::utils::TCamera & getCameraParamsIntensity() const
Get a const reference to the depth camera calibration parameters.
Declares a class derived from "CObservation" that encapsules a 3D range scan measurement,...
A class used to store a 3D pose (a 3D translation + a rotation in 3D).
Definition: CPose3D.h:73
This class allows loading and storing values and vectors of different types from a configuration text...
Structure to hold the parameters of a pinhole camera model.
Definition: TCamera.h:32
#define HWDRIVERS_IMPEXP
struct GUI_IMPEXP CDisplayWindowPtr
This is the global namespace for all Mobile Robot Programming Toolkit (MRPT) libraries.



Page generated by Doxygen 1.9.5 for MRPT 1.4.0 SVN: at Sun Nov 27 02:47:40 UTC 2022