auto_exposure_upper_limit (not for the blaze) The exposure search can be limited with an upper bound. This is to prevent very high exposure times and resulting timeouts. A typical value for this upper bound is ~2000000us. Beware that this upper limit is only set if startup_user_set is set to Default.

However, when you look at a star in the sky, these colors are reversed. The blue is on top and red on the bottom. What is going on here?

Note: The --symlink-install flag can be added to the colcon build command. This allows the installed files to be changed by changing the files in the source space (e.g., Python files or other not compiled resourced) for faster iteration (refer to the ROS2 documentation).

The GigE Vision implementation of Basler pylon software uses a thread for receiving image data. Basler pylon tries to set the thread priority for the receive thread to real-time thread priority. This requires certain permissions. The 'Permissions for Real-time Thread Priorities' section of the pylon INSTALL document describes how to grant the required permissions.

The pylon node defines the different interface names according to the following convention: [Camera name (= my_camera or my_blaze by default)]/[Node name (= pylon_ros2_camera_node)]/[Interface name] The camera and the node names can be set thanks respectively to the camera_name and node_name parameters.

camera_info_url (not for the blaze) The CameraInfo URL (Uniform Resource Locator) where the optional intrinsic camera calibration parameters are stored. This URL string will be parsed from the CameraInfoManager.

This image was over-sampled at 0.3127 arc seconds per pixel, so questions of the dispersion falling on a single pixel can be ruled out.

It turns out that it is the position of the eye or camera as it views the star's spectrum that makes the colors reversed.

frame_rate The desired publisher frame rate if listening to the topics. This parameter can only be set once at start-up. Calling the GrabImages-Action can result in a higher frame rate.

With pylon 7.5.0, if the pylon viewer does not start, this is due to a Qt dependency missing. Try installing the xcb-cursor0 library with the following command: sudo apt install libxcb-cursor0. If it does not solve your issue, try starting the pylon viewer from the bin directory of pylon: cd /opt/pylon/bin/ && ./pylonviewer. Report then the error message to the pylon support team.

camera_frame The tf2 frame under which the images were published. ROS2 provides a library called tf2 (TransForm version 2) to manage the coordinate transformations between the different frames (coordinate systems) defined by the user and assigned to the components of a robotics system.

The folder pylon_ros2_camera_wrapper/test includes different test programs. testing specific functionalities implemented by the driver. These programs are for testing purposes and should be adapted according to one's needs.

exposure_search_timeout (not for the blaze) The timeout while searching the exposure which is connected to the desired brightness. For slow system this has to be increased.

The system's maximum UDP receive buffer size should be increased to ensure a stable image acquisition. A maximum size of 2 MB is recommended. This can be achieved by issuing the sudo sysctl net.core.rmem_max=2097152 command. To make this setting persistent, you can add the net.core.rmem_max setting to the /etc/sysctl.conf file.

gige/frame_transmission_delay (not for the blaze) In most cases, this parameter should be set to 0. However, if your network hardware can't handle spikes in network traffic (e.g., if you are triggering multiple camera simultaneously), you can use the frame transmission delay parameter to stagger the start of image data transmissions from each camera.

The official pylon ROS2 driver for Basler GigE Vision, Basler USB3 Vision and Basler blaze 3D cameras (Humble Hawksbill)

Acquisition from a specific camera is possible by setting the device_user_id parameter. If no specific camera is specified, the first available camera is connected automatically.

grab_strategy (not for the blaze) Camera grab strategy: 0 = GrabStrategy_OneByOne / 1 = GrabStrategy_LatestImageOnly / 2 = GrabStrategy_LatestImages

Many GigE network adapters support so-called jumbo frames, i.e., network packets larger than the usual 1500 bytes. To enable jumbo frames, the maximum transfer unit (MTU) size of the PC's network adapter must be set to a high value. We recommend using a value of 8192.

To be sure to be able to connect to a specific camera, its network configuration must be manually set through Basler's pylon IP configurator. To do so, click on the camera in the list of connected devices, select the Static IP option, set an IP Address within the same range as the one of your computer, and set the same Subnet Mask as the one from your computer.

gain (not for the blaze) The target gain in percent of the maximal value the camera supports. For USB cameras, the gain is in dB, for GigE cameras it is given in so called 'device specific units'.

The Precision Time Protocol (PTP) camera feature allows you to synchronize multiple GigE cameras in the same network. It enables a camera to use the following features, if available:

On the Digital_Astro group on Yahoo,Howard Ritter pointed out that spectrum is dispersed about 4 arc seconds at an elevation of 33 degrees. And that the spectrum of the two stars would overlap because their separation is 1.7 arc seconds. This seems to be to me the correct explanation.

Some classes includes in their constructor the following command, commented by default: rcutils_ret_t __attribute__((unused)) res = rcutils_logging_set_logger_level(LOGGER_BASE.get_name(), RCUTILS_LOG_SEVERITY_DEBUG); Uncomment it to display on your terminal more detailed debug information.

The following discussion is about the prismatic atmospheric dispersion that is visible in the image. Some people questioned whether this is what actually caused the color, or if it was an artifact of the camera's Bayer array or an artifact of image processing.

It is easily possible to connect to a specific camera through its user id. This user id can be set through the parameter device_user_id listed in the .yaml user parameter file loaded at launch time (by default pylon_ros2_camera_wrapper/config/default.yaml). It is up to the user to create specific launch files, loading specific .yaml user parameter files, which would specify the user ids of the cameras that need to be connected. If no specific camera is specified, either because the device_user_id parameter is not set or no .yaml user parameter file is loaded, the first available camera is connected automatically.

If you hot-swap the camera with a different camera with a non-compatible pixel encoding format (e.g., mono and color cameras), you need to restart the ROS system to replace the encoding value or replace the rosparam directly by setting the image_encoding parameter. e.g.,: rosparam set /pylon_camera_node/image_encoding "mono8"

You will notice that the stars are not circular, but are tear-drop shaped. This is because different wavelengths of light form Airy disks of different sizes. The red Airy disk at bottom is larger than the blue Airy disk at top. The overlapping different sizes cause the tear-drop shape. See the article Atmospheric dispersion and its effect on the Airy pattern by Andre Paquette for a more detailed explanation.

ROS2 includes a standardised camera intrinsic calibration process through the camera_calibration package. This calibration process generates a file, which can be processed by the pylon ROS2 driver by setting the camera_info_url parameter in the pylon_ros2_camera_wrapper/config/default.yaml file (it is the user parameter file loaded by default through the driver main launch file) to the correct URI (e.g., file:///home/user/data/calibrations/my_calibration.yaml).

BaslercameraGigE

It is not possible to count correctly the number of subscribers to the image_raw and image_rect topics because of a known issue with the function CameraPublisher::getNumSubscribers. That is why this image_common package, fixing this issue, needs to be cloned and compiled together with the pylon_ros2_camera_node.

binning_x & binning_y (not for the blaze) Binning factor to get downsampled images. It refers here to any camera setting which combines rectangular neighborhoods of pixels into larger "super-pixels." It reduces the resolution of the output image to (width / binning_x) x (height / binning_y). The default values binning_x = binning_y = 0 are considered the same as binning_x = binning_y = 1 (no subsampling).

The Bayer matrix in the camera puts red, green and blue filters over individual pixels in a square with two green, one red, and one blue filter. The color information for each individual pixel in created by interpolating colors from neighboring pixels. So the color resolution is not quite as high as the luminance, which gets luminance information for each individual pixel even though it has a filter over it. In any event, the luminance and color information is pretty good for a normal subject, such as a white star. It is only for narrowband color images that resolution is lost in a Bayer image.

When light goes through something other than a vacuum, such as air, or water, or glass, it slows down. The speed of light is different in different media.

white_balance_ratio_red & white_balance_ratio_green & white_balance_ratio_blue (not for the blaze) Camera white balance ratio.

baslercameraaca2500-14gc

As observed from the Earth, the two identical mag 3.5, FO spectral class stars had an apparent separation of 1.7 arc seconds when this image was taken on March 15 of 2012. This is a tiny distance and double stars this close can only be seen in a telescope at high power. The two star's combined magnitudes give Porrima an apparent visual magnitude of 2.74.

2 : When the user calls the set_image_encoding service to use 16-bits encoding, the driver will check first for the availability of the requested 16-bits encoding to set it, when the requested 16-bits image encoding is not available, then the driver will check the availability of the equivalent 12-bits encoding to set it. When both 16-bits and 12-bits image encoding are not available then an error message will be returned.

If you are working with the pylon Viewer application, you can set the packet size by first selecting a camera from the tree in the "Device" pane. In the "Features" pane, expand the features group that shows the camera's name, expand the "Transport Layer" parameters group, and set the "Packet Size" parameter to 8192. If you write your own application, use the camera API to set the PacketSize parameter to 8192.

Porrima, Gamma Virginis, is a binary or double star comprised of two stars that closely orbit each other in a 169-year elongated elliptical orbit. Located in the constellation of Virgo, Porrima is about 38.6 light years away from the Earth.

In the ROS pylon implementation, the activate_autoflash_output and set_user_output service servers are shutdowned when the connection with a camera is lost. It is not possible for now to do so with ROS2 without shutting down the whole node (see here). There is no way to overcome this issue at the moment.

startup_user_set (not for the blaze) Flag specifying if a given user set is used when starting the camera. Can be set to Default, UserSet1, UserSet2, UserSet3, and CurrentSetting.

The ros2 launch mechanism doesn't allow to access stdin through a terminal (see here and here). This is solved in this implementation by installing and using xterm to emulate a terminal with possible user interaction.

downsampling_factor_exposure_search (not for the blaze) To speed up the exposure search, the mean brightness is not calculated on the entire image, but on a subset instead. The image is downsampled until a desired window hight is reached. The window hight is calculated out of the image height divided by the downsampling_factor_exposure search.

The pylon_ros2_camera_node can be started thanks to a dedicated launch file thanks to the command: ros2 launch pylon_ros2_camera_wrapper pylon_ros2_camera.launch.py or ros2 launch pylon_ros2_camera_wrapper my_blaze.launch.py for the blaze Several parameters can be set through the launch file and the user parameter file loaded through it (the pylon_ros2_camera_wrapper/config/default.yaml user parameter file is loaded by default, pylon_ros2_camera_wrapper/config/my_blaze.yaml for the blaze).

We can most easily see this effect with a prism, where light is broken up into its constituent colors - the colors of a rainbow - red, orange, yellow, green, blue, indigo and violet.

When light travels from one medium to another and enters or leaves at an angle, this speed change and angle of entry cause it to bend and change direction. This is called refraction. The greater the angle of incidence, the greater the angle of refraction.

Acquisition images are published through the [Camera name]/[Node name]/[image_raw] topic, only if a subscriber to this topic has been registered. To visualize the images, rqt can be used. Add an image viewer plugin through thanks to the contextual menu (Plugin -> Visualization -> Image View) and select the [Camera name]/[Node name]/[image_raw] topic to display the acquired and published images. The 3d point clouds acquired by the blaze can be visualized thanks to rviz2.

Install the ROS2 dependencies required by the pylon ROS2 packages: cd ~/dev_ws && rosdep install --from-paths src --ignore-src -r -y You may experience some problems with the diagnostic_updater and pcl_ros dependencies. In this case, install them by executing the following commands:

BaslercameraSoftware

The default trigger mode is set to software trigger. This means that the image acquisition is triggered with a certain frame rate, which may be lower than the maximum camera frame rate. The maximum camera frame rate can be reached when running a camera in a free-run or a hardware trigger mode.

gamma (not for the blaze) Gamma correction of pixel intensity. Adjusts the brightness of the pixel values output by the camera's sensor to account for a non-linearity in the human perception of brightness or of the display system (such as CRT).

Looking at the pixel matrix that makes up the stars in the image, we can see that for each star, the red-wavelength image is on the bottom and the blue is on the top. In between the two stars, where the red from the top star overlaps the blue from the bottom star, the color is more green, which is a combination of red and blue, which is exactly what we would expect.

Note: The packages are built in Release by default. The build type can be modfied by using the --cmake-args flag (for instance colcon build --symlink-install --cmake-args=-DCMAKE_BUILD_TYPE=Debug).

Starting the pylon_ros2_camera_node starts the acquisition from a given Basler camera. The nodes allow as well to access many camera parameters and parameters related to the grabbing process itself.

We see the effects of atmospheric prismatic dispersion in a telescope as a blue fringe on the top of the object, and the red fringe on the bottom, such as in the example of Porrima below.

BaslerCameraprice

If your network adapter supports jumbo frames, you set the adapter's MTU to 8192 as described above. In order to take advantage of the adapter's jumbo frame capability, you must also set the packet size used by the camera to 8192.

brightness (not for the blaze) The average intensity value of the images. It depends the exposure time as well as the gain setting. If 'exposure' is provided, the interface will try to reach the desired brightness by only varying the gain. (What may often fail, because the range of possible exposure values is many times higher than the gain range). If 'gain' is provided, the interface will try to reach the desired brightness by only varying the exposure time. If 'gain' AND 'exposure' are given, it is not possible to reach the brightness, because both are assumed to be fix.

1 : 12-bits image will be remapped to 16-bits using bit shifting to make it work with the ROS2 16-bits sensor standard message.

If the camera image acquistion is triggered by sofware trigger (default setting), then it is not possible to get the maximum frame rate, because the image acquisition is sequentially triggered, which is not overlapping then. Several possible solutions are mentionned and tested in issue #21, #28, #29, #81, #116, #147, and #200. Please refer to them for more information.

For faster USB transfers you should increase the packet size. You can do this by changing the "Stream Parameters" -> "Maximum Transfer Size" value from inside the pylon Viewer or by setting the corresponding value via the API. After increasing the package size you will likely run out of kernel space and see corresponding error messages on the console. The default value set by the kernel is 16 MB. To set the value (in this example to 1000 MB) you can execute as root: echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb This would assign a maximum of 1000 MB to the USB stack.

Different wavelengths (colors) of light bend different amounts. This is called dispersion. Shorter, more energetic, blue wavelengths bend the most. Longer red wavelengths bend the least.

In the image of Porrima originally presented, you can not see individual pixels, even though they are exactly the same as presented here. Because of this, we perceive the individual color pixels as averaged. The top star has just a few more blue pixels and the bottom star just a few more red pixels. When the color saturation was increased a bit in post processing, this exaggerated this effect a little bit. The result is that the top star is perceived as mostly blue, and the bottom star as mostly red, or actually gold in this case because the red is mixed with yellow from thick atmospheric haze. The modified camera's increased red sensitivity also comes into play here in making the red portion of the star more visible.

The Earth's atmosphere can also act as a prism when light enters the atmosphere from the near vacuum of space. The lower in elevation above the horizon a celestial object is, the greater the angle of incidence to the atmosphere, and the greater the bending of the light, and the greater the dispersion of its constituent colors.

For camera models other than the blaze, the camera-characteristic parameter such as height, width, projection matrix (by ROS2 convention, this matrix specifies the intrinsic (camera) matrix of the processed (rectified) image - see the CameraInfo message definition for detailed information) and camera_frame were published over the /camera_info topic. Furthermore, an action-based image grabbing with desired exposure time, gain, gamma and / or brightness is provided. Hence, one can grab a sequence of images with above target settings as well as a single image. Grabbing images through this action can result in a higher frame rate.

Prismatic atmospheric dispersion, however, is a widely known phenomenon, and I have seen it both visually and in images taken over the years.

gige/mtu_size (not for the blaze) The MTU size. Only used for GigE cameras. To prevent lost frames configure the camera has to be configured with the MTU size the network card supports. A value greater 3000 should be good (1500 for single-board computer)

"Lucky imaging" was used to mitigate the effects of seeing with only the sharpest video frames from the Canon 550D (T2i) full-resolution 640x480 movie crop mode being selected in Registax.

The following settings do NOT have to be set. Each camera has default values which provide an automatic image adjustment resulting in valid images.

gige/inter_pkg_delay (not for the blaze) The inter-packet delay in ticks. Only used for GigE cameras. To prevent lost frames it should be greater than 0. For most of GigE cameras, a value of 1000 is reasonable. For GigE cameras used on single-board computer, this value should be set to 11772.

In addition to being able to do so through the pylon Viewer provided by Basler, it is possible to set the device user id with the command: ros2 run pylon_ros2_camera_component set_device_user_id [-sn SERIAL_NB] your_device_user_id. If no serial number is specified thanks to the option -sn, the specified device user id your_device_user_id will be assigned to the first available camera. USB cameras must be disconnected and then reconnected after setting a new device user id. USB cameras keep their old user id otherwise.

For camera models other than the blaze, specific user set can be specified thanks to the startup_user_set parameter. ros2 launch pylon_ros2_camera_wrapper pylon_ros2_camera.launch.py startup_user_set:=Default or ros2 launch pylon_ros2_camera_wrapper pylon_ros2_camera.launch.py startup_user_set:=UserSet1 or ros2 launch pylon_ros2_camera_wrapper pylon_ros2_camera.launch.py startup_user_set:=UserSet2 or ros2 launch pylon_ros2_camera_wrapper pylon_ros2_camera.launch.py startup_user_set:=UserSet3

Baslercameraselector

Under perfect conditions, the C11 at f/10 can form a star with an Airy disk diameter of about 14.6 microns in red light at 600nm, and 11 microns at 450nm. However, many other factors influence the star size, with seeing being the most prominent, and focus, collimation, tracking, exposure length, and ISO coming into play. Seeing was bad to mediocre because of the low elevation of Porrima at the time this image was taken. Focus and collimation were good. Tracking was good because exposure length was short at 1/60th of a second. ISO was at 100 so there should have been little star spreading at this ISO.

auto_flash (not for the blaze) Flag that indicates if the camera has a flash connected, which should be on exposure. Only supported for GigE cameras. Default: false.

auto_flash_line_2 (not for the blaze) Flag that indicates if the camera has a flash connected on line 2, which should be on exposure. Only supported for GigE cameras. Default: true.

If you look really closely, you will notice that each star in the double is blue on top and red on the bottom. These colors, however, are not from the stars themselves. The blue and red effect is actually caused by the Earth's atmosphere by prismatic atmospheric dispersion. This is an effect seen in images and visually when an object is viewed or photographed at a low altitude and the light has to go through a lot of the Earth's atmosphere. The air mass acts as a prism, splitting the wavelengths of white light and diffracting the red light to the bottom of the object, and the blue light to the top. At the time this image was taken, Porrima was only about 33 degrees above the horizon.

Please Note: This project is offered with no technical support by Basler AG. You are welcome to post any questions or issues on GitHub

Baslercameradriver

If the calibration is valid, the rectified images are published through the [Camera name]/[Node name]/[image_rect] topic, only if a subscriber to this topic has been registered.

Each star image is about 8 pixels across in the original image. Each pixel in the Canon 550D is 4.3 microns. So that means each star is 8 * 4.3 = 34 microns. The difference in theoretical size vs actual size is probably due mostly to exposure and seeing effects, and the anti-aliasing filter in front of the sensor.

I have to admit that I did not understand this, even though Roger Cowley explained it to me at a meeting of the Willingboro Astronomy Society. It was not until Joe Stieber was kind enough to supply the following illustration to explain it that I finally "got it."

You might also notice that in the illustration of the prism at the top of this page, the red color is on the top and the blue color is on the bottom when they come out of the prism.

Basler VisionCamera

baslercameraaca1920-155um

Start the driver: ros2 launch pylon_ros2_camera_wrapper pylon_ros2_camera.launch.py or ros2 launch pylon_ros2_camera_wrapper my_blaze.launch.py to start the acquisition through the blaze.

shutter_mode (not for the blaze) Set mode of camera's shutter if the value is not empty. The supported modes are 'rolling', 'global' and 'global_reset'. Default value is '' (empty)

With a star diameter of 8 pixels, at 0.3127 arc seconds per pixel, each star has a diameter of about 2.5 arc seconds, a little better than the average seeing because of the lucky imaging technique employed. There are about 4 pixels between the star centers, with a diagonal of each pixel being about 6 microns. This corresponds to the separation of Porrima's components of about 1.7 arc seconds, exactly as we expect.

auto_flash_line_3 (not for the blaze) Flag that indicates if the camera has a flash connected on line 3, which should be on exposure. Only supported for GigE cameras. Default: true.

It is possible to change the packet size by changing the default value of the mtu_size parameter in the pylon ROS2 wrapper launch file. When the camera is grabbing, it is not possible to modify this parameter.

Depending on the camera model, it is possible to grab one or several images or 3d data sets (3d point cloud, intensity, confidence, depth map, depth color map) through the dedicated action with user-specified parameters (e.g., exposure time, brightness value, etc.). Refer to the action definitions to get more information.

image_encoding (not for the blaze) The encoding of the pixels -- channel meaning, ordering, size taken from the list of strings in include file sensor_msgs/image_encodings.h. The supported encodings are 'mono8', 'bgr8', 'rgb8', 'bayer_bggr8', 'bayer_gbrg8' and 'bayer_rggb8'. Default values are 'mono8' and 'rgb8'.

exposure_auto & gain_auto (not for the blaze) Only relevant, if 'brightness' is set: If the camera should try to reach and / or keep the brightness, hence adapting to changing light conditions, at least one of the following flags must be set. If both are set, the interface will use the profile that tries to keep the gain at minimum to reduce white noise. The exposure_auto flag indicates, that the desired brightness will be reached by adapting the exposure time. The gain_auto flag indicates, that the desired brightness will be reached by adapting the gain.

Beware that some parameters implemented by the driver, like for instance the parameter startup_user_set, can be set through 1. the pylon_ros2_camera_wrapper/config/default.yaml user parameter file, 2. the pylon_ros2_camera.launch.py driver launch file, and 3. the command line arguments of the launch command to start the driver. A parameter value set as an argument of the launch command to start the driver will overwrite the value set in the driver launch file itself, that will overwrite the value set in the user parameter file.

brightness_continuous (not for the blaze) Only relevant, if 'brightness' is set: The brightness_continuous flag controls the auto brightness function. If it is set to false, the brightness will only be reached once. Hence changing light conditions lead to changing brightness values. If it is set to true, the given brightness will be reached continuously, trying to adapt to changing light conditions. This is only possible for values in the possible auto range of the pylon API which is e.g., [50 - 205] for acA2500-14um and acA1920-40gm.