Time to complete: 5 minutes
Freedom Robotics enables low-latency teleoperation by establishing a WebRTC connection with your robot. To access these features, you’ll need to install the Freedom Robotics agent on your robot with WebRTC enabled.
WebRTC allows the lowest latency connection for driving a robot possible given your current network connection. It will use a local connection if available. Otherwise, it searches for the fastest available connection to your robot anywhere in the world.
On high-performance machines on a local network, WebRTC provides <100ms round trip latency. For real-world cellular connections, 200-400ms latencies are typical, and are limited by network connectivity, video frame frequency, and CPU load.
You'll need to (re)install the agent on your robot with WebRTC enabled. To do this:
- Select your device and navigate to SETTINGS → INSTALLATION.
- Click to show the ADVANCED options, and be sure that
- Reinstall the Agent on your robot. You’re now ready to stream camera feeds and send commands with low latency.
- Python 3.6+
- FFMPEG 3.2+
- Ubuntu 14.04+ (18.04+ recommended)
WebRTC requires the Python 3.6+ library, so even if you are running your code in Python 2.7, the install will request that you upgrade any older Python 3.x. Ubuntu <= 16.04 has Python 3.5 or below installed so a 3.6 version will be installed in parallel.
If you use an older Ubuntu version and do not have all dependencies pre-installed, the install will walk you through the process.
Older Ubuntu Versions
WebRTC relies on the latest compression, security, and other modules, so for Ubuntu <18.04 and ARM-based processors, which do not have pre-compiled Debian packages and python wheels, the initial install can take a while as it needs to compile the latest packages on the machine.
Once you have successfully installed WebRTC, when you navigate to the PILOT tab of your device and click TAKE OVER, the system will promote your connection to WebRTC.
In the PILOT view, wait for your system to connect with WebRTC. Then press
~ to display the diagnostics panel, which includes the round-trip-latency, frames per second, and other parameters.
~ shows diagnostic information such as FPS, latency and dropped frames.
The FPS for WebRTC is not tied to your SETTINGS→BANDWIDTH page. You simply need to upload a video (
sensor_msgs/Image) topic at some rate (1 Hz recommended), and when the WebRTC connection is created, it will start separately sending video (10 FPS default).
You can customize the WebRTC FPS by navigating to SETTINGS→PILOT→ENVIRONMENT→ADVANCED and selecting 2, 5, 10 or 15 FPS.
If you are running over a mesh network or a 3G cellular card, moving to a lower latency network infrastructure will help. Using a dual-sim Peplink, Cradlepoint, or other industrial cellular interface can significantly increase quality.
Many times, a few specific topics, such as navigation / planning ones, have large payloads and take up a large portion of the bandwidth. You can go to the SYSTEM page for your device and click on each topic to see its upload frequency and bandwidth usage.
If you can lower the upload frequency of certain topics without hurting performance, this will lower your overall bandwidth usage.
On lower-power machines (Nano, RPi 4, etc.), we have seen higher latencies on the same networks where high-performance systems work well. This is because WebRTC does real-time compression on the video stream which can take a lot of CPU. To improve the latency, you can lower the WebRTC FPS to 5. This often helps, but remember that sending frames at 5 FPS creates a 200 ms interval without taking latency into account.
Updated 7 months ago