The new ROS2TFClient + React application

[pfarinha91]

Hello!

So, following up on the recent PR #731 of adding a TF client for ROS 2 on roslibjs. I decided to try and use it in a React application that my team is working on for using Three.js (react-three-fiber) for visualizing ROS 2 data, but I've run into a issue. I will describe what's happening and provide some code snippets and we can follow up a discussion on the matter as proposed by @EzraBrooks on #731.

So, I changed the Roslibjs version on package.json to github:RobotWebTools/roslibjs#develop and installed it. Then, realized that actions are still not implemented on the main release of rosbridge for Ros Humble, so I cloned and compiled the most recent version of the rosbridge_suite (branch ros2), launched the new rosbridge_websocketand the tf2_web_republisher fork from @pac48 .

I'm now able to get TFs arriving, but something wrong is happening with the rosbridge as I stop being able to unsubscribe from other topic subscribers and can't dispose ROS2TFClient when the component unmounts and eventually WebSocketClosedError: Tried to write to a closed websocket errors start raining on the rosbridge websocket if I try to reconnect.

Maybe I'm doing something wrong, but this was my approach (sample code, some logic is not there like changing topics, etc):

import React, { useState, useEffect } from 'react';
import { Canvas } from '@react-three/fiber';
import * as ROSLIB from 'roslib';

const Component = ({ ros, topic, type, frame }) => {

  const [msg, setMsg] = useState(null);
  const [exampleSubscriber, setExampleSubscriber] = useState(null);

  const [tf, setTf] = useState(null);
  const [tfclient, setTfclient] = useState(null);
  
  useEffect(() => {
    if (!exampleSubscriber) {
      setExampleSubscriber(new ROSLIB.Topic({
        ros: ros,
        name: topic,
        messageType: type,
      }));
    }
  },[topic])

  useEffect(() => {
    exampleSubscriber?.subscribe(setMsg);
    //Unsubscibe on component unmount
    return () => {
      exampleSubscriber?.unsubscribe();
    }
  },[exampleSubscriber]);

  useEffect(() => {  
    if (!tfclient) {
      setTfclient(new ROSLIB.ROS2TFClient({
        ros : ros, 
        fixedFrame : "map"}
      ));
    } ;
  },[]);

  useEffect(() => {
    tfclient?.subscribe('base_link', setTf);
    //Unsubscibe on component unmount
    return () => {
      tfclient?.unsubscribe('base_link', setTf);
      tfclient?.dispose();
    }
  },[tfclient])

  console.log({msg,tf})

  return (
    // three.js canvas and its components
  );
};

So I launch everything:
ros2 run tf2_web_republisher tf2_web_republisher_node
ros2 launch rosbridge_server rosbridge_websocket_launch.xml
The rosbag:
ros2 bag play example_rosbag

And launch my app and connect to the rosbridge_websocket and both msg and tf start arriving:

But, I can no longer unsubscribe correctly from topic subscribers (exampleSubscriber in this case).

Before, on disconnect and component unmount, the rosbridge disconnects cleanly:

Now, with an active subscription on TF client, I try to disconnect/unmount and this happens:

The subscriber topic does not unsubscribe and on the next connection it creates new subscribers and duplicates it. I think the TFClient may actually unsubscribe (I've printed and checked the object and the frame_id is removed) but there is no feedback on the rosbridge). This only happens if I call a ROS2TFClient.subscribe(), if I don't subscribe a TF, everything is ok.

Am I doing something wrong or is the rosbridge not stable with actions yet, or something? Can someone reproduce this?

If you can make it work, please give me some insights on your approach!

• Library version: https://github.com/RobotWebTools/roslibjs (develop branch)
• Rosbridge version: https://github.com/RobotWebTools/rosbridge_suite (ros2 branch)
• tf2_web_republisher version: https://github.com/pac48/tf2_web_republisher
• ROS version: ROS Humble
• Node version: 18.15
• React version: 18.3.1
• Platform / OS: Ubuntu 22.04

Thank you :)

[EzraBrooks]

Just to help narrow down the issue and make sure we're seeing a roslibjs issue as opposed to a React lifecycle issue, I have a couple recommendations:

Instead of doing useState(null) and then using an effect to set the Topic state, try using useMemo like so:

const exampleSubscriber = useMemo(() => new ROSLIB.Topic({
  ros: ros,
  name: topic,
  messageType: type,
}), [ros, topic, type]);

You can apply a similar model to defining your ROS2TFClient.

I suspect you are actually causing extremely high-rate re-renders on your React component and things are getting confused. As a general React tip, you should never call a state setter within a high-rate callback - so if you need to have references to the "most recent message" like you have here, I would recommend using useRef instead of useState for the TF and the message.

[pfarinha91]

Thank you for your tips!

I have made the changes from useState to useMemo on the Topic and ROS2TFClient and yes it makes sense. But I couldn't make the useRef work for the message and the TF. I have a sub component that gets both:

<Component tf={tf} msg={msg}/>

And the ref update does not trigger a prop change / re-render on the sub component, but I'm not sure I've done it right. Also, I've been using the state setters for some time for Topic subscription callbacks and they work fine even with high-rates, and it is working well for the TF as well. Note that I'm getting the messages and the TF perfectly.

The problem is when I disconnect / reconnect to the rosbridge or when the component is unmounted that the rosbridge doesn't seem to be able to unsubscribe from anything, it disconnects and the subscribers are still hanging in the rosbridge even with no client connected (checked through ros2 topic info).

Note that if I don't call ROS2TFClient.subscribe() the other Topic unsubscribes perfectly and the rosbridge disconnect cleanly without hanging topics and throwing multiple WebSocketClosedError.

So, I'm guessing that I'm not calling ROS2TFClient.subscribe() and .unsubscribe() correctly.

I'm using an useEffect for that, after the useMemos:

useEffect(() => {
  exampleSubscriber?.subscribe(setMsg);
  tfClient?.subscribe(frame, setTf);
  //Unsubscibe on component unmount
  return () => {
    exampleSubscriber?.unsubscribe();
    tfClient?.unsubscribe(frame, setTf);
  }
},[]);

Yes, maybe I shouldn't use state setters, but even if I put a simple console.log() callback it also works, but the rosbridge problem persists. Are you doing the subscribe / unsubscribe the same way?

Thank you for your patience :)

[pfarinha91]

Update:

If I launch the rosbridge with send_action_goals_in_new_thread and call_services_in_new_thread as true, I no longer have problems unsubscribing the Topics, but I still have the WebSocketClosedError: Tried to write to a closed websocket errors after disconnecting.

I guess the problem is with the tf2_web_republisher from @pac48 that does not unsubscribe cleanly from the rosbridge or maybe the roslibjs unsubscribe call of ROS2TFClient is not being called correctly?

[EzraBrooks]

The ref update should not trigger a render, that is correct. I would recommend in your R3F component checking the contents of the refs in a useFrame hook instead of checking them in the render loop. This is what is advised in https://docs.pmnd.rs/react-three-fiber/advanced/pitfalls

[EzraBrooks]

but even if I put a simple console.log() callback it also works, but the rosbridge problem persists.

Okay, that's very interesting - that might be definitive evidence that the issue is in roslibjs or tf2_web_republisher. Could you create a minimum reproduction without React in the loop?