Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support tele-op for Franka Kitchen environments ("vel" mode) #142

Open
wants to merge 5 commits into
base: dev
Choose a base branch
from
Open

Support tele-op for Franka Kitchen environments ("vel" mode) #142

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1ad7a4f
fix normalize for "vel" mode of robot
sriramsk1999 Apr 9, 2024
f256763
support teleop with "vel" mode environments
sriramsk1999 Apr 9, 2024
ccd8ede
fix qvel calculation
sriramsk1999 Apr 29, 2024
7381aa4
refactor calculation of action for vel/pos modes
sriramsk1999 May 1, 2024
32619e5
move goal site to end effector on init
sriramsk1999 May 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion robohive/envs/multi_task/common/kitchen/franka_kitchen.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
<geom type="box" group="3" pos='0 0 .142' size="0.02 0.10 0.03" contype="0" conaffinity="0" rgba=".9 .7 .95 1" euler="0 0 -.785"/>
</body>

<site name='target' pos='0 0 0' size='0.1' rgba='0 2 0 .2'/>
<site name='target' type='box' size='.03 .07 .04' pos='-0.4 0.4 2.1' group='1' rgba='0 1 .4 0' euler="0 3.14 3.14"/>
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make it a general solution

  1. we can keep the site in its default position.
  2. When teleOp is initialized, we should programmatically move this site to where the end effector is. That will avoid any initial jump in teleOp during connection to the robot.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! It's working as expected, but I'm seeing some weird behaviour from the rpFrankaRobotiqData-v0 environment:

The target site isn't placed exactly at the end effector. It's placed a little bit above + rotated compared to the actual location of the end effector. For the Franka Kitchen environment, the goal site is centered exactly on the end effector.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like an issue with conventions. Double-check where all the site frames are (both position and orientations)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like we see this issue because the end_effector site is defined for the Franka gripper, and the Robotiq gripper doesn't redefine it. Because of the grippers' different dimensions, it doesn't align perfectly for the Robotiq gripper.

  • The overall difference is fairly small (see images below) and probably doesn't matter much for teleop.
  • This issue can also be observed in the main branch and is not introduced in this PR.

franka_gripper

robotiq_gripper

<camera name='left_cam' pos='-1.4 -0.75 3' quat='0.78 0.49 -0.22 -0.32' />
<camera name='right_cam' pos='1.4 -0.75 3' quat='0.76 0.5 0.21 0.35'/>
<camera name='top_cam' pos='-.3 -.95 3.5' euler='0.785 0 0'/>
Expand Down
2 changes: 1 addition & 1 deletion robohive/robot/robot.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -574,7 +574,7 @@ def normalize_actions(self, controls, out_space='sim', unnormalize=False):
act_rng = (actuator['pos_range'][1]-actuator['pos_range'][0])/2.0
elif self._act_mode == "vel":
act_mid = (actuator['vel_range'][1]+actuator['vel_range'][0])/2.0
act_rng = (actuator['vel_range'][1]-actuator['pos_range'][0])/2.0
act_rng = (actuator['vel_range'][1]-actuator['vel_range'][0])/2.0
vikashplus marked this conversation as resolved.
Show resolved Hide resolved
else:
raise TypeError("Unknown act mode: {}".format(self._act_mode))

Expand Down
18 changes: 14 additions & 4 deletions robohive/tutorials/ee_teleop.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
- NOTE: Tutorial is written for franka arm and robotiq gripper. This demo is a tutorial, not a generic functionality for any any environment
EXAMPLE:\n
- python tutorials/ee_teleop.py -e rpFrankaRobotiqData-v0\n
- python tutorials/ee_teleop.py -e FK1_LightOnFixed-v4 --goal_site target\n
"""
# TODO: (1) Enforce pos/rot/grip limits (b) move gripper to delta commands

Expand Down Expand Up @@ -251,13 +252,22 @@ def main(env_name, env_args, reset_noise, action_noise, input_device, output, ho
if ik_result.success==False:
print(f"IK(t:{i_step}):: Status:{ik_result.success}, total steps:{ik_result.steps}, err_norm:{ik_result.err_norm}")
else:
act[:7] = ik_result.qpos[:7]
act[7:] = gripper_state
target_qpos = ik_result.qpos[:7]
if env.env.robot._act_mode == "pos":
act[:7] = target_qpos
act[7:] = gripper_state
elif env.env.robot._act_mode == "vel":
curr_qpos = env.sim.get_state()['qpos'][:7]
qvel = (target_qpos - curr_qpos) / env.dt
act[:7] = qvel
act[7:] = gripper_state
else:
raise TypeError("Unknown act mode: {}".format(env.env.robot._act_mode))

if action_noise:
act = act + env.env.np_random.uniform(high=action_noise, low=-action_noise, size=len(act)).astype(act.dtype)
if env.normalize_act:
act = env.env.robot.normalize_actions(act)

# nan actions for last log entry
act = np.nan*np.ones(env.action_space.shape) if i_step == horizon else act

Expand Down Expand Up @@ -294,4 +304,4 @@ def main(env_name, env_args, reset_noise, action_noise, input_device, output, ho


if __name__ == '__main__':
main()
main()
Loading