Improve arm hand RL

[wilsonchenghy]

Latest Training Results

                      Learning iteration 610/1000                       

                       Computation: 22776 steps/s (collection: 4.229s, learning 0.087s)
             Mean action noise std: 1.92
          Mean value_function loss: 0.0010
               Mean surrogate loss: -0.0035
                 Mean entropy loss: 29.8465
                       Mean reward: 4.77
               Mean episode length: 180.00
Episode_Reward/end_effector_position_tracking: -0.0262
Episode_Reward/end_effector_position_tracking_fine_grained: 0.4237
        Episode_Reward/action_rate: -0.0005
          Episode_Reward/joint_vel: -0.0006
    Metrics/ee_pose/position_error: 0.0115
 Metrics/ee_pose/orientation_error: 2.0047
      Episode_Termination/time_out: 22.5833
--------------------------------------------------------------------------------
                   Total timesteps: 60063744
                    Iteration time: 4.32s
                      Time elapsed: 00:43:28
                               ETA: 00:27:45

[wilsonchenghy]

42% faster while with better RL performance

                      Learning iteration 201/1000                       

                       Computation: 24720 steps/s (collection: 3.886s, learning 0.090s)
             Mean action noise std: 0.58
          Mean value_function loss: 0.0023
               Mean surrogate loss: -0.0067
                 Mean entropy loss: 15.5734
                       Mean reward: 2.12
               Mean episode length: 180.00
Episode_Reward/end_effector_position_tracking: -0.0546
Episode_Reward/end_effector_position_tracking_fine_grained: 0.3049
        Episode_Reward/action_rate: -0.0000
          Episode_Reward/joint_vel: -0.0759
    Metrics/ee_pose/position_error: 0.0365
 Metrics/ee_pose/orientation_error: 1.5907
      Episode_Termination/time_out: 22.7500
--------------------------------------------------------------------------------
                   Total timesteps: 19857408
                    Iteration time: 3.98s
                      Time elapsed: 00:13:49
                               ETA: 00:54:42