Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
SLAP, PerAct baseline and Execution Code on Stretch for longitudinal …
…task execution (#198) * PointConv->PointNetConv; adding autoregressive training * fixed bugs; overfits 1traj,1waypt to mm; 1traj, 3 waypts to 1cm * time as one-hot appended to proprio; also per-actn cmd * remove time from proprio; add separate mlp * added in GRU unit; NN too simple me feels * clean-up * making network narrower and deeper * added mdn; requires tweaking * starter code * first draft of driver code + agent * drafting agent class * extending home_robot; fixing class methods * return types * bug-fixing * bug-fixing * update recorder to use new StretchClient * set position mode before moving * use velocity to control base * remove debug statement * add task-name in config * store camera pose rotated * revised interaction point detection logic * utility to collect top N episodes from each h5 * lower sleep * persistent pressing for moving base * persistent pressing for moving base * shutdown timers or robot will run away * bug-fix * tweaking vel-scale * tweaking vel-scale, again * diff scale for rot and trans * adding skills as code * 1st version of lang to execution working * changing env and detic to suit lang planner needs * added data for cmd2steps * get codelist from task cmd * small bug-fixes * fix for new client * faster motion * faster motion;slower rot * works for sequence of multiple tasks with detic object change on fly * extended support to be able to search for objects-on-receptacles * pre-commit formatting * pick, place and find all three work; loads of offset though * reverting unnecessary changes * reverting unnecessary changes, debugging * hopefully smoother execution on arm * todo's and prep for using demo-success/fail flag * reset to manip mode per episode for consistency * QoL updates * move to manip posture b4 starting episode * move to pre-demo position; diff from pre-manip * bug-fix * instructions * Slap integration and new actions * unnecessary changes * Merge * habitat-lab as in main * not sure what is going on * cleaning up old agent * SLAP flows through SM; basic YAML for task info; script for adding Detic features * renaming * state-machine works for variable # of keypoints * documentation * can read feats, passes through all methods; need avging * bringing in averaging which also considers semantic-feats * pulled in everything required to process semantic feats during training/validation on h5s * renaming and recfging for new feats/trials * APM, IPM work with semantic feats; utility scripts for H5s * do not read trials with demo_status==0 * todo and skeleton for skill-planner * start-up instructions * updating task-list * weights and gripper arch which works * testing perception for SLAP inference * instructions for env management to lessen pain * obs to IPM input pipeline works; test IPM next; then APM * goto nav mode once episode is done * inference works; can visualize actions with respect to local cropped-PCD * robustifying so you can choose trial by number; Nth from the beginning * robot-side code for closing loop on SLAP prediction for 1 skill * policy-side code for debugging + closing the loop on the robot * one-stop shop for all your H5+detic needs * try blocking motions and all that to see if obs stream makes sense * new scripts and debug * stupid bug; removing brkpoints * debug print-and-stop * eval llm task plans * modify path * remove interactive cmts; * loads of debugging to make coordinate frames work * more visualizer calls and types * can I stop farther from goal? * take out debug * debugging code, lininterpolation, adding back removed param * this version works for e2e for open-object-drawer * handover works too * last-mile changes to make experiments easier * worked for devfair env * updated installatn instructions * changes to execution code * corl-experiment weights and data-massaging * all-skills, make training and testing the same * Experiment structure * bug-fix * leftover debug * execution settings to work with SLAP assumptions * remove debug and fix labeling * training ease for tuning; peract * ignore training stuff * remove stuff with personal paths * peract-loader dbugged * quality of life; labeling bug * good weights, etc * fix mask; fix p_i logic; relabeled data * important rotations for obs from nav mod * consistency in training anf debugging * viz OK; data OK; train * remove abs path * adjustment * Experiment ready * experiment version * other 2 tasks * small adjustments for experiments * trining ready * apm specific change * per-skill experiments * adding per-skill experiments * specific to non-PnP skills * configs for peract * peract agent with end2end predictions + execution * also send gripper information * updates for rolling out with the new version * formatting? * task definition for experiment * peract debugged; better scenebounds for robot setup * include transformations from ROS; consistent quaternion * change training/inference for peract to be per waypoint * adding parameters for specific use-case; adding SLAP config * arm goes higher; handling None val for keys; adding paramaters for specific use-cases * adding details for LLM inferred language * addig parameters from main * adding QoL feature for easier experimentation; porting to new config * modifying semantic-sensor dataflow to respect new responsibilities; code for LLM inferred plans * save top 5% mask not rgb * borrowing get_obs from parent-class; modifying set_goal for specific use-case * verifying dataloaders and integrity of peract input * remove TODOs * correct bounds * remove dead code * shaping for torch tensors to make sense * removing dead code * option to filter in method rather than loader body * removing dead code * simplifying training/eval loop * executing peract * input consumption and output shaping * execution of point-to-point motion * validation on roll-out+simplification * feats is optional; logic for point-to-point motion planning * adding gripper-state as a signal * evaluation loop for PerAct agent * bounds and everything * lint fixes * documentation * docstrings, types, size is param * dead code * remove data splits and references * ignore asset yamls * adding license * cleanup * revert * remove hard-coded arm-height; document ros viz * renaming to avoid confusion * removing dead code and stale TODO/FIXME * fixing * moved saving to helper func, visualize and saving params * config documentation for inference * removing defunct params; docstring, hints in slap_agent * docstring, hints * +docstring, hints * docstring, hints; nice paths * missed out * params, defaults * docstring, hint * docstrings, deleting defunct code * more license, docstrings, corner-case safeguards * removing problematic components (license-wise) * licenses * problematic license * licenses * document everything * documentations * no more nested configs * better docstring * Script docstring * revert changes * resolving duplication * better docstring? * comment * consolidating functions; docstrings; etc * flagging, documentation * why config * documentation and proper usage * deduplication of reset; docstrings, type hints * Documentation for all scripts * helpful documentation * documenting params * comment based doc * docstring and more comments * fixed installation instructions, troubleshoot, etc * comment-based documentation * parameterize PerAct * proper documentation; merging SLAP with LangAgent * last mile changes for proper compilation and deduplication * removing duplication * resolving licenses; parameterizing preferences; comments * config to param; sim-test * bug-fix --------- Co-authored-by: Vidhi Jain <[email protected]> Co-authored-by: keke-220 <[email protected]>
- Loading branch information
3 people
authored
Jul 18, 2023
1 parent
d2e1157
commit 3b25bdf