Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull latest main into GOAT #307

Draft
wants to merge 127 commits into
base: goat
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
127 commits
Select commit Hold shift + click to select a range
4ce19a9
add test config paths for remote eval (#287)
yvsriram Jul 17, 2023
156200c
Remove episode argument in agent reset [needs hw test] (#285)
yvsriram Jul 17, 2023
d2e1157
Refactor: Remove episode argument in agent reset (#289)
rpartsey Jul 18, 2023
3b25bdf
SLAP, PerAct baseline and Execution Code on Stretch for longitudinal …
zephirefaith Jul 18, 2023
79732fd
Offline instance mapping (#290)
yvsriram Jul 18, 2023
26f5f9c
updated docs/challenge.md (#291)
rpartsey Jul 19, 2023
e2769aa
update discrete step, rendering for eval (#292)
yvsriram Jul 19, 2023
0dc5079
Add MaskRCNN perception module (#296)
theophilegervet Jul 24, 2023
50824db
add some notes on ROS setup on the workstation (#295)
cpaxton Jul 24, 2023
7c0a4dc
Automatic install script for downloading data (#299)
cpaxton Jul 25, 2023
0b8d6bc
added receptacles data collection script (#293)
anuragajay Jul 26, 2023
8dc5cbf
Baseline agent Dockerfile update (#298)
rpartsey Jul 26, 2023
cb90554
fix a broken readme link in the docs (#302)
cpaxton Jul 26, 2023
95c2f75
Add install script for first-time users (#304)
cpaxton Jul 26, 2023
635534f
Conditionally import habitat dependencies (#303)
cpaxton Jul 27, 2023
a5732be
Fix license headers (#309)
cpaxton Jul 27, 2023
b98bd05
docs: fix link to training DD-PPO (#312)
acmiyaguchi Jul 31, 2023
c12b562
Add Grounded mobile sam for perception (#294)
yvsriram Aug 1, 2023
1ae2f48
Rewrite installation instructions (#308)
cpaxton Aug 1, 2023
e8e3425
Minor readme fixes missed in the last PR (#314)
cpaxton Aug 1, 2023
90bef70
Add specific phase commands since they are misleading (#326)
cpaxton Aug 7, 2023
6174367
Fix bug with agent evaluation (#327)
cpaxton Aug 7, 2023
4e31491
Extra note about python (#331)
cpaxton Aug 9, 2023
647f997
added ubuntu20.04 base challenge Docker image; updated challenge read…
rpartsey Aug 14, 2023
28149f5
add README for installing perception modules (#315)
yvsriram Aug 14, 2023
2972734
minor fix info is not Nnne (#337)
yvsriram Aug 15, 2023
50780f7
Add quality of life checks to make sure robots are homed and initaliz…
keke-220 Aug 16, 2023
b643698
Updates for fixing head tilt + motion in smaller kitchen (#241)
cpaxton Aug 16, 2023
7c6d099
Fix conda cuda incompatibility and upgrade to torch 2.0.1 and cuda 11.8
Aug 18, 2023
744b06f
Instance association using IoU of 3d bounding boxes (#342)
yvsriram Aug 24, 2023
74910f4
ScanNet data creation, dataset, and visualization (#343)
alexsax Aug 24, 2023
cb6c025
allow rl eval to use different discrete step (#345)
yvsriram Aug 25, 2023
aa854b6
Prototype sampling-based motion planning + 3d representations (#323)
cpaxton Aug 25, 2023
f413405
adds latest train/val episodes (#348)
yvsriram Aug 28, 2023
a7663d6
Try 2.0.0
cpaxton Aug 29, 2023
5146dca
Merge branch 'main' into torch201cu118
cpaxton Aug 29, 2023
ab978cc
Add referit3d and scanrefer to scannet dataset (#346)
alexsax Aug 30, 2023
56ccf2e
Updated category mapping in episodes (#359)
yvsriram Sep 1, 2023
76154c9
Merge branch 'main' into torch201cu118
cpaxton Sep 7, 2023
a0e4a24
branch tweaks
cpaxton Sep 7, 2023
fb2255b
cleanup env code
cpaxton Sep 7, 2023
9fcb2f5
add some deps
cpaxton Sep 7, 2023
1ddaa99
remove pytorch
cpaxton Sep 7, 2023
02645f1
fixing some issues installing branch with opencv
cpaxton Sep 7, 2023
ca25dc9
try adding pyg
cpaxton Sep 7, 2023
219b898
remove cluster install
cpaxton Sep 7, 2023
c377e5f
add pin so that we can also download boost
cpaxton Sep 7, 2023
912eb36
Add RRT motion planner which integrates into 3d world representations…
cpaxton Sep 8, 2023
3c2241a
changes for binit
cpaxton Sep 8, 2023
0c9b069
force update ovmm scenes directory (#377)
yvsriram Sep 9, 2023
95477ab
Update docker file to v0.2 (#370)
cpaxton Sep 11, 2023
2ba2761
update code
cpaxton Sep 11, 2023
bdfcca7
Merge branch 'main' into torch201cu118
cpaxton Sep 12, 2023
645a338
use minival episodes for local docker evaluations (#378)
yvsriram Sep 14, 2023
ff707f3
remove unused code
cpaxton Sep 18, 2023
6eebf7a
Add object det and referring expression det on ScanNet (#358)
alexsax Sep 19, 2023
8759cba
Merge branch 'main' of github.com:facebookresearch/home-robot into cp…
cpaxton Sep 19, 2023
ee8cb38
fix in sparse voxel map code
cpaxton Sep 19, 2023
0b74f64
Fix issues with pkl files and add geodesic distance stuff
cpaxton Sep 20, 2023
369d1ed
planner fixes
cpaxton Sep 20, 2023
87346e7
update code to switch to yield loops
cpaxton Sep 20, 2023
cd110b9
need exploration to be a bit better
cpaxton Sep 20, 2023
428943f
update voxel code and move it around a bit
cpaxton Sep 20, 2023
975e3d6
mprove 3d map setup
cpaxton Sep 20, 2023
48de5fe
explore -> run_explore
cpaxton Sep 20, 2023
b164e41
update motion planning code
cpaxton Sep 20, 2023
13c9bfc
add reasons for failure
cpaxton Sep 20, 2023
c01fd73
bug fix when no goals left
cpaxton Sep 20, 2023
09dd598
final fixes
cpaxton Sep 20, 2023
e9e85fd
fix an issue with trajectory following
cpaxton Sep 20, 2023
ff52e10
update fmm planner code
cpaxton Sep 21, 2023
db04ab1
update some things and let us make exploration a tiny bit more careful
cpaxton Sep 21, 2023
8d60180
disable verbose outputs by default
cpaxton Sep 21, 2023
ab9d89b
Merge pull request #380 from facebookresearch/cpaxton/voxel-fmm-front…
cpaxton Sep 21, 2023
9f2f333
scannet dataset can pick the number of classes to use and referring e…
alexsax Sep 22, 2023
7e2251e
Installable project that gives access to the continual stretch enviro…
SamNPowers Sep 26, 2023
23f653e
Merge branch 'main' of github.com:facebookresearch/home-robot into to…
cpaxton Sep 26, 2023
bc5655a
updates to setup
cpaxton Sep 26, 2023
90095e5
update some issues
cpaxton Sep 26, 2023
c14c5df
fix minor typos
yvsriram Sep 26, 2023
a8762d1
add back CLIP dependency
yvsriram Sep 27, 2023
61f18d8
get 2d association for IM working for OVMM, add other fixes from GOAT…
yvsriram Sep 27, 2023
d43b825
update README
cpaxton Sep 27, 2023
c6b1943
Merge branch 'cpaxton/cuda-install' of github.com:facebookresearch/ho…
cpaxton Sep 27, 2023
601debd
Merge pull request #395 from facebookresearch/cpaxton/cuda-install
cpaxton Sep 28, 2023
5f9fcca
Merge branch 'main' of https://github.com/facebookresearch/home-robot…
yvsriram Sep 29, 2023
3ce6e45
move interpolate_image to under utils file
yvsriram Sep 29, 2023
89b212f
Merge pull request #399 from facebookresearch/yvsriram/ovmm_im_fixes
yvsriram Sep 29, 2023
8c38519
correcly drop instances whose class not in subset (#401)
alexsax Oct 3, 2023
77746a7
OVMM v0.2: Upgrade habitat-lab and habitat-sim to v0.2.5 (#392)
yvsriram Oct 3, 2023
bed589f
update circleci config (#408)
cpaxton Oct 4, 2023
4b3c1e1
Fix some issues with angle interpolation (#410)
cpaxton Oct 5, 2023
1f393f3
updates hab-lab commit that fixes camera pose (#443)
yvsriram Nov 2, 2023
f724f3b
[Major update] Spot support, open-world exploration, instance mapping…
cpaxton Dec 21, 2023
9c43fa9
fix: protect against NoneError in heuristic place policy (#456)
villekuosmanen Jan 1, 2024
881059c
remove data and add a downloader script
cpaxton Jan 4, 2024
2b74b4a
update scripts to make sure we dont accidentally download all this data
cpaxton Jan 4, 2024
710499c
Update download_data.sh: update hssd-hab's branch to ovmm
yvsriram Jan 4, 2024
54d3b09
Update .gitmodules: remove entries for ovmm datasets
yvsriram Jan 4, 2024
0c9a099
Update download_data.sh
cpaxton Jan 8, 2024
c4571b5
Update download_data.sh
cpaxton Jan 8, 2024
1bec0c7
Update download_data.sh
cpaxton Jan 8, 2024
000fa10
update scripts for data downloading
cpaxton Jan 8, 2024
fbeb159
code cleanup for linter
cpaxton Jan 8, 2024
7ec07ef
remove lseg
cpaxton Jan 9, 2024
ca661d7
isort cfg
cpaxton Jan 9, 2024
0e99b26
update isort
cpaxton Jan 9, 2024
0eddd30
update cfg to skip contact graspnet server since it's a mess
cpaxton Jan 9, 2024
5456307
velocity control fixes
Jdvakil Jan 9, 2024
ded2781
reformatting manip
cpaxton Jan 11, 2024
93a6a69
reformatting and adding commment
cpaxton Jan 11, 2024
a23b25f
feat: specify detic object recognition confidence threshold in agent …
villekuosmanen Jan 11, 2024
65f5997
Merge pull request #461 from facebookresearch/anygrasp_fixes
Jdvakil Jan 11, 2024
e4149d5
Merge pull request #457 from facebookresearch/no-data
Jdvakil Jan 11, 2024
867b800
add openai client
cpaxton Jan 16, 2024
7022421
Add troubleshooting docs and fix some config issues (#463)
cpaxton Jan 18, 2024
a32a23d
Camera pose sequence length and hssd_demo fixes (#468)
yvsriram Jan 18, 2024
7033d48
update openai client
cpaxton Jan 20, 2024
71f8ad6
Merge pull request #466 from facebookresearch/cpaxton/openai-integration
Jdvakil Jan 25, 2024
cd71ecd
update troubleshooting readme (#474)
Jdvakil Feb 28, 2024
1e3caab
Update sophuspy version
yvsriram Mar 2, 2024
cbbfae0
Update sophuspy version in requirements.txt
yvsriram Mar 2, 2024
5eca396
Merge pull request #478 from yvsriram/main
Jdvakil Mar 5, 2024
5308ee4
Changes for v0.2.5 support for challenge (#488)
yvsriram Apr 29, 2024
cb6fe5a
Spot-Sim2Real submodule fix (#495)
Jdvakil May 15, 2024
825fca8
Bug Fix Pillow Denial of Service vulnerability (#487)
imhunterand May 24, 2024
ede6a67
Patch for issues (#491)
yvsriram May 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
12 changes: 10 additions & 2 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ version: 2.1

gpu: &gpu
machine:
image: ubuntu-2004-cuda-11.4:202110-01
image: linux-cuda-11:default
resource_class: gpu.nvidia.medium
environment:
FPS_THRESHOLD: 900
Expand Down Expand Up @@ -31,7 +31,12 @@ commands:
- run:
name: Setup env
command: |
[ -d ~/env_home_robot ] || mamba env create -f src/home_robot/environment.yml -p ~/env_home_robot
[ -d ~/env_home_robot ] || conda env create -f src/home_robot/environment.yml -p ~/env_home_robot
- run:
name: Add pinocchio IK dependency
command: |
conda activate ~/env_home_robot
conda install pinocchio>=2.6.17 -c conda-forge -y
- save_cache:
key: deps1-{{ .Branch }}-{{ checksum "src/home_robot/environment.yml" }}
paths:
Expand Down Expand Up @@ -67,6 +72,9 @@ jobs:
curl \
vim \
ca-certificates \
g++ \
python-dev autotools-dev libicu-dev libbz2-dev \
libboost-all-dev \
unzip || true
- conda_env_setup
- run:
Expand Down
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.ipynb filter=strip-notebook-output
5 changes: 5 additions & 0 deletions .gitconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[filter "strip-notebook-output"]
clean = "jupyter nbconvert --ClearOutputPreprocessor.enabled=True --ClearMetadataPreprocessor.enabled=True --to=notebook --stdin --stdout --log-level=ERROR"
smudge = "cat"
required

36 changes: 35 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
.DS_Store
.vscode
*.swp
*.swn
*.swo
*.code-workspace
debug/
Expand All @@ -24,12 +25,20 @@ datasets/
scene_datasets/
datadump/
*/datasets/
!/src/home_robot/home_robot/datasets
data/
/models/

# Archive files
*.pkl
*.pcd
*.zip

# Visualizations
*.ogv
*.pcd
*.PCD

# Git stuff
*~HEAD

Expand All @@ -38,13 +47,38 @@ data/
*.hdf5
*.mp4
*.glb
# used by habitat for object info
*.json
*.png
*.jpg
*.jpeg
*.npz
*.pickle
data/**
data*/**
data/***
# Mesh files
*.stl
*.STL
*.mtl
*.navmesh
*.obj
*.dae
*.DAE

# Detic model files
# Detic and other torch model files
*.pth
*.ply
*.pt

# Training stuff
projects/**/wandb
core
projects/**/multirun/**/**
projects/slap_manipulation/assets/**

# configs
projects/habitat_ovmm/configs/agent/generated/*

# Emacs
*~
20 changes: 10 additions & 10 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,19 @@
url = https://github.com/facebookresearch/habitat-lab
[submodule "src/home_robot/home_robot/agent/imagenav_agent/SuperGluePretrainedNetwork"]
path = src/home_robot/home_robot/agent/imagenav_agent/SuperGluePretrainedNetwork
url = https://github.com/magicleap/SuperGluePretrainedNetwork.git
url = https://github.com/mukulkhanna/SuperGluePretrainedNetwork.git
[submodule "src/third_party/detectron2"]
path = src/third_party/detectron2
url = https://github.com/facebookresearch/detectron2.git
[submodule "src/third_party/contact_graspnet"]
path = src/third_party/contact_graspnet
url = https://github.com/cpaxton/contact_graspnet.git
[submodule "data/datasets/ovmm"]
path = data/datasets/ovmm
url = https://huggingface.co/datasets/ai-habitat/OVMM_dataset
[submodule "data/objects"]
path = data/objects
url = https://huggingface.co/datasets/ai-habitat/OVMM_objects
[submodule "data/hssd-hab"]
path = data/hssd-hab
url = https://huggingface.co/datasets/hssd/hssd-hab
[submodule "src/third_party/MiDaS"]
path = src/third_party/MiDaS
url = [email protected]:MatthewChang/MiDaS.git
[submodule "src/home_robot/home_robot/perception/detection/grounded_sam/Grounded-Segment-Anything"]
path = src/home_robot/home_robot/perception/detection/grounded_sam/Grounded-Segment-Anything
url = https://github.com/IDEA-Research/Grounded-Segment-Anything
[submodule "src/third_party/spot-sim2real"]
path = src/third_party/spot-sim2real
url = [email protected]:facebookresearch/spot-sim2real.git
5 changes: 5 additions & 0 deletions .isort.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[settings]
py_version=39
known_first_party=home_robot,home_robot_hw,home_robot_sim,home_robot_spot
known_third_party=contact_grasp_estimator
skip=src/home_robot_hw/home_robot_hw/nodes/contact_graspnet_server.py
5 changes: 5 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,8 @@ repos:
| src/home_robot_sim/
| src/home_robot_hw/
) | setup.py$

- repo: https://github.com/kynan/nbstripout
rev: 0.6.1
hooks:
- id: nbstripout
Loading