Strike (With) A Pose

This is the companion application to the paper:

Michael A. Alcorn, Qi Li, Zhitao Gong, Chengfei Wang, Long Mai, Wei-Shinn Ku, and Anh Nguyen. Strike (with) a pose: Neural networks are easily fooled by strange poses of familiar objects. arXiv 1811.11553. 2018.

The application allows you to generate adversarial poses of objects with a graphical user interface. Please note that the included jeep object (freely available here) does not meet the realism standards set in the paper. Unfortunately, the school bus object shown in the GIF is proprietary and cannot be distributed with the application. A web port of the application (created by Zhitao Gong) can be found here.

If you use this application for your own research, please cite:

@article{alcorn-2018-strike-with-a-pose,
    Author = {Alcorn, Michael A. and Li, Qi and Gong, Zhitao and Wang, Chengfei and Mai, Long and Ku, Wei-Shinn and Nguyen, Anh},
    Title = {{Strike (with) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects}},
    Journal = {arXiv preprint},
    Year = {2018}
}

Requirements

• Git (Linux users only)
• OpenGL ≥ 3.3 (many computers satisfy this requirement)
  • On Linux, you can check your OpenGL version with the following command (requires glx-utils on Fedora or mesa-utils on Ubuntu):

  glxinfo | grep "OpenGL version"

  • On Windows, you can use this tool.
• Python 3 (Mac, Windows)
  • Mac users might also have to install SSL certificates for Python by double-clicking the file found at:

  /Applications/Python\ 3.x/Install\ Certificates.command

  where x is the minor version of your particular Python 3 install. If you used the above link to install Python, the file will be at:

  /Applications/Python\ 3.6/Install\ Certificates.command

  • Windows users should make sure they select the option to "Add Python 3.x to PATH" where "x" is the minor version of the Python 3 you are installing. If you forget to select this option, you can also add Python to the PATH yourself.

Install/Run

Note: the application takes a little while to start the first time it's run because it has to download the neural network.

Linux

In the terminal, enter the following commands:

# Clone the strike-with-a-pose repository.
git clone https://github.com/airalcorn2/strike-with-a-pose.git
# Move to the strike-with-a-pose directory.
cd strike-with-a-pose
# Install strike-with-a-pose.
pip3 install .
# Run strike-with-a-pose.
strike-with-a-pose

You can also run the application (after installing) by starting Python and entering the following:

from strike_with_a_pose import app
app.run_gui()

Mac

1. Click here to download the application ZIP.
2. Extract the ZIP somewhere convenient (like your desktop).
3. Double-click install.command in the strike-with-a-pose-master/ directory.

• Note: you may have to adjust your security settings to allow applications from "unidentified developers".

4. Double-click strike-with-a-pose.command in the strike-with-a-pose-master/run/ directory.

Windows

1. Click here to download the application ZIP.
2. Extract the ZIP somewhere convenient (like your desktop).
3. Double-click install.bat in the strike-with-a-pose-master\ directory.

• Note: you may need to click "More info" and then "Run anyway".

4. Double-click strike-with-a-pose.bat in the strike-with-a-pose-master\run\ directory.

For Experts

Users can test their own objects and backgrounds in strike-with-a-pose by (1) modifying settings.py, (2) adding the associated files to the scene_files/ directory, and (3) running the following inside the strike-with-a-pose/ directory:

PYTHONPATH=strike_with_a_pose python3 -m strike_with_a_pose.app

Pressing L will toggle Live mode. When on, the neural network will continuously generate predictions. Note, Live mode can cause considerable lag if you do not have each of (1) a powerful GPU, (2) CUDA installed, and (3) a PyTorch version with CUDA support installed.

Pressing X will toggle the object's teXture. This is useful for making sure the directional light is properly interacting with your object. If the light looks funny, swapping/negating the vertex normal coordinates can usually fix it. See the script below for an example.

input_f = "frame.obj"
output_f = "frame_new.obj"

lines = open(input_f).read().strip().split("\n")
out = open(output_f, "w")
for line in lines:
    if line[:2] == "vn":
        parts = line.split()
        # Play around with this line.
        (parts[2], parts[3]) = (parts[3], str(-float(parts[2])))
        line = " ".join(parts)

    print(line, file=out)

out.close()