If you're using OpenAI Gym we will automatically log videos of your environment generated by gym.wrappers.Monitor. Just set the monitor_gym keyword argument to wandb.init to True or call wandb.gym.monitor().
Here's a small example project using the OpenAI Gym to play Snake: https://github.com/Xyzrr/rl-snake