AntMaze environment - visited_blocks contains None

[mazpie]

I added a print in the rate_buffer function to check what blocks were being visited:

def rate_buffer(buffer):
    visited_blocks = [get_state_block(state) for state in buffer.states]
    print(set(visited_blocks))
    n_unique = len(set(visited_blocks))
    return n_unique

and, from the very beginning, I get a print of: {0, None} and a coverage value equal to 2.

Is this an intended behaviour?

EDIT: a few additional details.

The (x_block, y_block) for the 'None' values are ('mid','left').
Also, when the coverage raises to 3, the visited blocks are {0,None,6}, which are 'low,left', 'mid,left', 'high,left'. Thus, I suppose there is an error in the way the blocks are assigned to numbers in the get_state_block function.

If you agree on this, I can make a PR to fix it.