BUG: factorize with objects differentiate 0/1 from False, True

[jbrockmendel]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

ser = pd.Series([0, 1, True, False])

>>> ser.factorize()[1]
Index([0, 1], dtype='object')

Issue Description

factorize on object dtype doesn't differentiate 0 from False or 1 from True.

This is half of the issue behind #21108.

Expected Behavior

I'd expect the uniques to include [True, False, 1, 0]

Installed Versions

Replace this line with the output of pd.show_versions()

[rhshadrach]

np.bool_(False), np.bool_(True), np.int64(0), and np.int64(1) etc. also suffer the same fate. It seems to me any solution here would have to be specialized for this (and perhaps a handful of other?) case. If two unknown objects present themselves as equal, I think pandas should treat them as such. But agree that in this special case it'd be great to make an exception if we can.

I'm having a hard time thinking of a performant way to do this though.