-
Notifications
You must be signed in to change notification settings - Fork 14
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
2 changed files
with
30 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,34 @@ | ||
# Datasets | ||
|
||
In this package you will find scripts to process or generate the datasets from the paper: | ||
|
||
- [Synthetic data generation](synthetic/) | ||
- [US Census 1990](uscensus/) | ||
|
||
## Loading and saving | ||
|
||
We work either with dense or sparse numpy arrays. The module `multi_categorical_gans.datasets.formats` presents some | ||
functions to operate with both data formats in an abstract way. | ||
|
||
## Train and test split | ||
|
||
Example of how to split a dataset into 90% train and 10% test: | ||
|
||
```bash | ||
python multi_categorical_gans/datasets/train_test_split.py \ | ||
data/uscensus/USCensus1990.features.npz \ | ||
--percent 90 \ | ||
data/uscensus/USCensus1990-train.features.npz \ | ||
data/uscensus/USCensus1990-test.features.npz | ||
``` | ||
|
||
For more information about the split run: | ||
|
||
```bash | ||
python multi_categorical_gans/datasets/train_test_split.py -h | ||
``` | ||
|
||
## The dataset wrapper | ||
|
||
The class `multi_categorical_gans.datasets.dataset.Dataset` can wrap a dense numpy array to provide simple operations | ||
for training, like `split(proportion)` (useful for validation) or `batch_iterator(batch_size, shuffle=True)`. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters