Skip to content

Commit

Permalink
Use cog to update --help in README, closes #82
Browse files Browse the repository at this point in the history
  • Loading branch information
simonw committed Nov 18, 2021
1 parent b686230 commit 1fce578
Show file tree
Hide file tree
Showing 3 changed files with 100 additions and 59 deletions.
142 changes: 85 additions & 57 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,62 +105,90 @@ They will be populated with IDs that reference the new derived tables.

## csvs-to-sqlite --help

Usage: csvs-to-sqlite [OPTIONS] PATHS... DBNAME

PATHS: paths to individual .csv files or to directories containing .csvs

DBNAME: name of the SQLite database file to create

Options:
-s, --separator TEXT Field separator in input .csv
-q, --quoting INTEGER Control field quoting behavior per csv.QUOTE_*
constants. Use one of QUOTE_MINIMAL (0),
QUOTE_ALL (1), QUOTE_NONNUMERIC (2) or
QUOTE_NONE (3).
--skip-errors Skip lines with too many fields instead of
stopping the import
--replace-tables Replace tables if they already exist
-t, --table TEXT Table to use (instead of using CSV filename)
-c, --extract-column TEXT One or more columns to 'extract' into a
separate lookup table. If you pass a simple
column name that column will be replaced with
integer foreign key references to a new table
of that name. You can customize the name of the
table like so:
state:States:state_name
This will pull unique values from the 'state'
column and use them to populate a new 'States'
table, with an id column primary key and a
state_name column containing the strings from
the original column.
-d, --date TEXT One or more columns to parse into ISO formatted
dates
-dt, --datetime TEXT One or more columns to parse into ISO formatted
datetimes
-df, --datetime-format TEXT One or more custom date format strings to try
when parsing dates/datetimes
-pk, --primary-key TEXT One or more columns to use as the primary key
-f, --fts TEXT One or more columns to use to populate a full-
text index
-i, --index TEXT Add index on this column (or a compound index
with -i col1,col2)
--shape TEXT Custom shape for the DB table - format is
csvcol:dbcol(TYPE),...
--filename-column TEXT Add a column with this name and populate with
CSV file name
--fixed-column <TEXT TEXT>... Populate column with a fixed string
<!-- [[[cog
import cog
from csvs_to_sqlite import cli
from click.testing import CliRunner
runner = CliRunner()
result = runner.invoke(cli.cli, ["--help"])
help = result.output.replace("Usage: cli", "Usage: csvs-to-sqlite")
cog.out(
"```\n{}\n```".format(help)
)
]]] -->
```
Usage: csvs-to-sqlite [OPTIONS] PATHS... DBNAME
PATHS: paths to individual .csv files or to directories containing .csvs
DBNAME: name of the SQLite database file to create
Options:
-s, --separator TEXT Field separator in input .csv
-q, --quoting INTEGER Control field quoting behavior per csv.QUOTE_*
constants. Use one of QUOTE_MINIMAL (0),
QUOTE_ALL (1), QUOTE_NONNUMERIC (2) or
QUOTE_NONE (3).
--skip-errors Skip lines with too many fields instead of
stopping the import
--replace-tables Replace tables if they already exist
-t, --table TEXT Table to use (instead of using CSV filename)
-c, --extract-column TEXT One or more columns to 'extract' into a
separate lookup table. If you pass a simple
column name that column will be replaced with
integer foreign key references to a new table
of that name. You can customize the name of
the table like so: state:States:state_name
This will pull unique values from the 'state'
column and use them to populate a new 'States'
table, with an id column primary key and a
state_name column containing the strings from
the original column.
-d, --date TEXT One or more columns to parse into ISO
formatted dates
-dt, --datetime TEXT One or more columns to parse into ISO
formatted datetimes
-df, --datetime-format TEXT One or more custom date format strings to try
when parsing dates/datetimes
-pk, --primary-key TEXT One or more columns to use as the primary key
-f, --fts TEXT One or more columns to use to populate a full-
text index
-i, --index TEXT Add index on this column (or a compound index
with -i col1,col2)
--shape TEXT Custom shape for the DB table - format is
csvcol:dbcol(TYPE),...
--filename-column TEXT Add a column with this name and populate with
CSV file name
--fixed-column <TEXT TEXT>... Populate column with a fixed string
--fixed-column-int <TEXT INTEGER>...
Populate column with a fixed integer
Populate column with a fixed integer
--fixed-column-float <TEXT FLOAT>...
Populate column with a fixed float
--no-index-fks Skip adding index to foreign key columns
created using --extract-column (default is to
add them)
--no-fulltext-fks Skip adding full-text index on values extracted
using --extract-column (default is to add them)
--just-strings Import all columns as text strings by default
(and, if specified, still obey --shape,
--date/datetime, and --datetime-format)

--version Show the version and exit.
--help Show this message and exit.
Populate column with a fixed float
--no-index-fks Skip adding index to foreign key columns
created using --extract-column (default is to
add them)
--no-fulltext-fks Skip adding full-text index on values
extracted using --extract-column (default is
to add them)
--just-strings Import all columns as text strings by default
(and, if specified, still obey --shape,
--date/datetime, and --datetime-format)
--version Show the version and exit.
--help Show this message and exit.
```
<!-- [[[end]]] -->
3 changes: 1 addition & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,7 @@ def get_long_description():
"py-lru-cache~=0.1.4",
"six",
],
extras_require={"test": ["pytest"]},
tests_require=["csvs-to-sqlite[test]"],
extras_require={"test": ["pytest", "cogapp"]},
entry_points="""
[console_scripts]
csvs-to-sqlite=csvs_to_sqlite.cli:cli
Expand Down
14 changes: 14 additions & 0 deletions tests/test_csvs_to_sqlite.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
from click.testing import CliRunner
from csvs_to_sqlite import cli
from six import string_types, text_type
from cogapp import Cog
import sys
from io import StringIO
import pathlib
import sqlite3

CSV = """county,precinct,office,district,party,candidate,votes
Expand Down Expand Up @@ -688,3 +692,13 @@ def test_just_strings_with_date_specified():

for name, gross, dt in actual:
assert isinstance(gross, text_type)


def test_if_cog_needs_to_be_run():
_stdout = sys.stdout
sys.stdout = StringIO()
readme = pathlib.Path(__file__).parent.parent / "README.md"
result = Cog().main(["cog", str(readme)])
output = sys.stdout.getvalue()
sys.stdout = _stdout
assert output == readme.read_text()

0 comments on commit 1fce578

Please sign in to comment.