Merge pull request #70 from Promptly-Technologies-LLC/28-refactor-and…

…-improve-the-documentation-website

28 refactor and improve the documentation website

.github/workflows/publish.yml

@@ -22,7 +22,7 @@ jobs:
           python-version-file: "pyproject.toml"
 
       - name: Install dependencies
-        run: uv sync --extra dev
+        run: uv sync
 
       - name: Set up Quarto
         uses: quarto-dev/quarto-actions/setup@v2

.github/workflows/release.yml

@@ -40,7 +40,7 @@ jobs:
 
       - name: Install dependencies
         run: |
-          uv sync --extra dev
+          uv sync
 
       - name: Python Semantic Release
         id: release
@@ -52,7 +52,7 @@ jobs:
       - name: Build project wheel and test that it installs without errors
         run: |
           uv build
-          uv sync --extra dev
+          uv sync
           uv pip install dist/imfp-*.whl
 
       - name: Run tests

.github/workflows/test.yml

@@ -29,7 +29,7 @@ jobs:
 
       - name: Install dependencies and test that documentation renders and tests pass
         run: |
-          uv sync --extra dev
+          uv sync
           uv run quarto render
           uv run pytest tests
     outputs:

README.md

@@ -49,11 +49,11 @@ We welcome contributions to improve `imfp`! Here's how you can help:
 2. To fix a bug:
    - Fork and clone the repository and open a terminal in the repository directory
    - Install [uv](https://astral.sh/setup-uv/) with `curl -LsSf https://astral.sh/uv/install.sh | sh`
-   - Install the dependencies with `uv sync --extra dev`
+   - Install the dependencies with `uv sync`
    - Install a git hook to enforce conventional commits with `curl -o- https://raw.githubusercontent.com/tapsellorg/conventional-commits-git-hook/master/scripts/install.sh | sh`
    - Create a fix, commit it with an ["Angular-style Conventional Commit"](https://www.conventionalcommits.org/en/v1.0.0-beta.4/) message, and push it to your fork
    - Open a pull request to our `main` branch
 
-Note that if you want to change and/or render the documentation, you will need to install the [Quarto CLI tool](https://quarto.org/docs/download/).
+Note that if you want to change and preview the documentation, you will need to install the [Quarto CLI tool](https://quarto.org/docs/download/).
 
 Version incrementing, package building, testing, changelog generation, documentation rendering, publishing to PyPI, and Github release creation is handled automatically by the GitHub Actions workflow based on the commit messages.

_quarto.yml

@@ -1,15 +1,31 @@
 project:
   type: website
   output-dir: _site
+  render:
+    - "*.qmd"
+    - "!changelog.md"
 
 website:
   title: "imfp"
+  search: true
   navbar:
     left:
-      - href: index.qmd
-        text: Home
-      - href: docs/demo.qmd
-        text: Demo
+      - text: Quickstart
+        href: index.qmd
+      - text: Installation
+        href: docs/installation.qmd
+      - text: Databases
+        href: docs/databases.qmd
+      - text: Parameters
+        href: docs/parameters.qmd
+      - text: Datasets
+        href: docs/datasets.qmd
+      - text: Usage
+        href: docs/usage.qmd
+      - text: Rate Limits
+        href: docs/rate_limits.qmd
+      - text: Demo
+        href: docs/demo.qmd
     right:
       - icon: github
         href: https://github.com/Promptly-Technologies-LLC/imfp
@@ -22,4 +38,4 @@ website:
 format:
   html:
     theme: cosmo
-    toc: true
+    toc: true

docs/databases.qmd

@@ -0,0 +1,68 @@
+---
+title: "Working with Databases"
+---
+
+## Understanding IMF Databases
+
+The IMF serves many different databases through its API, and the API needs to know which of these many databases you're requesting data from. Before you can fetch any data, you'll need to:
+
+1. Get a list of available databases
+2. Find the database ID for the data you want
+
+Then you can use that database ID to fetch the data.
+
+## Fetching the Database List
+
+### Fetching an Index of Databases with the `imf_databases` Function
+
+To obtain the list of available databases and their corresponding IDs, use `imf_databases`:
+
+``` {python}
+import imfp
+
+#Fetch the list of databases available through the IMF API
+databases = imfp.imf_databases()
+databases.head()
+```
+
+
+This function returns the IMF’s listing of 259 databases available through the API. (In reality, a few of the listed databases are defunct and not actually available. The databases FAS_2015, GFS01, FM202010, APDREO202010, AFRREO202010, WHDREO202010, BOPAGG_2020, and DOT_2020Q1 were unavailable as of last check.)
+
+## Exploring the Database List
+
+To view and explore the database list, it’s possible to explore subsets of the data frame by row number with `databases.loc`:
+
+``` {python}
+# View a subset consisting of rows 5 through 9
+databases.loc[5:9]
+```
+
+
+Or, if you already know which database you want, you can fetch the corresponding code by searching for a string match using `str.contains` and subsetting the data frame for matching rows. For instance, here’s how to search for commodities data:
+
+``` {python}
+databases[databases['description'].str.contains("Commodity")]
+```
+
+See also [Working with Large Data Frames](usage.qmd#working-with-large-data-frames) for sample code showing how to view the full contents of the data frame in a browser window. 
+
+## Best Practices
+
+1. **Cache the Database List**: The database list rarely changes. Consider saving it locally if you'll be making multiple queries. See [Caching Strategy](rate_limits.qmd#caching-strategy) for sample code.
+
+2. **Search Strategically**: Use specific search terms to find relevant databases. For example:
+
+   - "Price" for price indices
+   - "Trade" for trade statistics
+   - "Financial" for financial data
+
+3. **Use a Browser Viewer**: See [Working with Large Data Frames](usage.qmd#working-with-large-data-frames) for sample code showing how to view the full contents of the data frame in a browser window.
+
+4. **Note Database IDs**: Once you find a database you'll use frequently, note its database ID for future reference.
+
+## Next Steps
+
+Once you've identified the database you want to use, you'll need to:
+
+1. Get the list of parameters for that database (see [Parameters](parameters.qmd))
+2. Use those parameters to fetch your data (see [Datasets](datasets.qmd))

docs/datasets.qmd

@@ -0,0 +1,80 @@
+---
+title: "Requesting Datasets"
+---
+
+## Making a Request
+
+To retrieve data from an IMF database, you'll need the database ID and any relevant [filter parameters](parameters.qmd). Here's a basic example using the Primary Commodity Price System (PCPS) database:
+
+``` {python}
+import imfp
+
+# Get parameters and their valid codes
+params = imfp.imf_parameters("PCPS")
+
+# Fetch annual coal price index data
+df = imfp.imf_dataset(
+    database_id="PCPS",
+    freq=["A"],  # Annual frequency
+    commodity=["PCOAL"],  # Coal prices
+    unit_measure=["IX"],  # Index
+    start_year=2000,
+    end_year=2015
+)
+```
+
+This example creates two objects we'll use in the following sections:
+
+- `params`: A dictionary of parameters and their valid codes
+- `df`: The retrieved data frame containing our requested data
+
+## Decoding Returned Data
+
+When you retrieve data using `imf_dataset`, the returned data frame contains columns that correspond to the parameters you specified in your request. However, these columns use input codes (short identifiers) rather than human-readable descriptions. To make your data more interpretable, you can replace these codes with their corresponding text descriptions using the parameter information from `imf_parameters`, so that codes like "A" (Annual) or "W00" (World) become self-explanatory labels.
+
+For example, suppose we want to decode the `freq` (frequency), `ref_area` (geographical area), and `unit_measure` (unit) columns in our dataset. We'll merge the parameter descriptions into our data frame:
+
+``` {python}
+# Decode frequency codes (e.g., "A" → "Annual")
+df = df.merge(
+    # Select code-description pairs
+    params['freq'][['input_code', 'description']],
+    # Match codes in the data frame
+    left_on='freq',
+    # ...to codes in the parameter data
+    right_on='input_code',
+    # Keep all data rows
+    how='left'
+).drop(columns=['freq', 'input_code']
+).rename(columns={"description": "freq"})
+
+# Decode geographic area codes (e.g., "W00" → "World")
+df = df.merge(
+    params['ref_area'][['input_code', 'description']],
+    left_on='ref_area',
+    right_on='input_code',
+    how='left'
+).drop(columns=['ref_area', 'input_code']
+).rename(columns={"description":"ref_area"})
+
+# Decode unit codes (e.g., "IX" → "Index")
+df = df.merge(
+    params['unit_measure'][['input_code', 'description']],
+    left_on='unit_measure',
+    right_on='input_code',
+    how='left'
+).drop(columns=['unit_measure', 'input_code']
+).rename(columns={"description":"unit_measure"})
+
+df.head()
+```
+
+After decoding, the data frame is much more human-interpretable. This transformation makes the data more accessible for analysis and presentation, while maintaining all the original information.
+
+## Understanding the Data Frame
+
+Also note that the returned data frame has additional mysterious-looking codes as values in some columns.
+
+Codes in the `time_format` column are ISO 8601 duration codes. In this case, “P1Y” means “periods of 1 year.” See [Time Period Conversion](usage.qmd#time-period-conversion) for more information on reconciling time periods.
+
+The `unit_mult` column represents the number of zeroes you should add to the value column. For instance, if value is in millions, then the unit multiplier will be 6. If in billions, then the unit multiplier will be 9. See [Unit Multiplier Adjustment](usage.qmd#unit-multiplier-adjustment) for more information on reconciling units.

docs/demo.qmd

@@ -31,7 +31,7 @@ In this project, we explored the following:
 * Regression Analysis
 * Time Series Analysis
 
-## Suggested packages
+## Utility Functions
 The integration of other Python tools not only streamlined our computational processes but also ensured consistency across the project.
 
 A custom module is written to simplify the process of making API calls and fetching information with imfp library. `load_or_fetch_databases`, `load_or_fetch_parameters` `load_or_fetch_dataset` load and retreive database, parameters, and dataset from a local or remote source. `view_dataframe_in_browser` displays dataframe in a web browser.
@@ -124,6 +124,7 @@ def load_or_fetch_dataset(database_id, indicator):
 
 ```
 
+## Dependencies
 Here is a brief introduction about the packages used:
 
 `pandas`: view and manipulate data frame
@@ -654,6 +655,7 @@ model.summary()
 Time series analysis allows us to explore how the relationship between GII and GDP change vary across different time periods, accounting for lagged effects. 
 
 Here was a quick summary of the result:
+
 * Both GII and GDP change time series were stationary.
 
 * Past GII change values significantly influenced cuurent GII change values.

docs/installation.qmd

@@ -0,0 +1,59 @@
+---
+title: "Installation"
+---
+
+## Prerequisites
+
+To install the latest version of `imfp`, you will need to have [Python 3.10 or later](https://www.python.org/downloads/) installed on your system.
+
+If you don't already have Python, we recommend installing [the `uv` package manager](https://astral.sh/setup-uv/) and installing Python with `uv python install`.
+
+## Installation
+
+To install the latest stable `imfp` wheel from PyPi using pip:
+
+``` bash
+pip install --upgrade imfp
+```
+
+Alternatively, to install from the source code on Github, you can use the following command:
+
+``` bash
+pip install --upgrade git+https://github.com/Promptly-Technologies-LLC/imfp.git
+```
+
+You can then import the package in your Python script:
+
+``` python
+import imfp
+```
+
+## Suggested Dependencies for Data Analysis
+
+`imfp` outputs data in a `pandas` data frame, so you will want to use the `pandas` package (which is installed with `imfp`) for its functions for viewing and manipulating this object type. For data visualization, we recommend installing these additional packages:
+
+``` bash
+pip install -q matplotlib seaborn
+```
+
+You can then import these packages in your Python script:
+
+``` python
+import pandas as pd
+import matplotlib.pyplot as plt
+import seaborn as sns
+```
+
+## Development Installation
+
+To get started with development of `imfp`,
+
+1. Fork and clone the repository
+2. Install [uv](https://astral.sh/setup-uv/) with `curl -LsSf https://astral.sh/uv/install.sh | sh`
+3. Install the dependencies with `uv sync`
+4. Install a git pre-commit hook to enforce conventional commits:
+   ``` bash
+   curl -o- https://raw.githubusercontent.com/tapsellorg/conventional-commits-git-hook/master/scripts/install.sh | sh
+   ```
+
+To edit and preview the documentation, you'll also want to install the [Quarto CLI tool](https://quarto.org/docs/download/).