Docs update for PyPI refactor (#695)

.github/workflows/pr.yml

@@ -4,6 +4,9 @@ on:
   pull_request:
     branches:
       - master
+  push:
+    branches:
+      - master
 
 env:
   POETRY_VERSION: "1.3.2"

.readthedocs.yml

@@ -1,7 +1,17 @@
 version: 2
 formats: all
+
+# Set Python version & poetry tools
+build:
+  os: ubuntu-22.04
+  tools:
+    python: '3.10'
+  jobs:
+    post_install:
+      - pip install poetry myst-parser
+      - poetry config virtualenvs.create false
+      - poetry install
+
+# Build documentation with Sphinx
 sphinx:
   configuration: doc/conf.py
-python:
-   install:
-      - requirements: doc/requirements.txt

README.md

@@ -1,4 +1,8 @@
-Highly Agile Masks Made Effortlessly from RTL (HAMMER)
+![Logo design by @kenhoberkeley](doc/logo_transparent.png)
+
+![Hammer PR CI](https://github.com/ucb-bar/hammer/actions/workflows/pr.yml/badge.svg?event=push) ![Hammer publish CI](https://github.com/ucb-bar/hammer/actions/workflows/publish.yml/badge.svg)
+
+Highly Agile Masks Made Effortlessly from RTL (Hammer)
 =============================================
 
 This is the master (unstable) development branch.
@@ -8,17 +12,17 @@ Documentation
 The documentation is found at https://hammer-vlsi.readthedocs.io
 Doc sources are available at `doc/` for offline reading.
 
-Hammer currently requires Python 3.8+.
+Hammer currently requires Python 3.9+.
 
 For a deeper dive into available options and environment configuration:
 
-* [Core Hammer settings](src/hammer-vlsi/defaults.yml)
-* [Documentation for hammer-vlsi](src/hammer-vlsi/README.md)
-* [Hammer technology library schema](src/hammer-tech/schema.json)
+* [Core Hammer settings](hammer/config/defaults.yml)
+* [Setup documentation](doc/Hammer-Basics/Hammer-Setup.md)
+* [Hammer technology library schema](hammer/tech/__init__.py)
 * For CAD tool settings, please see the relevant `defaults.yml` for those plugins.
 
 Hammer is an integral component of UC Berkeley Architecture Research's [Chipyard framework](https://github.com/ucb-bar/chipyard).
-Useful documentation for how an example block is pushed through the VLSI flow with Hammer in the free ASAP7 PDK is at https://chipyard.readthedocs.io/en/latest/VLSI.
+Useful documentation for how an example block is pushed through the VLSI flow with Hammer in the free ASAP7 and Sky130 PDKs is at https://chipyard.readthedocs.io/en/latest/VLSI.
 
 History
 =======

doc/CAD-Tools/Hammer-CAD-Tools.rst

@@ -3,7 +3,7 @@
 Hammer CAD Tools
 ===============================
 
-Hammer currently has open-source CAD tool plugins in the ``hammer/src/hammer-vlsi/`` folder and three repos for CAD tools from commercial vendors: ``hammer-cadence-plugins``, ``hammer-synopsys-plugins``, and ``hammer-mentor-plugins``. ``hammer-cadence-plugins`` is a public repo but the others are private since they contain tool-specific commands not yet cleared for public release. Access to them may be granted for Hammer users who already have licenses for those tools. See the note about :ref:`plugins access <plugins-access>` for instructions for how to request access.
+Hammer currently has open-source CAD tool plugins in the ``hammer/<action>`` folders and three repos for CAD tools from commercial vendors: ``hammer-cadence-plugins``, ``hammer-synopsys-plugins``, and ``hammer-mentor-plugins``. ``hammer-mentor-plugins`` is a private repo since it contains tool-specific commands not yet cleared for public release. Access to them may be granted for Hammer users who already have licenses for those tools. See the note about :ref:`plugins access <plugins-access>` for instructions for how to request access.
 
 The structure of each repository is as follows:
 
@@ -14,7 +14,7 @@ The structure of each repository is as follows:
         * TOOL_NAME
 
             * ``__init__.py`` contains the methods needed to implement the tool
-            * ``defaults.yml`` contains the default Hammer IR needed by the tool
+            * ``defaults.yml`` contains the default :ref:`config` needed by the tool
 
 
 ACTION is the Hammer action name (e.g. ``par``, ``synthesis``, ``drc``, etc.).

doc/CAD-Tools/OpenROAD.rst

@@ -0,0 +1 @@
+.. include:: ../../hammer/par/openroad/README.md

doc/CAD-Tools/Tool-Plugin-Setup.rst

@@ -1,7 +1,7 @@
 Setting up a Hammer CAD Tool Plugin
 ================================================
 
-This guide will discuss what a Hammer user may do if they want to implement their own CAD tool plugin or extend the current CAD tool plugins. There are some basic mock-up examples of how this can be done in the ``par`` and ``synthesis`` directories inside ``hammer/src/hammer-vlsi/``.
+This guide will discuss what a Hammer user may do if they want to implement their own CAD tool plugin or extend the current CAD tool plugins. There are some basic mock-up examples of how this can be done in the ``par`` and ``synthesis`` directories inside ``hammer/``.
 
 Tool Class
 ------------------------------------------------
@@ -51,3 +51,8 @@ The name of the output JSON files will be related to the activity type (e.g. ``p
 The ``-full`` verison contains the entire Hammer IR database, while the other version contains only the output entries created by this activity.
 The individual fields are created when the ``export_config_outputs`` method is called.
 Each implementation of this tool must override this method with a new one that calls its ``super`` method and appends any additional output fields to the output dictionary, as necessary.
+
+defaults.yml
+-----------------------------------------------
+
+Each tool may optionally implement a set of default :ref:`config`. These defaults may set different values for global Hammer IR configuration, or add new tool-specific keys. Tool-specific keys must have a namespace in the format ``<action>.<tool_name>.<key_name>``. This file should have a companion ``defaults_types.yml`` that defines the required types as well.

doc/CAD-Tools/index.rst

@@ -1,16 +1,16 @@
-Hammer CAD Tool Plugins
+Hammer Tool Plugins
 ========================================
 
 This guide discusses the use and creation of CAD tool plugins in Hammer. A CAD tool plugin provides the actual implementation of Hammer APIs and outputs the TCL necessary to control its corresponding CAD tool.
 
-Hammer imports libraries as Python modules. For example, if dc was a tool, it would be either 1) a folder named "dc" with __init__.py which follows the given format; 2) a file named dc.py which follows the given format.
-
-The module should contain an class object named 'tool', since hammer-vlsi will do `import dc.tool`, for example, and use it to create an instance of the tool.
-`tool` should be a class object of an appropriate subclass of HammerTool (e.g. `HammerSynthesisTool`).
+Tool plugins must be structured as Python packages under the ``hammer.action`` package hierarchy. For example, if ``dc`` is a synthesis tool, it must be contained under ``hammer/synthesis/dc``. 
+The package should contain an class object named 'tool' to create an instance of the tool.
+``tool`` should be a class object of an appropriate subclass of HammerTool (e.g. ``HammerSynthesisTool``).
 
 .. toctree::
    :maxdepth: 2
    :caption: Hammer CAD Tools:
 
    Hammer-CAD-Tools
    Tool-Plugin-Setup
+   OpenROAD

doc/Examples/e2e.rst

@@ -0,0 +1,8 @@
+End-to-End Integration Tests
+============================
+
+In the ``e2e/`` folder, there is a example of a bare Hammer project setup with tool plugin submodules, Hammer IR snippets in the ``configs/``, ``env/``, and ``pdks/`` subfolders, and Verilog files in the ``src/`` subfolder. 
+
+``pyproject.toml`` is an example of how to install Hammer as a source dependency, and for Berkeley affiliates, the envs are particularly useful for setting up your tool paths and license servers.
+
+Refer to ``e2e/README.md`` for details of how to run the tests.

doc/Examples/index.rst

@@ -7,5 +7,5 @@ The following are prebuilt example designs, toolchain and/or flows you can use
    :maxdepth: 2
    :caption: Examples:
 
-   :caption: OpenROAD and Sky130:
+   e2e
    openroad-sky130

doc/Hammer-Basics/Hammer-Overview.rst

@@ -35,9 +35,9 @@ A techonology plugin consists of two or more files: a ``*.tech.json`` and a ``de
 
 The ``*.tech.json`` contains pointers to relevant PDK files and fundamental technology constants.  These values are not meant to be overriden, nor can they be for the time being.
 
-``defaults.yml`` sets default technology variables for Hammer to consume, which may be specific to this technology or generic to all. These values may be overriden by design-specific configurations. An example of this is shown in the open-source technology plugins in ``hammer/src/hammer-vlsi/technology/``, such as ``asap7``, and how to setup a technology plugin is documented in more detail in the :ref:`technology` section.
+``defaults.yml`` sets default technology variables for Hammer to consume, which may be specific to this technology or generic to all. These values may be overriden by design-specific configurations. An example of this is shown in the open-source technology plugins in ``hammer/technology/``, such as ``asap7``, and how to setup a technology plugin is documented in more detail in the :ref:`technology` section.
 
-.. note:: Unless you are a UCB BAR or BWRC affiliate or have set up a 3-way technology NDA with us, we cannot share pre-built technology plugin repositories.
+.. note:: Unless you are a UCB BAR or BWRC affiliate or have set up a 3-way technology NDA with us, we cannot share pre-built proprietary technology plugin repositories.
 
 Tool Plugins
 -------------------------------
@@ -46,12 +46,12 @@ A Hammer tool plugin actually implements tool-specific steps of the VLSI flow in
 The TCL commands input to the tool are created using technology and design settings provided by the designer.
 
 
-There are currently three Hammer tool plugin repositories for commercial tools: ``hammer-cadence-plugins``, ``hammer-synopsys-plugins``, and ``hammer-mentor-plugins``. In them are tool plugin implementations for actions including synthesis, place-and-route, DRC, LVS, and simulation. ``hammer-cadence-plugins`` is publicly available; however, users must request access to ``hammer-synopsys-plugins`` and ``hammer-mentor-plugins``:
-
-There are also a set of open-source tools (e.g. Yosys, OpenROAD, Magic, Netgen) provided in ``hammer/src/hammer-vlsi/`` under their respective actions.
+There are currently three Hammer tool plugin repositories for commercial tools: ``hammer-cadence-plugins``, ``hammer-synopsys-plugins``, and ``hammer-mentor-plugins``. In them are tool plugin implementations for actions including synthesis, place-and-route, DRC, LVS, and simulation. ``hammer-cadence-plugins`` and ``hammer-synopsys-plugins`` are publicly available; however, users must request access to ``hammer-mentor-plugins``.
 
 .. _plugins-access:
-.. note:: If you are not a UCB BAR or BWRC affiliate and have access to tools from a specific vendor, please email [email protected] with a request for which plugin(s) you would like access to. MAKE SURE TO INCLUDE YOUR GITHUB ID IN YOUR EMAIL AND YOUR ASSOCIATION TO SHOW YOU HAVE LICENSED ACCESS TO THOSE TOOLS. There will be no support guarantee for the plugin repositories, but users are encouraged to file issues and contribute patches where needed.
+.. note:: If you are not a UCB BAR or BWRC affiliate and have access to Mentor Graphics (now Siemens) tools, please email [email protected] with a request for access to the hammer-mentor-plugins repository. MAKE SURE TO INCLUDE YOUR GITHUB ID IN YOUR EMAIL AND YOUR ASSOCIATION TO SHOW YOU HAVE LICENSED ACCESS TO THE TOOLS. There will be no support guarantee for the plugin repositories, but users are encouraged to file issues and contribute patches where needed.
+
+There are also a set of open-source tools (e.g. Yosys, OpenROAD, Magic, Netgen) provided in ``hammer/`` under their respective actions.
 
 These plugins implement many of the common steps of a modern physical design flow. However, a real chip flow will require many custom settings and steps that may not be generalizable across technology nodes.
 Because of this, Hammer has an "escape-hatch" mechanism, called a hook, that allows the designer to inject custom steps between the default steps provided by the CAD tool plugin.
@@ -62,7 +62,7 @@ Hooks are discussed in more detail in the "Example usage" portion of the Hammer
 Calling Hammer
 -------------------------------
 
-To use Hammer on the command line, the designer will invoke the ``hammer-vlsi`` utility included in the core Hammer repo.
+To use Hammer on the command line, the designer will invoke the ``hammer-vlsi`` utility.
 This is calling the ``__main__()`` method of the ``CLIDriver`` class. An example invocation is below:
 
 .. _call-example:
@@ -83,6 +83,7 @@ These environment settings will not be propagated to the output configuration fi
 
 Any number of other YML or JSON files can then be passed in using the ``-p`` flag.
 In this case, there is only one, ``config.yml``, and it needs to set all the required keys for the step of the flow being run.
+Passing in multiple files looks like ``-p config1.yml -p config2.yml``. Refer to the :ref:`config` section for the implications of multiple config files.
 
 ``--obj_dir build`` designates what directory Hammer should use as a working directory.
 All default action run directories and output files will be placed here.
@@ -91,4 +92,4 @@ Finally, ``par`` designates that this is a place-and-route action.
 
 In this case, Hammer will write outputs to the path ``$PWD/build/par-rundir``.
 
-For the full list of Hammer command-line arguments, run `hammer-vlsi --help` or take a peek in the ``src/hammer-vlsi/hammer_vlsi/cli_driver.py`` file.
+For the full list of Hammer command-line arguments, run `hammer-vlsi --help` or take a peek in the ``hammer/vlsi/cli_driver.py`` file.

doc/Hammer-Basics/Hammer-Setup.md

@@ -6,7 +6,7 @@ The default technology, ASAP7, has some extra requirements. See its [README](htt
 
 ## User Setup
 
-You can install hammer from PyPI:
+You can install Hammer from PyPI:
 
 ```shell
 pip install hammer-vlsi
@@ -28,18 +28,19 @@ Note: certain tools and technologies will have additional system requirements. F
 
 ### Installing Hammer as a Source Dependency
 
-In some cases, it is useful to install hammer as a source dependency.
-For instance, when developing tool or PDK plugins alongside a new feature or API changes in main hammer, installing hammer as a source dependency will allow you to make changes in main hammer and see them reflected immediately when running code for your tool/PDK plugin.
+In some cases, it is useful to install Hammer as a source dependency.
+For instance, when developing tool or PDK plugins alongside a new feature or API changes in main Hammer, installing hammer as a source dependency will allow you to make changes in main hammer and see them reflected immediately when running code for your tool/PDK plugin.
 
+(poetry_project)=
 #### From Another Poetry Project
 
 Hammer tool (e.g. `hammer-cadence-plugins`) and PDK plugin repositories are poetry projects (with a `pyproject.toml` in their root).
-To depend on hammer as a source dependency, first clone hammer somewhere on your disk.
-Then add this snippet to the tool/PDK plugin repo's `pyproject.toml` (and remove any PyPI dependency on hammer):
+To depend on Hammer as a source dependency, first clone Hammer somewhere on your disk.
+Then add this snippet to the tool/PDK plugin repo's `pyproject.toml` (and remove any PyPI dependency on Hammer):
 
 ```toml
 [tool.poetry.dependencies]
-#hammer-vlsi = "^0.1.0"
+#hammer-vlsi = "^1.0.0"
 hammer-vlsi = {path = "path/to/hammer", extras = ["asap7"], develop = true}
 ```
 
@@ -49,17 +50,17 @@ You only need to specify `extras` if you need the `asap7` optional dependency (g
 
 #### From a Generic Python Project
 
-Other repos, such as Chipyard, are not poetry projects, but still depend on hammer.
-To use hammer as a source dependency:
+Other repos, such as Chipyard, are not poetry projects, but still depend on Hammer.
+To use Hammer as a source dependency:
 
 1. Remove the PyPI hammer-vlsi dependency from the project (e.g. by editing a conda env.yml file and rerunning dependency resolution)
-1. Clone hammer somewhere on your disk
+1. Clone Hammer somewhere on your disk
 1. Activate the virtualenv of the project (e.g. Chipyard)
-1. Run `pip install -e .` from the root of hammer *within the project's virtualenv*
+1. Run `pip install -e .` from the root of Hammer *within the project's virtualenv*
 
 ## Developer Setup
 
-1. Clone hammer with `git`
+1. Clone Hammer with `git`
 
 ```shell
 git clone [email protected]:ucb-bar/hammer
@@ -79,7 +80,7 @@ poetry config virtualenvs.in-project true
 poetry install
 ```
 
-4. Activate the virtualenv. Within the virtualenv, hammer is installed and you can access its scripts defined in
+4. Activate the virtualenv. Within the virtualenv, Hammer is installed and you can access its scripts defined in
     `pyproject.toml` (in `[tool.poetry.scripts]`)
 
 ```shell
@@ -94,7 +95,7 @@ You should install the [Pydantic plugin](https://plugins.jetbrains.com/plugin/12
 
 ### Unit Tests with pytest
 
-Within the poetry virtualenv, from the root of hammer, run the tests (`-v` will print out each test name explicitly)
+Within the poetry virtualenv, from the root of Hammer, run the tests (`-v` will print out each test name explicitly)
 
 ```shell
 pytest tests/ -v
@@ -136,7 +137,7 @@ There is a [small issue with the ruamel.yaml package typechecking](https://githu
 touch .venv/lib/python3.10/site-packages/ruamel/py.typed
 ```
 
-Inside your poetry virtualenv, from the root of hammer, run:
+Inside your poetry virtualenv, from the root of Hammer, run:
 
 ```shell
 mypy --namespace-packages --warn-unused-ignores -p hammer
@@ -150,7 +151,7 @@ Success: no issues found in 25 source files
 
 ### Testing Different Python Versions with tox
 
-hammer is supposed to work with Python 3.9+, so we run its unit tests on all supported Python versions using `tox` and `pyenv`.
+Hammer is supposed to work with Python 3.9+, so we run its unit tests on all supported Python versions using `tox` and `pyenv`.
 
 1. [Install `pyenv`](https://github.com/pyenv/pyenv-installer)
 
@@ -204,7 +205,7 @@ tox -e py39 -- -k "lsf" -v
 To add a new Python (`pip`) dependency, modify `pyproject.toml`.
 If the dependency is only used for development, add it under the key `[tool.poetry.dev-dependencies]`, otherwise add it under the key `[tool.poetry.dependencies]`.
 Then run `poetry update` and `poetry install`.
-The updated `poetry.lock` file should be committed to hammer.
+The updated `poetry.lock` file should be committed to Hammer.
 
 To update an existing dependency, modify `pyproject.toml` with the new version constraint.
 Run `poetry update` and `poetry install` and commit `poetry.lock`.