Merge pull request #226 from firesim/dev

Release 1.12.1

.github/linters/.flake8

@@ -0,0 +1,2 @@
+[flake8]
+ignore: E501,E121,E123,E126,E226,E24,E704,W503,W504

.github/workflows/linter.yml

@@ -0,0 +1,54 @@
+---
+#################################
+#################################
+## Super Linter GitHub Actions ##
+#################################
+#################################
+name: Lint Code Base
+
+#
+# Documentation:
+# https://help.github.com/en/articles/workflow-syntax-for-github-actions
+#
+
+#############################
+# Start the job on all push #
+#############################
+on:
+  pull_request:
+    branches: [master, dev]
+
+###############
+# Set the Job #
+###############
+jobs:
+  build:
+    # Name the Job
+    name: Lint Code Base
+    # Set the agent to run on
+    runs-on: ubuntu-latest
+
+    ##################
+    # Load all steps #
+    ##################
+    steps:
+      ##########################
+      # Checkout the code base #
+      ##########################
+      - name: Checkout Code
+        uses: actions/checkout@v2
+        with:
+          # Full git history is needed to get a proper list of changed files within `super-linter`
+          fetch-depth: 0
+
+      ################################
+      # Run Linter against code base #
+      ################################
+      - name: Lint Code Base
+        uses: docker://ghcr.io/github/super-linter:slim-v4
+        env:
+          VALIDATE_ALL_CODEBASE: false
+          VALIDATE_PYTHON_FLAKE8: true
+          FILTER_REGEX_EXCLUDE: linux busybox buildroot riscv-isa-sim
+          DEFAULT_BRANCH: master
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.gitignore

@@ -4,3 +4,4 @@ wlutil/_command.sh
 *__pycache__
 .doit.db*
 marshal-config.yaml
+.ropeproject

CHANGELOG.md

@@ -3,6 +3,28 @@
 This changelog follows the format defined here: https://keepachangelog.com/en/1.0.0/
 Versioning follows semantic versioning as described here: https://semver.org/spec/v2.0.0.html
 
+## [1.12.1] - 2022-01-21
+This maintenence release mostly fixes bugs and improves some under-the-covers
+behaviors.
+
+### Added
+
+### Fixed
+* PR #200 makes disk mounting/unmounting more reliable
+* PRs #207 and #222 change how build parallelism is handled. jLevels now
+  default to the number of available cores and buildroot uses the user-provided
+  jlevel.
+* PR #224 fixes hard-coded disk images. These seem to have stopped working
+  properly at some point. Incidentally, this also allows bare-metal workloads
+  to specify a hard-coded disk image that will be installed to FireSim but does
+  nothing in functional simulation.
+
+### Changed
+* PR #203 switches to a more recent and minimal fedora base image
+* PR #211 Changes the configuration loading process to only attempt to load
+  workloads that were actually required. Previously, FireMarshal would load all
+  configs in its path, even if it wouldn't use them.
+
 ## [1.12.0] - 2021-04-11
 This is a fairly small release that adds initial support for a chip tapeout
 prototyping board, yaml support, and a few improvements to the buildroot distro

CONTRIBUTING.md

@@ -9,7 +9,11 @@ serves as the stable release branch.
 
 Before submitting a PR:
 * Merge origin/dev to ensure you have the latest changes.
-* Run ./full_test.sh to ensure the basic unit-tests still work
+* Run ./fullTest.py to ensure the basic unit-tests still work
+* Ensure all code passes lint (this will be enforced by github). You can
+  manually check each file (or a glob) with the following command:
+
+    pylama --ignore="E501" --linters="pycodestyle,pyflakes" FILENAME.py
 
 ## Getting Help / Discussion:
 * For general questions, help, and discussion: use the FireSim user forum: https://groups.google.com/forum/#!forum/firesim

README.md

@@ -15,12 +15,18 @@ The easiest way to use Marshal is to run it via Chipyard
 FireMarshal independently, you will need the following dependencies:
 
 ## Standard Packages
-centos-requirements.txt is a list of packages for centos7 that are needed by
+``centos-requirements.txt`` is a list of packages for centos7 that are needed by
 marshal. You can install these with:
 ```
 cat centos-requirements.txt | sudo xargs yum install -y
 ```
 
+``ubuntu-requirements.txt`` is a list of packages for Ubuntu 18.04 that are needed by marshal.
+You can install these with:
+```
+cat ubuntu-requirements.txt | sudo xargs apt-get install -y
+```
+
 Package names may be different on other distributions.
 
 ### Note for Ubuntu

boards/default/distros/bare/__init__.py

@@ -1 +1 @@
-from .bare import *
+from .bare import *  # NOQA

boards/default/distros/bare/bare.py

@@ -2,47 +2,47 @@
 
 bare_dir = os.path.dirname(os.path.realpath(__file__))
 
+
 def hashOpts(opts):
     return None
 
+
 def mergeOpts(base, new):
-    return base 
+    return base
+
 
 def initOpts(cfg):
     pass
 
+
 class Builder:
     """Bare is basically a noop just to have consistency with other distros"""
     def __init__(self, opts):
         pass
 
-
     def getWorkload(self):
         return {
-                'name' : 'bare',
-                'isDistro' : True,
-                'distro' : {
-                    'name' : 'bare',
-                    'opts' : {}
+                'name': 'bare',
+                'isDistro': True,
+                'distro': {
+                    'name': 'bare',
+                    'opts': {}
                 },
-                'workdir' : bare_dir,
-                'qemu' : None,
-                'builder' : self
+                'workdir': bare_dir,
+                'qemu': None,
+                'builder': self
                 }
 
-
     def buildBaseImage(self):
         raise NotImplementedError("Baremetal workloads currently do not support disk images")
 
-
     def upToDate(self):
         """Report whether the distro is up to date or not.
 
         Trivially true because the bare-distro doesn't actually do anything
         """
         return [True]
 
-
     # Set up the image such that, when run in qemu, it will run the script "script"
     # If None is passed for script, any existing bootscript will be deleted
     @staticmethod

boards/default/distros/br/.ropeproject/config.py

@@ -0,0 +1,114 @@
+# The default ``config.py``
+# flake8: noqa
+
+
+def set_prefs(prefs):
+    """This function is called before opening the project"""
+
+    # Specify which files and folders to ignore in the project.
+    # Changes to ignored resources are not added to the history and
+    # VCSs.  Also they are not returned in `Project.get_files()`.
+    # Note that ``?`` and ``*`` match all characters but slashes.
+    # '*.pyc': matches 'test.pyc' and 'pkg/test.pyc'
+    # 'mod*.pyc': matches 'test/mod1.pyc' but not 'mod/1.pyc'
+    # '.svn': matches 'pkg/.svn' and all of its children
+    # 'build/*.o': matches 'build/lib.o' but not 'build/sub/lib.o'
+    # 'build//*.o': matches 'build/lib.o' and 'build/sub/lib.o'
+    prefs['ignored_resources'] = ['*.pyc', '*~', '.ropeproject',
+                                  '.hg', '.svn', '_svn', '.git', '.tox', 'buildroot']
+
+    # Specifies which files should be considered python files.  It is
+    # useful when you have scripts inside your project.  Only files
+    # ending with ``.py`` are considered to be python files by
+    # default.
+    # prefs['python_files'] = ['*.py']
+
+    # Custom source folders:  By default rope searches the project
+    # for finding source folders (folders that should be searched
+    # for finding modules).  You can add paths to that list.  Note
+    # that rope guesses project source folders correctly most of the
+    # time; use this if you have any problems.
+    # The folders should be relative to project root and use '/' for
+    # separating folders regardless of the platform rope is running on.
+    # 'src/my_source_folder' for instance.
+    # prefs.add('source_folders', 'src')
+
+    # You can extend python path for looking up modules
+    # prefs.add('python_path', '~/python/')
+
+    # Should rope save object information or not.
+    prefs['save_objectdb'] = True
+    prefs['compress_objectdb'] = False
+
+    # If `True`, rope analyzes each module when it is being saved.
+    prefs['automatic_soa'] = True
+    # The depth of calls to follow in static object analysis
+    prefs['soa_followed_calls'] = 0
+
+    # If `False` when running modules or unit tests "dynamic object
+    # analysis" is turned off.  This makes them much faster.
+    prefs['perform_doa'] = True
+
+    # Rope can check the validity of its object DB when running.
+    prefs['validate_objectdb'] = True
+
+    # How many undos to hold?
+    prefs['max_history_items'] = 32
+
+    # Shows whether to save history across sessions.
+    prefs['save_history'] = True
+    prefs['compress_history'] = False
+
+    # Set the number spaces used for indenting.  According to
+    # :PEP:`8`, it is best to use 4 spaces.  Since most of rope's
+    # unit-tests use 4 spaces it is more reliable, too.
+    prefs['indent_size'] = 4
+
+    # Builtin and c-extension modules that are allowed to be imported
+    # and inspected by rope.
+    prefs['extension_modules'] = []
+
+    # Add all standard c-extensions to extension_modules list.
+    prefs['import_dynload_stdmods'] = True
+
+    # If `True` modules with syntax errors are considered to be empty.
+    # The default value is `False`; When `False` syntax errors raise
+    # `rope.base.exceptions.ModuleSyntaxError` exception.
+    prefs['ignore_syntax_errors'] = False
+
+    # If `True`, rope ignores unresolvable imports.  Otherwise, they
+    # appear in the importing namespace.
+    prefs['ignore_bad_imports'] = False
+
+    # If `True`, rope will insert new module imports as
+    # `from <package> import <module>` by default.
+    prefs['prefer_module_from_imports'] = False
+
+    # If `True`, rope will transform a comma list of imports into
+    # multiple separate import statements when organizing
+    # imports.
+    prefs['split_imports'] = False
+
+    # If `True`, rope will remove all top-level import statements and
+    # reinsert them at the top of the module when making changes.
+    prefs['pull_imports_to_top'] = True
+
+    # If `True`, rope will sort imports alphabetically by module name instead
+    # of alphabetically by import statement, with from imports after normal
+    # imports.
+    prefs['sort_imports_alphabetically'] = False
+
+    # Location of implementation of
+    # rope.base.oi.type_hinting.interfaces.ITypeHintingFactory In general
+    # case, you don't have to change this value, unless you're an rope expert.
+    # Change this value to inject you own implementations of interfaces
+    # listed in module rope.base.oi.type_hinting.providers.interfaces
+    # For example, you can add you own providers for Django Models, or disable
+    # the search type-hinting in a class hierarchy, etc.
+    prefs['type_hinting_factory'] = (
+        'rope.base.oi.type_hinting.factory.default_type_hinting_factory')
+
+
+def project_opened(project):
+    """This function is called after opening the project"""
+    # Do whatever you like here!

boards/default/distros/br/__init__.py

@@ -1 +1 @@
-from .br import *
+from .br import *  # NOQA