Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2

# Set the OS, Python version, and other tools you might need
build:
os: ubuntu-24.04
tools:
python: "3.12"

# Build documentation in the "docs/" directory with Sphinx
sphinx:
configuration: docs/source/conf.py

# Optionally, but recommended,
# declare the Python requirements required to build your documentation
# See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
python:
install:
- requirements: docs/requirements.txt
- requirements: requirements.txt


1 change: 1 addition & 0 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
sphinx_rtd_theme
30 changes: 30 additions & 0 deletions docs/source/_static/css/custom.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
@import "theme.css";

.wy-side-nav-search {
background: #BEBEBE;
}
.wy-nav-side {
background: #F5F5F5;
}

.wy-menu-vertical p.caption{
color: #BEBEBE;
}
.wy-menu-vertical a{
color: black;
}

.wy-menu-vertical a:hover {
background-color: #BEBEBE;
color: black;
}

.wy-table-responsive table td,
.wy-table-responsive table th {
white-space: normal !important; /* Allow text to wrap */
word-wrap: break-word !important;
}

.wy-nav-content {
max-width: 1000px
}
15 changes: 2 additions & 13 deletions docs/source/comparisons.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,23 +33,12 @@ For comparative models, we use the following cumulative totals, subsequently der
* Task transfer rates: These are described in the `graph` construction in the parametric model. These are used when determining the total offline workflow runtime.


.. code-block::
:caption: A cool example

testing
testing


The output of this line starts with four spaces.

At :numref:`label` you can see

Compute values
---------------

Compute valuates


----------------

References
-----------
Expand All @@ -58,4 +47,4 @@ References

.. [2] https://github.com/ska-telescope/sdp-par-model/blob/master/notebooks/SKA1_Scheduling_New.ipynb

.. [3] Nikolic, B. (2016). SDP Memo: Estimating the SDPComputational Efficiency (Memo SKA-TEL-SDP-0000086). http://ska-sdp.org/sites/default/files/attachments/ska-tel-sdp-0000086_c_rep_sdpmemoefficiencyprototyping_-_signed.pdf
.. [3] Nikolic, B. (2016). SDP Memo: Estimating the SDPComputational Efficiency (Memo SKA-TEL-SDP-0000086)
34 changes: 22 additions & 12 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html
# http://www.sphinx-doc.org/en/master/config

# -- Path setup --------------------------------------------------------------

Expand All @@ -13,27 +13,28 @@
import os
import sys

sys.path.insert(0, os.path.abspath('../../'))
sys.path.insert(0, os.path.abspath('../../../'))

import sphinx

# -- Project information -----------------------------------------------------

project = 'TopSim Pipelines'
copyright = '2021, RW Bunney'
project = 'SKAWorkflows'
copyright = '2019-2025, RW Bunney'
author = 'RW Bunney'

master_doc = 'index'

# -- General configuration ---------------------------------------------------

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ['sphinx.ext.autodoc', 'sphinx.ext.napoleon',
'sphinx.ext.autosummary',
'sphinx.ext.viewcode']
extensions = [
'sphinx.ext.autodoc', 'sphinx.ext.napoleon', 'sphinx.ext.autosummary',
'sphinx.ext.viewcode'
]

# Add any paths that contain templates here, relative to this directory.
# Add any paths that contain templates here, relative to this sdirectory.
templates_path = ['_templates']

# List of patterns, relative to source directory, that match files and
Expand All @@ -44,15 +45,24 @@
autodoc_member_order = 'bysource'
autosummary_generate = True


# -- Options for HTML output -------------------------------------------------
# -- Options for HTML output ---------------------q----------------------------

# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'pydata_sphinx_theme'
html_theme = 'sphinx_rtd_theme'

# html_theme_options = {"full_logo":"true"}
html_theme_options = {
# "show_toc_level": True,
# "collapse_navigation": False,
# "navigation_depth": 2
}
html_logo = '' #'new_logo.svg'
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
html_css_files = ["css/custom.css"]
html_style = "css/custom.css"
html_title = 'SKAWorkflows'
16 changes: 11 additions & 5 deletions docs/source/data/origins.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
========
============
Data origins
========
============

System sizing
--------------
Expand All @@ -10,18 +10,24 @@ work is spread throughout the SDP design documentation. The following is a
list of where the information for certain values may be found.

**Average FLOP-rate**:
~~~~~~~~~~~~~~~~~~~~~~
- SKA1 SDP System Sizing, 3.1 (Table 2).

- Weighted average FLOPs are put at 13.6 PFLOPs

- SDP Memo 025; Updated SDP Cost basis of Estimate June 2016.
- Based on this document, which produces the system specification, this
is reflecting a ~20% efficiency.

- Based on this document, which produces the system specification, this is reflecting a ~20% efficiency.
- This efficiency is reported in Broekma et al 2015
- This efficiency was updated in 2016 due to further analysis (~10%).
- However, the parametric model uses adjusted 9.623 (~13% efficiency).

- SKA1 Scheduling and Archive Constraints, 3.3 pg. 13

- 10% efficiency estimations are used here.

**Total buffer size**:
Total buffer size
~~~~~~~~~~~~~~~~~~~~~~
- SKA SDP system sizing states 46PB for Low (Section 4)
- SKA Memo 025 states ~67PB (and also 60.7)
- Parametric model uses ~69PB
Expand Down
7 changes: 2 additions & 5 deletions docs/source/generating_config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,9 @@ required to produce a simulation configuration file and produces it, along
with subdirectories containing each observations' workflow file.


.. currentmodule:: pipelines.hpso_to_observation
.. currentmodule:: skaworkflows.workflow.hpso_to_observation

.. autofunction:: compile_observations_and_workflows

.. autoclass:: pipelines.hpso_to_observation.Observation
.. autoclass:: skaworkflows.workflow.hpso_to_observation.Observation
:members:

--------------------
Expand All @@ -29,5 +27,4 @@ Where is everything?
Recommended user 'flow'
-------------------------

There are two approaches one can use to generate configuration files for the

6 changes: 5 additions & 1 deletion docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,12 @@ http://github/top-sim/ organisation.

.. toctree::
:maxdepth: 2
:caption: Contents:
:caption: Contents

quickstart
data/origins
telescope/telescope_overview
generating_config
comparisons
observations/observations_overview
workflows/workflow_origins
Empty file.
23 changes: 9 additions & 14 deletions docs/source/observations/observations_overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,14 @@ In the simulation configuration, they are a sub-component of the `telescope`
key, due to the Instrument actor needing to know its observation schedule.
However, observations in the SKA workflows modules have a variety of
components to themselves, and so it is necessary to create a separate series
of pages for them.
of pages for them::

"observations": [
{
"name": "hpso4a_0",
"start": 0,
"duration": 120,
"instrument_demand": 128,
"data_product_rate": 256
},

-----
Name
-----

The name of an observation
{
"name": "hpso4a_0",
"start": 0,
"duration": 120,
"instrument_demand": 128,
"data_product_rate": 256
},
]
4 changes: 3 additions & 1 deletion docs/source/temporary.rst → docs/source/quickstart.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@

## Features necessary to produce a TopSim configuration file
Quickstart
##########


In order to produce an observation workflow with appropriate node and edge costs (tasks and data transfer, respectively), the following is required:

Expand Down
Empty file removed docs/source/requirements.rst
Empty file.
2 changes: 0 additions & 2 deletions docs/source/system_sizing.rst

This file was deleted.

18 changes: 0 additions & 18 deletions docs/source/telescope/generating_telescope_config.rst

This file was deleted.

26 changes: 13 additions & 13 deletions docs/source/telescope/telescope_overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,19 @@ focus on the workflows, and have the telescope act as a 'logical instrument'
rather than a significant component of its own. The main details of the
telescope in the characterisation of workflows, and in the configuration, is
in the maximum number arrays that are available, and the maximum ingest rate
expected from observations.

"telescope": {
"total_arrays": 36,
max_ingest_resources
"pipelines:{
"hpso4a_0": {
"workflow": "test/data/config/workflow_config.json",
"ingest_demand": 64
},
...
}
...
expected from observations::

"telescope": {
"total_arrays": 36,
max_ingest_resources
"pipelines:{
"hpso4a_0": {
"workflow": "test/data/config/workflow_config.json",
"ingest_demand": 64
},
...
}
...


Arrays
Expand Down
14 changes: 0 additions & 14 deletions docs/source/workflows/data.rst

This file was deleted.