-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy patharchitecture.html
90 lines (87 loc) · 5.91 KB
/
architecture.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
<html>
<head>
<meta charset="utf-8">
<title>DIT4C architecture</title>
<link rel="canonical" href="false"/>
<meta name="viewport" content="width=device-width">
<link rel="shortcut icon" href="/images/favicon.ico" type="image/x-icon"/>
<link rel="apple-touch-icon" sizes="60x60" href="/images/apple-touch-icon-60x60.png"/>
<link rel="apple-touch-icon" sizes="76x76" href="/images/apple-touch-icon-76x76.png"/>
<link rel="apple-touch-icon" sizes="120x120" href="/images/apple-touch-icon-120x120.png"/>
<link rel="apple-touch-icon" sizes="152x152" href="/images/apple-touch-icon-152x152.png"/>
<link rel="icon" sizes="36x36" href="/images/icon-36x36.png"/>
<link rel="icon" sizes="48x48" href="/images/icon-48x48.png"/>
<link rel="icon" sizes="72x72" href="/images/icon-72x72.png"/>
<link rel="icon" sizes="96x96" href="/images/icon-96x96.png"/>
<link rel="icon" sizes="128x128" href="/images/icon-128x128.png"/>
<link rel="icon" sizes="144x144" href="/images/icon-144x144.png"/>
<link rel="icon" sizes="192x192" href="/images/icon-192x192.png"/>
<script src="/libs/webcomponentsjs/webcomponents-lite.js"></script>
<link rel="import" href="/components/base-style.html"/>
</head>
<body>
<header>
<a href="/">
<img class="img-responsive center-block"
src="/images/logo.png"
style="height: 80px; padding: 5px"/>
</a>
</header>
<article>
<h1 id="dit4c-architecture">DIT4C Architecture</h1>
<h2 id="basic-components">Basic Components</h2>
<p>DIT4C has four core components:</p>
<ul>
<li>Portal</li>
<li>Scheduler</li>
<li>Routing server (can use <a href="https://ngrok.com/">ngrok</a> in dev)</li>
<li>Compute Node</li>
</ul>
<p>This is enough to provide access to containerized tools to a user's web browser. Like <a href="https://github.com/jupyterhub/jupyterhub">Jupyter Hub</a>, but able to run more than Jupyter.</p>
<p><img src="images/diagrams/high-level-arch-basic.dot.svg" alt=""></p>
<p>The portal and routing server are the only components directly accessed by users. The scheduler and compute node can live on a private network without open ports (even a home router using NAT!).</p>
<h2 id="optional-components">Optional Components</h2>
<p>In addition, DIT4C can be run with a:</p>
<ul>
<li>Image server</li>
<li>Storage/file server (experimental)</li>
</ul>
<p>An <em>image server</em> provides a mechanism for saving resulting container images. This allows a user to checkpoint their work in case of hardware failure, share images with other users, and generally use the system for more long-term work.</p>
<p>A <em>file server</em> provides shared persistent storage between different user instances. A user can copy outputs from one research tool to storage, and then use them from another instance. It appears as a simple file-system mount under <code>/mnt</code>.</p>
<p><img src="images/diagrams/high-level-arch-extra.dot.svg" alt=""></p>
<p>Both services rely on the portal to authenticate access to them. No direct user interaction happens with them, so they don't need to be public to the internet, but they do need to be accessible from all compute nodes and the portal.</p>
<h2 id="pod-components">Pod Components</h2>
<p>Each container instance runs in a pod with a number of helper containers. This allows a compute node to avoid installing any software that needs maintenance or detailed configuration.</p>
<ul>
<li>Listener helper - listeners for traffic sent to a public routing server.</li>
<li>Auth helper - reverse proxy which authenticates traffic before sending to the user container instance. It acts as an OAuth 2 client against the portal's OAuth 2 server.</li>
<li>Storage helper - mounts persistent remote storage into the user container instance.</li>
<li>Upload helper - once the instance has terminated, this helper uploads the saved image to the image server.</li>
</ul>
<p><img src="images/diagrams/pod-components.dot.svg" alt=""></p>
<h3 id="listener-helper">Listener Helper</h3>
<p>The listener helper is responsible for exposing the app to the rest of the world. It connects to an external public routing server and forwards all incoming traffic to the app instance via the auth helper. Once connected, the listener is responsible for informing the portal of its public URL.</p>
<p><img src="images/diagrams/helper-listener.dot.svg" alt=""></p>
<h3 id="auth-helper">Auth Helper</h3>
<p>The auth helper is responsible for ensuring that traffic to the app is from an authorized source. It connects to the DIT4C portal and carries out OAuth 2 authentication of the user. Once authentication is confirmed, the auth helper provides the client with a session cookie and forwards all traffic associated with that session to the app instance.</p>
<p><img src="images/diagrams/helper-auth.dot.svg" alt=""></p>
<h3 id="storage-helper">Storage Helper</h3>
<p>The storage helper mounts remote user storage and exposes it to the app instance. The file server is responsible for authenticating the storage helper connection by looking up the published instance public keys from the portal.</p>
<p><img src="images/diagrams/helper-storage.dot.svg" alt=""></p>
<h3 id="upload-helper">Upload Helper</h3>
<p>The upload helper sends the saved app instance image to the image server. The scheduler provides a image server base URL which includes authentication details, and the image server authenticates connections by passing request headers to portal. Once complete, the upload helper is responsible for notifying the portal of the new image URL.</p>
<p><img src="images/diagrams/helper-upload.dot.svg" alt=""></p>
<h2 id="further-detail">Further Detail</h2>
<p>See "<a href="./running.html">installation guide</a>" for further details on how the components fit together.</p>
<div class="timestamp">Last updated: 2017-03-24T02:02:10Z</div>
</article>
<footer>
<p>Want to know more?</p>
<p>
<a href="https://github.com/dit4c/dit4c">
DIT4C on GitHub
</a>
</p>
</footer>
</body>
</html>