-
Notifications
You must be signed in to change notification settings - Fork 29
/
task_execution_service.openapi.yaml
918 lines (870 loc) · 33.3 KB
/
task_execution_service.openapi.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
openapi: 3.0.1
info:
title: Task Execution Service
version: 1.1.0
x-logo:
url: 'https://www.ga4gh.org/wp-content/themes/ga4gh/dist/assets/svg/logos/logo-full-color.svg'
license:
name: Apache 2.0
url: 'https://raw.githubusercontent.com/ga4gh/task-execution-schemas/develop/LICENSE'
description: >
## Executive Summary
The Task Execution Service (TES) API is a standardized schema and API for
describing and executing batch execution tasks. A task defines a set of
input files, a set of containers and commands to run, a set of
output files and some other logging and metadata.
TES servers accept task documents and execute them asynchronously on
available compute resources. A TES server could be built on top of
a traditional HPC queuing system,
such as Grid Engine, Slurm or cloud style compute systems such as AWS Batch
or Kubernetes.
## Introduction
This document describes the TES API and provides details on the specific
endpoints, request formats, and responses. It is intended to provide key
information for developers of TES-compatible services as well as clients
that will call these TES services. Use cases include:
- Deploying existing workflow engines on new infrastructure. Workflow engines
such as CWL-Tes and Cromwell have extentions for using TES. This will allow
a system engineer to deploy them onto a new infrastructure using a job scheduling
system not previously supported by the engine.
- Developing a custom workflow management system. This API provides a common
interface to asynchronous batch processing capabilities. A developer can write
new tools against this interface and expect them to work using a variety of
backend solutions that all support the same specification.
## Standards
The TES API specification is written in OpenAPI and embodies a RESTful service
philosophy. It uses JSON in requests and responses and standard
HTTP/HTTPS for information transport. HTTPS should be used rather than plain HTTP
except for testing or internal-only purposes.
### Authentication and Authorization
Is is envisaged that most TES API instances will require users to authenticate to use the endpoints.
However, the decision if authentication is required should be taken by TES API implementers.
If authentication is required, we recommend that TES implementations use an OAuth2 bearer token, although they can choose other mechanisms if appropriate.
Checking that a user is authorized to submit TES requests is a responsibility of TES implementations.
### CORS
If TES API implementation is to be used by another website or domain it must implement Cross Origin Resource Sharing (CORS).
Please refer to https://w3id.org/ga4gh/product-approval-support/cors for more information about GA4GH’s recommendations and how to implement CORS.
security:
# Optionally, this section can be removed or adjusted by implementers based on their needs.
- BearerAuth: [] # Apply globally if authentication is required
- BasicAuth: [] # Alternatively, use Basic Authentication
servers:
- url: /ga4gh/tes/v1
paths:
/service-info:
get:
tags:
- TaskService
summary: GetServiceInfo
description: |-
Provides information about the service, this structure is based on the
standardized GA4GH service info structure. In addition, this endpoint
will also provide information about customized storage endpoints offered
by the TES server.
operationId: GetServiceInfo
responses:
200:
description: ""
content:
application/json:
schema:
$ref: '#/components/schemas/tesServiceInfo'
security:
- BearerAuth: [] # Endpoint secured with Bearer token
- BasicAuth: [] # Alternatively, secured with Basic Auth
/tasks:
get:
tags:
- TaskService
summary: ListTasks
description: |-
List tasks tracked by the TES server. This includes queued, active and completed tasks.
How long completed tasks are stored by the system may be dependent on the underlying
implementation.
operationId: ListTasks
parameters:
- name: name_prefix
in: query
description: |-
OPTIONAL. Filter the list to include tasks where the name matches this prefix.
If unspecified, no task name filtering is done.
schema:
type: string
- name: state
description: |-
OPTIONAL. Filter tasks by state. If unspecified,
no task state filtering is done.
in: query
required: false
schema:
$ref: '#/components/schemas/tesState'
- name: tag_key
description: |-
OPTIONAL. Provide key tag to filter. The field tag_key is an array
of key values, and will be zipped with an optional tag_value array.
So the query:
```
?tag_key=foo1&tag_value=bar1&tag_key=foo2&tag_value=bar2
```
Should be constructed into the structure { "foo1" : "bar1", "foo2" : "bar2"}
```
?tag_key=foo1
```
Should be constructed into the structure {"foo1" : ""}
If the tag_value is empty, it will be treated as matching any possible value.
If a tag value is provided, both the tag's key and value must be exact
matches for a task to be returned.
Filter Tags Match?
----------------------------------------------------------------------
{"foo": "bar"} {"foo": "bar"} Yes
{"foo": "bar"} {"foo": "bat"} No
{"foo": ""} {"foo": ""} Yes
{"foo": "bar", "baz": "bat"} {"foo": "bar", "baz": "bat"} Yes
{"foo": "bar"} {"foo": "bar", "baz": "bat"} Yes
{"foo": "bar", "baz": "bat"} {"foo": "bar"} No
{"foo": ""} {"foo": "bar"} Yes
{"foo": ""} {} No
in: query
required: false
schema:
type: array
items:
type: string
- name: tag_value
description: |-
OPTIONAL. The companion value field for tag_key
in: query
required: false
schema:
type: array
items:
type: string
- name: page_size
in: query
description: |-
Optional number of tasks to return in one page.
Must be less than 2048. Defaults to 256.
schema:
type: integer
format: int32
- name: page_token
in: query
description: |-
OPTIONAL. Page token is used to retrieve the next page of results.
If unspecified, returns the first page of results. The value can be found
in the `next_page_token` field of the last returned result of ListTasks
schema:
type: string
- $ref: '#/components/parameters/view'
responses:
200:
description: ""
content:
application/json:
schema:
$ref: '#/components/schemas/tesListTasksResponse'
security:
- BearerAuth: [] # Endpoint secured with Bearer token
- BasicAuth: [] # Alternatively, secured with Basic Auth
post:
tags:
- TaskService
summary: CreateTask
description: |-
Create a new task. The user provides a Task document, which the server
uses as a basis and adds additional fields.
operationId: CreateTask
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/tesTask'
required: true
responses:
200:
description: ""
content:
application/json:
schema:
$ref: '#/components/schemas/tesCreateTaskResponse'
x-codegen-request-body-name: body
security:
- BearerAuth: [] # Endpoint secured with Bearer token
- BasicAuth: [] # Alternatively, secured with Basic Auth
/tasks/{id}:
get:
tags:
- TaskService
summary: GetTask
description: |-
Get a single task, based on providing the exact task ID string.
operationId: GetTask
parameters:
- name: id
in: path
required: true
description: ID of task to retrieve.
schema:
type: string
- $ref: '#/components/parameters/view'
responses:
200:
description: ""
content:
application/json:
schema:
$ref: '#/components/schemas/tesTask'
security:
- BearerAuth: [] # Endpoint secured with Bearer token
- BasicAuth: [] # Alternatively, secured with Basic Auth
/tasks/{id}:cancel:
post:
tags:
- TaskService
summary: CancelTask
description: Cancel a task based on providing an exact task ID.
operationId: CancelTask
parameters:
- name: id
in: path
description: ID of task to be canceled.
required: true
schema:
type: string
responses:
200:
description: ""
content:
application/json:
schema:
$ref: '#/components/schemas/tesCancelTaskResponse'
security:
- BearerAuth: [] # Endpoint secured with Bearer token
- BasicAuth: [] # Alternatively, secured with Basic Auth
components:
securitySchemes:
BearerAuth:
type: http
scheme: bearer
bearerFormat: JWT # optional, can specify format if using JWT tokens
BasicAuth:
type: http
scheme: basic
parameters:
view:
name: view
in: query
description: |-
OPTIONAL. Affects the fields included in the returned Task messages.
`MINIMAL`: Task message will include ONLY the fields:
- `tesTask.Id`
- `tesTask.State`
`BASIC`: Task message will include all fields EXCEPT:
- `tesTask.ExecutorLog.stdout`
- `tesTask.ExecutorLog.stderr`
- `tesInput.content`
- `tesTaskLog.system_logs`
`FULL`: Task message includes all fields.
schema:
type: string
default: MINIMAL
enum:
- MINIMAL
- BASIC
- FULL
schemas:
tesCancelTaskResponse:
type: object
description: CancelTaskResponse describes a response from the CancelTask endpoint.
tesCreateTaskResponse:
required:
- id
type: object
properties:
id:
type: string
description: Task identifier assigned by the server.
description: |-
CreateTaskResponse describes a response from the CreateTask endpoint. It
will include the task ID that can be used to look up the status of the job.
tesExecutor:
required:
- command
- image
type: object
properties:
image:
type: string
example: ubuntu:20.04
description: |-
Name of the container image. The string will be passed as the image
argument to the containerization run command. Examples:
- `ubuntu`
- `quay.io/aptible/ubuntu`
- `gcr.io/my-org/my-image`
- `myregistryhost:5000/fedora/httpd:version1.0`
command:
type: array
description: |-
A sequence of program arguments to execute, where the first argument
is the program to execute (i.e. argv). Example:
```
{
"command" : ["/bin/md5", "/data/file1"]
}
```
items:
type: string
example: ["/bin/md5", "/data/file1"]
workdir:
type: string
description: |-
The working directory that the command will be executed in.
If not defined, the system will default to the directory set by
the container image.
example: /data/
stdin:
type: string
description: |-
Path inside the container to a file which will be piped
to the executor's stdin. This must be an absolute path. This mechanism
could be used in conjunction with the input declaration to process
a data file using a tool that expects STDIN.
For example, to get the MD5 sum of a file by reading it into the STDIN
```
{
"command" : ["/bin/md5"],
"stdin" : "/data/file1"
}
```
example: "/data/file1"
stdout:
type: string
description: |-
Path inside the container to a file where the executor's
stdout will be written to. Must be an absolute path. Example:
```
{
"stdout" : "/tmp/stdout.log"
}
```
example: "/tmp/stdout.log"
stderr:
type: string
description: |-
Path inside the container to a file where the executor's
stderr will be written to. Must be an absolute path. Example:
```
{
"stderr" : "/tmp/stderr.log"
}
```
example: "/tmp/stderr.log"
env:
type: object
additionalProperties:
type: string
description: |-
Enviromental variables to set within the container. Example:
```
{
"env" : {
"ENV_CONFIG_PATH" : "/data/config.file",
"BLASTDB" : "/data/GRC38",
"HMMERDB" : "/data/hmmer"
}
}
```
example:
"BLASTDB" : "/data/GRC38"
"HMMERDB" : "/data/hmmer"
ignore_error:
type: boolean
description: |-
Default behavior of running an array of executors is that execution
stops on the first error. If `ignore_error` is `True`, then the
runner will record error exit codes, but will continue on to the next
tesExecutor.
description: Executor describes a command to be executed, and its environment.
tesExecutorLog:
required:
- exit_code
type: object
properties:
start_time:
type: string
description: Time the executor started, in RFC 3339 format.
example: 2020-10-02T10:00:00-05:00
end_time:
type: string
description: Time the executor ended, in RFC 3339 format.
example: 2020-10-02T11:00:00-05:00
stdout:
type: string
description: |-
Stdout content.
This is meant for convenience. No guarantees are made about the content.
Implementations may chose different approaches: only the head, only the tail,
a URL reference only, etc.
In order to capture the full stdout client should set Executor.stdout
to a container file path, and use Task.outputs to upload that file
to permanent storage.
stderr:
type: string
description: |-
Stderr content.
This is meant for convenience. No guarantees are made about the content.
Implementations may chose different approaches: only the head, only the tail,
a URL reference only, etc.
In order to capture the full stderr client should set Executor.stderr
to a container file path, and use Task.outputs to upload that file
to permanent storage.
exit_code:
type: integer
description: Exit code.
format: int32
description: ExecutorLog describes logging information related to an Executor.
tesFileType:
type: string
description: |-
Define if input/output element is a file or a directory. It is not required
that the user provide this value, but it is required that the server fill in the
value once the information is avalible at run time.
default: FILE
enum:
- FILE
- DIRECTORY
tesInput:
required:
- path
type: object
properties:
name:
type: string
description:
type: string
url:
type: string
description: |-
REQUIRED, unless "content" is set.
URL in long term storage, for example:
- s3://my-object-store/file1
- gs://my-bucket/file2
- file:///path/to/my/file
- /path/to/my/file
example: s3://my-object-store/file1
path:
type: string
description: |-
Path of the file inside the container.
Must be an absolute path.
example: /data/file1
type:
$ref: '#/components/schemas/tesFileType'
content:
type: string
description: |-
File content literal.
Implementations should support a minimum of 128 KiB in this field
and may define their own maximum.
UTF-8 encoded
If content is not empty, "url" must be ignored.
streamable:
type: boolean
description: |-
Indicate that a file resource could be accessed using a streaming
interface, ie a FUSE mounted s3 object. This flag indicates that
using a streaming mount, as opposed to downloading the whole file to
the local scratch space, may be faster despite the latency and
overhead. This does not mean that the backend will use a streaming
interface, as it may not be provided by the vendor, but if the
capacity is avalible it can be used without degrading the
performance of the underlying program.
description: Input describes Task input files.
tesListTasksResponse:
required:
- tasks
type: object
properties:
tasks:
type: array
description: |-
List of tasks. These tasks will be based on the original submitted
task document, but with other fields, such as the job state and
logging info, added/changed as the job progresses.
items:
$ref: '#/components/schemas/tesTask'
next_page_token:
type: string
description: |-
Token used to return the next page of results. This value can be used
in the `page_token` field of the next ListTasks request.
description: ListTasksResponse describes a response from the ListTasks endpoint.
tesOutput:
required:
- path
- url
type: object
properties:
name:
type: string
description: User-provided name of output file
description:
type: string
description: Optional users provided description field, can be used for documentation.
url:
type: string
description: |-
URL at which the TES server makes the output accessible after the task is complete.
When tesOutput.path contains wildcards, it must be a directory; see
`tesOutput.path_prefix` for details on how output URLs are constructed in this case.
For Example:
- `s3://my-object-store/file1`
- `gs://my-bucket/file2`
- `file:///path/to/my/file`
path:
type: string
description: |-
Absolute path of the file inside the container.
May contain pattern matching wildcards to select multiple outputs at once, but mind
implications for `tesOutput.url` and `tesOutput.path_prefix`.
Only wildcards defined in IEEE Std 1003.1-2017 (POSIX), 12.3 are supported; see
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13
path_prefix:
type: string
description: |-
Prefix to be removed from matching outputs if `tesOutput.path` contains wildcards;
output URLs are constructed by appending pruned paths to the directory specfied
in `tesOutput.url`.
Required if `tesOutput.path` contains wildcards, ignored otherwise.
type:
$ref: '#/components/schemas/tesFileType'
description: Output describes Task output files.
tesOutputFileLog:
required:
- path
- size_bytes
- url
type: object
properties:
url:
type: string
description: URL of the file in storage, e.g. s3://bucket/file.txt
path:
type: string
description: Path of the file inside the container. Must be an absolute
path.
size_bytes:
type: string
description: |-
Size of the file in bytes. Note, this is currently coded as a string
because official JSON doesn't support int64 numbers.
format: int64
example:
- "1024"
description: |-
OutputFileLog describes a single output file. This describes
file details after the task has completed successfully,
for logging purposes.
tesResources:
type: object
properties:
cpu_cores:
type: integer
description: Requested number of CPUs
format: int32
example: 4
preemptible:
type: boolean
description: |-
Define if the task is allowed to run on preemptible compute instances,
for example, AWS Spot. This option may have no effect when utilized
on some backends that don't have the concept of preemptible jobs.
format: boolean
example: false
ram_gb:
type: number
description: Requested RAM required in gigabytes (GB)
format: double
example: 8
disk_gb:
type: number
description: Requested disk size in gigabytes (GB)
format: double
example: 40
zones:
type: array
description: |-
Request that the task be run in these compute zones. How this string
is utilized will be dependent on the backend system. For example, a
system based on a cluster queueing system may use this string to define
priorty queue to which the job is assigned.
items:
type: string
example: us-west-1
backend_parameters:
type: object
additionalProperties:
type: string
description: |-
Key/value pairs for backend configuration.
ServiceInfo shall return a list of keys that a backend supports.
Keys are case insensitive.
It is expected that clients pass all runtime or hardware requirement key/values
that are not mapped to existing tesResources properties to backend_parameters.
Backends shall log system warnings if a key is passed that is unsupported.
Backends shall not store or return unsupported keys if included in a task.
If backend_parameters_strict equals true,
backends should fail the task if any key/values are unsupported, otherwise,
backends should attempt to run the task
Intended uses include VM size selection, coprocessor configuration, etc.
Example:
```
{
"backend_parameters" : {
"VmSize" : "Standard_D64_v3"
}
}
```
example:
"VmSize" : "Standard_D64_v3"
backend_parameters_strict:
type: boolean
description: |-
If set to true, backends should fail the task if any backend_parameters
key/values are unsupported, otherwise, backends should attempt to run the task
format: boolean
default: false
example: false
description: Resources describes the resources requested by a task.
tesServiceType:
allOf:
- $ref: 'https://raw.githubusercontent.com/ga4gh-discovery/ga4gh-service-info/v1.0.0/service-info.yaml#/components/schemas/ServiceType'
- type: object
required:
- artifact
properties:
artifact:
type: string
enum: [tes]
example: tes
tesServiceInfo:
allOf:
- $ref: 'https://raw.githubusercontent.com/ga4gh-discovery/ga4gh-service-info/v1.0.0/service-info.yaml#/components/schemas/Service'
- type: object
properties:
storage:
type: array
description: |-
Lists some, but not necessarily all, storage locations supported
by the service.
items:
type: string
example:
- file:///path/to/local/funnel-storage
- s3://ohsu-compbio-funnel/storage
tesResources_backend_parameters:
type: array
description: |-
Lists all tesResources.backend_parameters keys supported
by the service
items:
type: string
example: ["VmSize"]
type:
$ref: '#/components/schemas/tesServiceType'
tesState:
type: string
readOnly: True
description: |-
Task state as defined by the server.
- `UNKNOWN`: The state of the task is unknown. The cause for this status
message may be dependent on the underlying system. The `UNKNOWN` states
provides a safe default for messages where this field is missing so
that a missing field does not accidentally imply that
the state is QUEUED.
- `QUEUED`: The task is queued and awaiting resources to begin computing.
- `INITIALIZING`: The task has been assigned to a worker and is currently preparing to run.
For example, the worker may be turning on, downloading input files, etc.
- `RUNNING`: The task is running. Input files are downloaded and the first Executor
has been started.
- `PAUSED`: The task is paused. The reasons for this would be tied to
the specific system running the job. An implementation may have the ability
to pause a task, but this is not required.
- `COMPLETE`: The task has completed running. Executors have exited without error
and output files have been successfully uploaded.
- `EXECUTOR_ERROR`: The task encountered an error in one of the Executor processes. Generally,
this means that an Executor exited with a non-zero exit code.
- `SYSTEM_ERROR`: The task was stopped due to a system error, but not from an Executor,
for example an upload failed due to network issues, the worker's ran out
of disk space, etc.
- `CANCELED`: The task was canceled by the user, and downstream resources have been deleted.
- `CANCELING`: The task was canceled by the user,
but the downstream resources are still awaiting deletion.
- `PREEMPTED`: The task is stopped (preempted) by the system. The reasons for this would be tied to
the specific system running the job. Generally, this means that the system reclaimed the compute
capacity for reallocation.
default: UNKNOWN
example: COMPLETE
enum:
- UNKNOWN
- QUEUED
- INITIALIZING
- RUNNING
- PAUSED
- COMPLETE
- EXECUTOR_ERROR
- SYSTEM_ERROR
- CANCELED
- PREEMPTED
- CANCELING
tesTask:
required:
- executors
type: object
properties:
id:
type: string
description: Task identifier assigned by the server.
readOnly: true
example: job-0012345
state:
$ref: '#/components/schemas/tesState'
name:
type: string
description: User-provided task name.
description:
type: string
description: |-
Optional user-provided description of task for documentation purposes.
inputs:
type: array
description: |-
Input files that will be used by the task. Inputs will be downloaded
and mounted into the executor container as defined by the task request
document.
items:
$ref: '#/components/schemas/tesInput'
example:
- { "url" : "s3://my-object-store/file1", "path" : "/data/file1" }
outputs:
type: array
description: |-
Output files.
Outputs will be uploaded from the executor container to long-term storage.
items:
$ref: '#/components/schemas/tesOutput'
example:
- { "path" : "/data/outfile", "url" : "s3://my-object-store/outfile-1", type: "FILE" }
resources:
$ref: '#/components/schemas/tesResources'
executors:
type: array
description: |-
An array of executors to be run. Each of the executors will run one
at a time sequentially. Each executor is a different command that
will be run, and each can utilize a different docker image. But each of
the executors will see the same mapped inputs and volumes that are declared
in the parent CreateTask message.
Execution stops on the first error.
items:
$ref: '#/components/schemas/tesExecutor'
volumes:
type: array
example:
- "/vol/A/"
description: |-
Volumes are directories which may be used to share data between
Executors. Volumes are initialized as empty directories by the
system when the task starts and are mounted at the same path
in each Executor.
For example, given a volume defined at `/vol/A`,
executor 1 may write a file to `/vol/A/exec1.out.txt`, then
executor 2 may read from that file.
(Essentially, this translates to a `docker run -v` flag where
the container path is the same for each executor).
items:
type: string
tags:
type: object
example:
"WORKFLOW_ID" : "cwl-01234"
"PROJECT_GROUP" : "alice-lab"
additionalProperties:
type: string
description: |-
A key-value map of arbitrary tags. These can be used to store meta-data
and annotations about a task. Example:
```
{
"tags" : {
"WORKFLOW_ID" : "cwl-01234",
"PROJECT_GROUP" : "alice-lab"
}
}
```
logs:
type: array
description: |-
Task logging information.
Normally, this will contain only one entry, but in the case where
a task fails and is retried, an entry will be appended to this list.
readOnly: true
items:
$ref: '#/components/schemas/tesTaskLog'
creation_time:
type: string
description: |-
Date + time the task was created, in RFC 3339 format.
This is set by the system, not the client.
example: 2020-10-02T10:00:00-05:00
readOnly: true
description: Task describes an instance of a task.
tesTaskLog:
required:
- logs
- outputs
type: object
properties:
logs:
type: array
description: Logs for each executor
items:
$ref: '#/components/schemas/tesExecutorLog'
metadata:
type: object
additionalProperties:
type: string
description: Arbitrary logging metadata included by the implementation.
example:
host: worker-001
slurmm_id: 123456
start_time:
type: string
description: When the task started, in RFC 3339 format.
example: 2020-10-02T10:00:00-05:00
end_time:
type: string
description: When the task ended, in RFC 3339 format.
example: 2020-10-02T11:00:00-05:00
outputs:
type: array
description: |-
Information about all output files. Directory outputs are
flattened into separate items.
items:
$ref: '#/components/schemas/tesOutputFileLog'
system_logs:
type: array
description: |-
System logs are any logs the system decides are relevant,
which are not tied directly to an Executor process.
Content is implementation specific: format, size, etc.
System logs may be collected here to provide convenient access.
For example, the system may include the name of the host
where the task is executing, an error message that caused
a SYSTEM_ERROR state (e.g. disk is full), etc.
System logs are only included in the FULL task view.
items:
type: string
description: TaskLog describes logging information related to a Task.