forked from Ensembl/ensembl-hive
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathChangelog
2525 lines (1744 loc) · 116 KB
/
Changelog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
---------------------[before EnsEMBL rel.75]---------------------------------------------------------------------------
* Wed Dec 11 12:55:58 2013 +0000 | Leo Gordon | updated schema diagram (PNG) and description (HTML)
* Mon Dec 9 14:19:48 2013 +0000 | Leo Gordon | bugfix: sqlite mode now works again
* Mon Dec 9 14:01:27 2013 +0000 | Leo Gordon | added Apache 2.0 license to all files
* Wed Dec 4 11:26:09 2013 +0000 | Leo Gordon | schema_change: switched some foreign keys to ON DELETE CASCADE (thanks, Harpreet!)
* Wed Dec 4 11:04:14 2013 +0000 | Matthieu Muffato | Updated the list of dependencies
* Wed Dec 4 10:53:17 2013 +0000 | Matthieu Muffato | Added info on how to run lsf_report.pl and generate_timeline.pl
* Wed Dec 4 10:31:51 2013 +0000 | Matthieu Muffato | Removed the option to use a logscale axis, and added a grid in the background
* Mon Dec 2 18:13:29 2013 +0000 | Matthieu Muffato | Another set of rounding errors
* Mon Dec 2 18:01:59 2013 +0000 | Matthieu Muffato | Added a mode to plot the number of pending workers for each analysis
* Mon Dec 2 18:01:30 2013 +0000 | Matthieu Muffato | Neater way to add/substract a worker
* Mon Dec 2 18:00:02 2013 +0000 | Matthieu Muffato | Added a mode to plot the amount of unused CPU cores each analysis
* Mon Dec 2 17:55:13 2013 +0000 | Matthieu Muffato | "Unused memory" instead of "Wasted memory"
* Mon Dec 2 17:01:52 2013 +0000 | Matthieu Muffato | Pulls in the time information (pending time, cpu usage, lifespan)
* Mon Dec 2 09:56:07 2013 +0000 | Matthieu Muffato | Improved the documentation
* Mon Dec 2 09:53:53 2013 +0000 | Matthieu Muffato | Added "verbose" mode
* Mon Dec 2 08:35:35 2013 +0000 | Matthieu Muffato | Not valid any more when counting the wasted memory (rounding errors)
* Sun Dec 1 23:20:35 2013 +0000 | Matthieu Muffato | Added a mode to plot the amount of wasted memory by each analysis
* Sun Dec 1 23:11:11 2013 +0000 | Matthieu Muffato | Also store the meadow_name in lsf_report
* Sun Dec 1 23:10:21 2013 +0000 | Matthieu Muffato | The unit conversion table is constant
* Sun Dec 1 23:03:56 2013 +0000 | Matthieu Muffato | dbname may be undefined
* Sun Dec 1 22:17:37 2013 +0000 | Matthieu Muffato | Added a mode to plot the number of CPU cores used by each analysis
* Sun Dec 1 22:13:24 2013 +0000 | Matthieu Muffato | Added a mode to plot the RAM used by each analysis
* Tue Dec 3 12:19:20 2013 +0000 | Leo Gordon | create a separate directory layer to group log files of the same iteration
* Tue Dec 3 11:56:07 2013 +0000 | Leo Gordon | separate output files by LSF_job_id and LSF_jobarray_index
* Tue Nov 26 11:08:31 2013 +0000 | Leo Gordon | simplify logging of submission output/error streams
* Wed Nov 27 12:19:20 2013 +0000 | Matthieu Muffato | s/profile/timeline/g
* Wed Nov 27 11:46:31 2013 +0000 | Matthieu Muffato | Gets the birth/death events instead of sampling the database. The "NOTHING" curve is not needed any more
* Thu Nov 14 01:01:33 2013 +0000 | Matthieu Muffato | Reads the data from the database once at the beginning, and process it offline
* Wed Sep 11 00:17:51 2013 +0100 | Matthieu Muffato | "DarkSlateGray" looks better for the "NOTHING" curve
* Wed Sep 11 00:09:00 2013 +0100 | Matthieu Muffato | Added documentation
* Wed Sep 11 00:08:42 2013 +0100 | Matthieu Muffato | GNUplot is now controlled via Chart::Gnuplot
* Tue Sep 10 14:31:15 2013 +0100 | Matthieu Muffato | Improved the GNU-plot output
* Tue Sep 10 01:20:28 2013 +0100 | Matthieu Muffato | Only gnuplot has to know about the filtered analysis, the CSV file should still contain all the data
* Tue Sep 10 00:51:55 2013 +0100 | Matthieu Muffato | First version of a script to generate the analysis profile of a pipeline
* Mon Nov 25 16:57:37 2013 +0000 | Leo Gordon | schema_change: detect and register RELOCATED events that used to mess up things on LSF 9.0 ("job rescheduled" in LSF parlance)
* Mon Nov 25 16:54:11 2013 +0000 | Leo Gordon | be more careful with fetch_overdue_workers (Use 5sec threshold to avoid checking recently active Workers. Do not use it at all when perf
orming -all_dead.)
* Mon Nov 25 16:47:35 2013 +0000 | Leo Gordon | bugfix: last_check_in should only be updated by register_worker_death if the Worker is burying itself
* Mon Nov 25 16:35:16 2013 +0000 | Leo Gordon | cosmetic: added (commented out) warning messages for every external system() call that LSF module runs - simplifies debugging a lot
* Mon Nov 25 14:08:52 2013 +0000 | Leo Gordon | ranked claiming technology: added support for both sqlite and pgsql drivers
* Thu Nov 21 15:40:31 2013 +0000 | Leo Gordon | The last resort: try claiming without an offset (risking a collision)
* Tue Nov 19 11:17:38 2013 +0000 | Leo Gordon | use OFFSET to separate jobs being claimed into ranges
* Mon Nov 18 14:55:11 2013 +0000 | Leo Gordon | No need to left join into worker table - thanks, Javier!
* Tue Nov 12 16:42:32 2013 +0000 | Leo Gordon | ResourceDescription expanded to include both submission_cmd_args and worker_cmd_args. Both args can be specified in a PipeConfig file.
* Tue Nov 12 11:15:56 2013 +0000 | Leo Gordon | increase TotalRunningWorkersMax to 2000
* Mon Nov 11 14:32:04 2013 +0000 | Leo Gordon | added an example of how to turn a csv into a list by param_substitute
* Wed Nov 6 11:13:35 2013 +0000 | Leo Gordon | introducing db_cmd() interface method that takes care of the path to db_cmd.pl
* Tue Nov 5 09:33:37 2013 +0000 | Matthieu Muffato | bugfix: the batch_size parameter should have a hyphen in front of it
* Fri Oct 25 15:28:42 2013 +0100 | Leo Gordon | (1) do not change SEMAPHORED jobs to READY and (2) support more flexibility in choosing which statuses to reset
* Fri Oct 25 11:35:57 2013 +0100 | Leo Gordon | schema change: turned all VARCHAR(<255) into VARCHAR(255) -- should improve experience with long host namest (thanks, MichaelP!)
* Fri Oct 25 10:24:45 2013 +0100 | Leo Gordon | param_required() now automatically sets transient_error(0) before dying, to avoid unnecessary retries (thanks, Matthieu!)
* Thu Oct 24 15:37:36 2013 +0100 | Matthieu Muffato | "expected_size" has to be substituted as well
* Tue Oct 15 11:21:16 2013 +0100 | Matthieu Muffato | bugfix: the query has to be re-substituted for each job
* Tue Oct 8 10:58:22 2013 +0100 | Matthieu Muffato | The SqlHealthcheck runnable can now perform multiple tests
* Fri Sep 27 18:16:11 2013 +0100 | Matthieu Muffato | -reg_conf and -reg_type can be ommitted in db_cmd.pl
---------------------[after Sept'2013 workshops]------------------------------------------------------------------------
* Tue Oct 1 16:30:14 2013 +0100 | Leo Gordon | newer Perl required, BioPerl no longer required, seed_pipeline.pl mentioned
* Tue Oct 1 13:03:21 2013 +0100 | Leo Gordon | pipeline_name is now automatically computed from ClassName; simplified workshop's example files and slides
* Fri Sep 27 15:21:04 2013 +0100 | Leo Gordon | added param_exists() method for checking whether a parameter has been initialized at all
* Thu Sep 26 23:57:55 2013 +0100 | Leo Gordon | cleaned up the last (optional) slide on pipeline_wide_parameters; removed the exercise about abstracting out the compressor (formerly from CompressFiles_conf)
* Thu Sep 26 23:54:55 2013 +0100 | Leo Gordon | separated out "long addition" functionality to concentrate on Hive API when writing the Runnable, and not on maths
* Thu Sep 26 10:53:44 2013 +0100 | Leo Gordon | bugfix: ENSCOMPARASW-131. Swapped two rearrange() calls for slicing a hashref
* Wed Sep 25 16:42:47 2013 +0100 | Leo Gordon | bugfix: ENSCOMPARASW-132. When all dependent jobs (>1) fail to be created due to unique constraint, they now correctly update status to READY
* Wed Sep 25 15:43:58 2013 +0100 | Leo Gordon | bugfix: make sure the pipeline works even when b_multiplier only contains digits 0 and 1
* Wed Sep 25 15:03:09 2013 +0100 | Leo Gordon | bugfix: properly support evaluation of complex substituted expressions that yield a hashref
---------------------[before Sanger workshop]---------------------------------------------------------------------------
* Mon Sep 23 12:29:44 2013 +0100 | Leo Gordon | added "git clone" option
* Mon Sep 23 12:22:07 2013 +0100 | Leo Gordon | some corrections to slides part2
* Sun Sep 22 20:18:42 2013 +0100 | Leo Gordon | part3 of the slides and the solutions (first version)
* Sat Sep 21 22:31:29 2013 +0100 | Leo Gordon | updated slides for parts 1 and 2 and solutions2.tar
* Thu Sep 19 11:25:37 2013 +0100 | Leo Gordon | Sanger version of the first part (re-made in LibreOffice)
* Mon Sep 16 09:30:15 2013 +0100 | Leo Gordon | bugfix: should not assume the presence of JobAdaptor in dataflow
* Fri Sep 13 16:28:13 2013 +0100 | Leo Gordon | alternative substitution syntax #expr( #alpha#*#beta# )expr# and a test script
* Fri Sep 13 11:17:45 2013 +0100 | Leo Gordon | cleanup: two templates that are no longer necessary
* Wed Sep 11 16:45:53 2013 +0100 | Leo Gordon | new colourscheme has arrived!
* Tue Sep 10 16:43:29 2013 +0100 | Leo Gordon | typo bugfix: jobs-->job in SQL
* Tue Sep 10 15:46:40 2013 +0100 | Leo Gordon | bugfix: reset the tried jobs to retry_count=1 and untried ones to retry_count=0 when doing a bulk reset
* Mon Sep 9 13:11:10 2013 +0100 | Leo Gordon | changes made before the talk
* Sun Sep 8 22:58:11 2013 +0100 | Leo Gordon | Preliminary version of slides for the second part of the workshop.
* Sun Sep 8 19:20:02 2013 +0100 | Leo Gordon | bugfix: we should allow any characters apart from { and } in the key
* Sun Sep 8 14:37:43 2013 +0100 | Leo Gordon | cosmetic: a hint for people working on the example
* Sat Sep 7 14:25:36 2013 +0100 | Leo Gordon | added support for EHIVE_HOST and EHIVE_PORT envariables; useful for the workshop environment
* Sat Sep 7 12:35:11 2013 +0100 | Leo Gordon | imported List::Util to be able to run max/min/sum of lists in substituted expressions
* Sat Sep 7 11:26:18 2013 +0100 | Leo Gordon | bugfix: now correctly supports directory names with dots in them
---------------------[before EBI workshop]---------------------------------------------------------------------------
* Thu Sep 5 16:55:44 2013 +0100 | Leo Gordon | PDF version of the workshop slides from GoogleDocs
* Thu Sep 5 09:37:00 2013 +0100 | Leo Gordon | adding new unit - T for terabytes (mainly to pacify EBIs LSF 8 with a reporting bug)
* Wed Sep 4 21:54:43 2013 +0100 | Leo Gordon | the initial state of MemlimitTest pipeline for the workshop
* Wed Sep 4 13:06:46 2013 +0100 | Leo Gordon | methods dbconn_2_mysql(), dbconn_2_pgsql(), db_connect_command(), db_execute_command() are DEPRECATED - use db_cmd.pl instead
* Wed Sep 4 12:49:04 2013 +0100 | Leo Gordon | added support for -pipeline_url as an input parameter (no need to supply hive_driver or password in this case)
* Wed Sep 4 11:53:23 2013 +0100 | Leo Gordon | allow the port number to be skipped but the colon to be present
* Fri Aug 30 15:09:05 2013 +0100 | Leo Gordon | a new example pipeline designed to fail because of MEMLIMIT in some of the cases
* Tue Aug 27 12:09:20 2013 +0100 | Leo Gordon | bugfix: properly use different memory units to compute the memory req in megabytes
* Fri Aug 23 14:40:51 2013 +0100 | Leo Gordon | now performing deep-stack substitution for whatever is dataflown into tables (rather than just dataflowing the output_id)
* Fri Aug 23 12:48:20 2013 +0100 | Leo Gordon | bugfix: substituting the accu signature on demand from the very depths of emitting job's param_stack
* Fri Aug 23 10:46:46 2013 +0100 | Leo Gordon | bugfix: make sure longer input_id hashes are correctly fetched from analysis_data table in "param stack" mode
* Thu Aug 22 15:49:32 2013 +0100 | Leo Gordon | Simplified interface: now db_cmd.pl understands 'CREATE DATABASE' and 'DROP DATABASE' without parameters, given a full URL.
* Thu Aug 22 14:40:11 2013 +0100 | Leo Gordon | Can now do a mysqldump given a URL or Registry data, using a newly supported -to_params option. Note the necessity of 'eval' before 'mysqldump' (it removes quotes around the password).
* Thu Aug 22 11:13:09 2013 +0100 | Leo Gordon | bugfix: sorting by job_id should be numeric, not alphabetic
* Wed Aug 21 16:13:26 2013 +0100 | Leo Gordon | renamed db_conn.pl to db_cmd.pl to avoid the name clash with already existing term
* Wed Aug 21 15:55:46 2013 +0100 | Leo Gordon | Updated schema diagram and description file that include param_id_stack and accu_id_stack in job table.
* Wed Aug 21 14:53:11 2013 +0100 | Leo Gordon | "parameter stack" implementation using two extra fields in job table. Accu content intended for any job_id has preference over Input_id content for the same job.
* Wed Aug 21 11:34:01 2013 +0100 | Leo Gordon | store and retrieve hive_meta.'hive_use_param_stack'
* Wed Aug 21 11:31:20 2013 +0100 | Leo Gordon | cosmetic: reduce the number of synonymous calls to DBI
* Wed Aug 21 10:14:00 2013 +0100 | Leo Gordon | Dataflowing minimal information out of Runnables, relying on templates in PipeConfig file to extend it if needed
* Tue Aug 20 14:32:51 2013 +0100 | Leo Gordon | shortened connection parameters in docs
---------------------[after EnsEMBL rel.73]---------------------------------------------------------------------------
* Thu Aug 15 16:18:49 2013 +0100 | Leo Gordon | Bugfixes to pacify pgsql: changed a non-functional "HAVING" into a nested SELECT, and changed unsupported SUM() into COUNT(CASE ... )
* Thu Aug 15 16:15:28 2013 +0100 | Leo Gordon | An important comment about UNIX sockets (without a port number) vs TCPIP sockets (with a port number).
* Thu Aug 15 14:30:40 2013 +0100 | Leo Gordon | Expose parts of pipeline_db, make them less EnsEMBL-specific, allow multiple failover initializers and use self-reference if none of them worked. Phasing out $self->o('ENV', ...) expressions
* Thu Aug 15 14:27:43 2013 +0100 | Leo Gordon | Allow skipping the port number; you no longer need to define your port if you are happy with driver's default (thanks to db_conn.pl and core's DBConnection)
* Wed Aug 14 18:44:38 2013 +0100 | Leo Gordon | Registry support is now cenralised in DBAdaptor, so scripts just pass reg_* options into the constructor. Passing -reg_type allows to connect to originally non-Hive Registry entries.
* Wed Aug 14 12:58:04 2013 +0100 | Leo Gordon | make sure diagrams are generated from non-Hive registry entries as long as they are Hive-hybrids
* Wed Aug 14 10:44:29 2013 +0100 | Leo Gordon | Support extra parameters added to the client's command line
* Tue Aug 13 17:13:07 2013 +0100 | Leo Gordon | Start using the new db_conn.pl script instead of building driver-specific commands and running them.
* Tue Aug 13 17:10:45 2013 +0100 | Leo Gordon | Execute individual SQL commands as well as sessions; translate some db-meta SQLite into Bash; control verbosity
* Tue Aug 13 15:18:28 2013 +0100 | Leo Gordon | Schema change: changed the data type of monitor.analysis to TEXT as per Michael Paulini's suggestion, to fit more and longer analysis names.
* Tue Aug 13 15:14:01 2013 +0100 | Leo Gordon | Make this patch less mysql-dependent. Needs testing with PostgreSQL.
* Tue Aug 13 15:12:04 2013 +0100 | Leo Gordon | Allow multiple driver-dependent versions of the same patch; suggest schema patching with db_conn.pl commands.
* Fri Aug 9 15:46:37 2013 +0100 | Leo Gordon | concession for Bio::EnsEMBL::DBSQL::DBConnection that does not support urls
* Fri Aug 9 15:20:49 2013 +0100 | Leo Gordon | A unified dispatching client for databases. Finds the correct database client via -url or -reg_conf/-reg_alias combination.
* Fri Aug 9 15:11:09 2013 +0100 | Leo Gordon | Give a more meaningful warning if EHIVE_ROOT_DIR is not set (probably because an external script is trying to run Hive API)
* Sun Jul 28 20:47:52 2013 +0100 | Leo Gordon | bugfix: count both DONE and PASSED_ON jobs when re-balancing semaphores
* Thu Jul 11 11:30:27 2013 +0100 | Leo Gordon | included a new -nosqlvc flag in beekeeper.pl and runWorker.pl to overcome the version restriction in non-critical cases
* Thu Jul 11 11:28:58 2013 +0100 | Leo Gordon | bugfix: propagate no_sql_schema_version_check parameter through the URLFactory/DBAdaptor loop (should be re-factored at some point)
* Wed Jul 10 16:18:37 2013 +0100 | Leo Gordon | cleaned up the pipeline_create_commands a bit
* Tue Jul 9 17:15:32 2013 +0100 | Leo Gordon | the actual schema change (log_message.worker_id DEFAULT NULL)
* Tue Jul 9 17:03:08 2013 +0100 | Leo Gordon | Log all instances when a semaphore had to be re-balanced
* Tue Jul 9 17:02:04 2013 +0100 | Leo Gordon | schema change: allow recording of log_messages with worker_id=NULL
* Tue Jul 9 16:15:19 2013 +0100 | Leo Gordon | changed the interface of balance_semaphores() : pass in $filter_analysis_id instead of $filter_analysis
* Tue Jul 9 15:59:59 2013 +0100 | Leo Gordon | support selective balancing of semaphores funneling into a specific analysis
* Tue Jul 9 15:44:45 2013 +0100 | Leo Gordon | automate the re-balancing of semaphore_counts - do it when there is nothing running
* Tue Jul 9 15:38:47 2013 +0100 | Leo Gordon | introduced a new -balance option for beekeeper.pl so that semaphore_counts could be force-balanced
* Mon Jul 8 15:48:38 2013 +0100 | Leo Gordon | bugfix: back to using CONCAT -- it looks like || operator is non-standard in MySQL
* Tue Jul 2 16:17:01 2013 +0100 | Leo Gordon | start using procedures.pgsql with two main views ("progress" and "msg")
* Tue Jul 2 16:16:00 2013 +0100 | Leo Gordon | start showing resource_class in "progress" view + some SQL unification
* Tue Jul 2 13:15:37 2013 +0100 | Leo Gordon | bugfix: produce more specific bug report (either cannot connect or hive_meta unavailable)
* Tue Jul 2 12:52:30 2013 +0100 | Leo Gordon | separated the task of URL parsing out of the dba caching mechanism (needs more work)
* Mon Jul 1 12:10:44 2013 +0100 | Leo Gordon | bugfix: make sure we are getting the actual meta_value for hive_use_triggers
* Fri Jun 28 16:53:58 2013 +0100 | Leo Gordon | added 'hive_meta' to the list of tables being dumped
* Fri Jun 28 16:35:59 2013 +0100 | Leo Gordon | docs: documented the -input_id command line option
* Fri Jun 28 16:32:24 2013 +0100 | Leo Gordon | optimization: no point in catching and re-throwing my own throw!
* Fri Jun 28 16:27:09 2013 +0100 | Leo Gordon | bugfix: do not attempt to show AnalysisStats in case of an unspecialized Worker
* Fri Jun 28 11:40:31 2013 +0100 | Leo Gordon | bugfix: substituted the hard-coded value for the formula
* Thu Jun 27 16:17:48 2013 +0100 | Leo Gordon | tell the user whether to update the code to match the database SQL schema version, or which SQL patches to apply to the database
* Thu Jun 27 09:24:33 2013 +0100 | Leo Gordon | start checking Hive SQL schema version (code version against db version) and die on mismatch
* Thu Jun 27 09:19:42 2013 +0100 | Leo Gordon | bugfix: make sure we are only getting one value, not the rowhash
* Wed Jun 26 17:35:03 2013 +0100 | Leo Gordon | use SqlSchemaAdaptor to detect the current code's sql version and record it in 'hive_meta' (leave it out of tables.*sql* files)
* Wed Jun 26 17:32:22 2013 +0100 | Leo Gordon | A new "adaptor" for detection of software's sql version based on the number of available sql patches.
* Tue Jun 25 10:35:25 2013 +0100 | Leo Gordon | move Core 'schema_version' out of tables.* files into HiveGeneric_conf (via ApiVersion), expose it for manipulation and make it available to PipeConfigs
* Tue Jun 25 17:08:48 2013 +0100 | Leo Gordon | re-based MetaContainer (now it has two parents, NakedTableAdaptor is first); using the new version
* Tue Jun 25 17:04:04 2013 +0100 | Leo Gordon | new method(s) to remove objects/rows by a given condition
* Tue Jun 25 10:26:25 2013 +0100 | Leo Gordon | new 'hive_meta' table to keep hive_sql_schema_version (=number of patches), hive_pipeline_name and hive_use_triggers
* Wed Jun 26 16:55:34 2013 +0100 | Leo Gordon | Changed an 'our' global variable to ENV{EHIVE_ROOT_DIR} to allow API-only users to set it and work as usual
* Tue Jun 25 15:35:16 2013 +0100 | Miguel Pignatelli | added -hive_force_init option to documentation
* Tue Jun 25 11:11:45 2013 +0100 | Leo Gordon | bugfix: make sure users' tweaking of Data::Dumper::Maxdepth does not mess up stringify()' s operation
* Mon Jun 24 11:27:33 2013 +0100 | Leo Gordon | cosmetic: moving the sorting of keys into an external subroutine (it will be extended later)
* Mon Jun 24 11:07:25 2013 +0100 | Leo Gordon | Utils/Config.pm no longer depends on ENSEMBL_CVS_ROOT_DIR, which becomes non-essential for non-EnsEMBL applications.
* Fri Jun 21 15:54:28 2013 +0100 | Leo Gordon | bugfix: make sure fetch_all() works with empty tables
* Tue Jun 18 20:11:19 2013 +0100 | Leo Gordon | avoid deadlocks when dataflowing under transactional mode (used in Ortheus Runnable for example)
* Tue Jun 18 18:38:26 2013 +0100 | Leo Gordon | print the failed query
---------------------[after EnsEMBL rel.72]---------------------------------------------------------------------------
* Fri Jun 14 15:17:45 2013 +0100 | Leo Gordon | PostgreSQL: connection parameters are now supplied on the command line (no need to set PG variables by hand)
* Thu Jun 13 16:48:01 2013 +0100 | Leo Gordon | given -job_id Scheduler should take the Analysis into account and only submit a Worker for this Analysis
* Thu Jun 13 16:08:12 2013 +0100 | Leo Gordon | renamed some old patch files so that they would all conform to the same naming format
* Thu Jun 13 16:02:23 2013 +0100 | Leo Gordon | Adding foreign keys to PostgreSQL schema by reusing the MySQL file (the syntax happens to be exactly the same!)
* Thu Jun 13 15:50:38 2013 +0100 | Leo Gordon | Rename tables.sql to tables.mysql (less confusion)
* Thu Jun 13 15:47:15 2013 +0100 | Leo Gordon | allow the accumulated values to be longer than 255 characters
* Thu Jun 13 15:34:40 2013 +0100 | Leo Gordon | synchronized all 3 schema files
* Wed Jun 12 12:21:00 2013 +0100 | Leo Gordon | First attempt to support PostgreSQL in eHive. Use with caution.
* Mon Jun 10 17:00:31 2013 +0100 | Leo Gordon | experimental support for undef values in default_options
* Mon Jun 10 11:25:36 2013 +0100 | Leo Gordon | make sure both DatabaseDumper.pm and drop_hive_tables() know about the 'accu' table
* Mon Jun 10 09:54:38 2013 +0100 | Leo Gordon | report job_id of a created job (STDOUT) or warn that it had been created before (STDERR)
* Thu Jun 6 17:18:11 2013 +0100 | Leo Gordon | sqlite mode now also supports "-hive_force_init 1" flag
* Thu Jun 6 11:50:40 2013 +0100 | Leo Gordon | bugfix: correct destringification of a single undef on a line
* Wed Jun 5 17:11:18 2013 +0100 | Leo Gordon | Slow the example down a bit and allow 2 Workers. In "-can_specialize 1" mode the two Workers will complete the whole pipeline.
* Wed Jun 5 17:08:33 2013 +0100 | Leo Gordon | Improved output to distinguish multiple Workers' output in the same stream
* Wed Jun 5 11:31:17 2013 +0100 | Leo Gordon | setting "-hive_force_init 1" will cause init_pipeline.pl to drop the database prior to creation (use with care!)
* Tue Jun 4 17:03:05 2013 +0100 | Leo Gordon | added support for stringification/destringification of accumulated values (an element is allowed to be a complex structure)
* Mon Jun 3 22:28:28 2013 +0100 | Leo Gordon | now supports sleeping for a floating point seconds; take_time can be given by a runtime-computed formula such as "1+rand(1)/1000"
* Mon Jun 3 14:12:27 2013 +0100 | Leo Gordon | stop complaining about undefined take_time parameter (set it to 0 by default)
* Mon Jun 3 14:05:53 2013 +0100 | Leo Gordon | Added optional sleeping functionality to Dummy runnable
* Mon Jun 3 11:46:27 2013 +0100 | Leo Gordon | a presentation introducing accumulated dataflow concept
* Sat Jun 1 21:31:34 2013 +0100 | Leo Gordon | added description attribute to Limiter class
* Thu May 30 16:01:33 2013 +0100 | Leo Gordon | bugfix: both queries modifying semaphore_count are wrapped in protected_prepare_execute
* Wed May 29 16:13:09 2013 +0100 | Leo Gordon | bugfix: allow #expr(...)expr# to be properly overriding in the templates as well
* Tue May 28 16:29:23 2013 +0100 | Leo Gordon | Simplified logic to decide whether Scheduler needs a resync. Temporarily ignore limiters and look at the number of workers initially required.
* Tue May 28 15:13:56 2013 +0100 | Leo Gordon | finally implemented LSF's version of count_running_workers() and a Valley aggregator for all visible meadows
* Tue May 28 13:09:39 2013 +0100 | Leo Gordon | bugfix: make sure specializing workers wait while their analysis is being sync'ed
* Tue May 28 12:34:51 2013 +0100 | Leo Gordon | No need to pass $total_workers_to_submit back to beekeeper anymore.
* Thu May 23 10:10:14 2013 +0100 | Leo Gordon | cosmetic: make it explicit that we are importing rearrange() and throw()
* Wed May 22 12:43:00 2013 +0100 | Leo Gordon | removed dependency on check_ref and assert_ref
* Wed May 22 11:13:38 2013 +0100 | Leo Gordon | Hive is no longer directly dependent on BioPerl
* Thu May 16 16:37:49 2013 +0100 | Leo Gordon | All Hive scripts now detect $::hive_root_dir and use it for setting the @INC so manual setting of PERL5LIB is only needed if using API directly
* Tue May 14 16:55:38 2013 +0100 | Leo Gordon | make sure beekeeper.pl runs runWorker.pl from its own scripts directory (ignore the one in the path)
* Tue May 14 16:14:47 2013 +0100 | Leo Gordon | allow the user to choose a particular hive_root_dir (esp. if there are many)
* Thu May 9 13:55:40 2013 +0100 | Leo Gordon | Copied the @-tag annotation from tables.sql to tables.sqlite. Unlike the original mysql version, the SQLite version gives no warnings when processed by sql2html.pl
* Fri May 3 14:46:03 2013 +0100 | Leo Gordon | bugfix: some farms have non-alphanumeric characters in their cluster name
* Wed May 1 11:48:23 2013 +0100 | Leo Gordon | added a new protected_prepare_execute() method to avoid deadlocks and used it twice in AnalysisJobAdaptor, to fix Stephen's deadlocks
* Wed May 1 11:46:48 2013 +0100 | Leo Gordon | moved Hive's extensions to DBConnection into a separate Hive::DBSQL::DBConnection class
* Wed May 1 12:03:10 2013 +0100 | Leo Gordon | added a patch to add 'accu' table to an existing database & fixed sqlite schema
* Tue Apr 30 13:12:33 2013 +0100 | Leo Gordon | updated schema documentation to reflect addition of 'accu' table
* Tue Apr 30 12:48:09 2013 +0100 | Leo Gordon | added support to generate_graph.pl to show accumulated dataflow on the diagram
* Tue Apr 30 11:38:44 2013 +0100 | Leo Gordon | bugfix: do not crash on encountering accumulated dataflow (just ignore it for the moment); work correctly in DisplayStretched mode
* Mon Apr 29 17:12:17 2013 +0100 | Leo Gordon | Modified the LongMult example to use accumulated dataflow
* Mon Apr 29 17:07:56 2013 +0100 | Leo Gordon | added schema & API support for accumulated dataflow
* Tue Apr 23 15:35:35 2013 +0100 | Leo Gordon | changed schema version to 72
---------------------[before EnsEMBL rel.72]---------------------------------------------------------------------------
* Tue Apr 23 14:50:55 2013 +0100 | Leo Gordon | bugfix: only create 'default' resource_class if it was not actually stored in the database
* Tue Apr 23 13:08:44 2013 +0100 | Leo Gordon | bugfix: check before storing rc (may be necessary in -analysis_topup mode) and warn about consequences of redefining it.
* Tue Apr 23 13:05:37 2013 +0100 | Leo Gordon | API extension: store() now also returns how many actual store operations (as opposed to fetching of already stored ones) it has performed
* Fri Apr 12 16:43:19 2013 +0100 | Leo Gordon | tables.sql was made compatible with Core/Production sql2html.pl and the result is kept in docs/
* Mon Apr 8 12:20:29 2013 +0100 | Miguel Pignatelli [prf1] | Runtime is recorded for failing jobs
* Wed Mar 27 12:16:35 2013 +0000 | Javier Herrero | Added 22 Feb 2013 eHive workshop slides and examples to docs/presentation/
* Tue Mar 26 15:40:19 2013 +0000 | Leo Gordon | Make sure we do not create an analysis with non-hash parameters
* Mon Mar 25 11:05:00 2013 +0000 | Leo Gordon | use param_required() calls wherever a parameter value is required
* Fri Mar 22 16:50:42 2013 +0000 | Leo Gordon | Back to num_required_workers' meaning "how many extra workers we need to add to this analysis"; fixing a scheduling bug/oversensitivity to manual change of batch_size
* Fri Mar 22 15:44:55 2013 +0000 | Leo Gordon | Moved runnable checks into a separate method Analysis::get_compiled_module_name()
* Wed Mar 20 22:44:04 2013 +0000 | Leo Gordon | Do not crash when asked to param_substitute a Regexp, but issue a warning
* Wed Mar 20 13:02:12 2013 +0000 | Leo Gordon | free 'Start' from dealing with 'a_multiplier' by using an input_id_template in PipeConfig instead; renamed 'Start' to 'DigitFactory' to reflect that
* Wed Mar 20 10:35:08 2013 +0000 | Leo Gordon | A new and friendlier README file; defines main concepts and provides contact data
* Thu Mar 14 09:15:53 2013 +0000 | Leo Gordon | bugfix: added missing quotes
* Tue Mar 12 21:45:23 2013 +0000 | Leo Gordon | A 3-analysis pipeline with almost exclusive use of #substitution#; mysql_conn() and mysql_dbname() modified to transform urls as well
* Tue Mar 12 12:06:37 2013 +0000 | Leo Gordon | Improved legend with useful commands
* Tue Mar 12 10:56:55 2013 +0000 | Leo Gordon | The smallest Hive pipeline example possible. Just one SystemCmd-based analysis.
* Mon Mar 11 23:59:20 2013 +0000 | Leo Gordon | A cleaner example of a two-analysis pipelines with better demonstration of #substitution# and only implicit $self->o() references
* Mon Mar 11 21:13:58 2013 +0000 | Leo Gordon | moved 'go_figure_dbc()' into Utils; supplied defaults for MySQLTransfer to make it quiet
* Tue Mar 12 21:04:14 2013 +0000 | emepyc | This file is now JSON strict
* Tue Mar 12 13:50:33 2013 +0000 | Matthieu Muffato | Do not buffer the resultset (only tested with MySQL)
* Tue Mar 12 11:07:23 2013 +0000 | Matthieu Muffato | bugfix: <= instead of <
* Fri Mar 8 18:41:39 2013 +0000 | Matthieu Muffato | In "topup" mode, concurrent inserts make the row count unreliable
* Tue Mar 5 17:05:21 2013 +0000 | Leo Gordon | Protect generate_graph.pl in table-drawing mode from printing too many rows (by setting a limit in JSON config)
* Tue Mar 5 13:12:32 2013 +0000 | Leo Gordon | Protect generate_graph.pl in job-drawing mode from printing too many jobs (by setting a limit in JSON config)
* Tue Mar 5 13:10:38 2013 +0000 | Leo Gordon | extend a method in JobAdaptor to return a limited number of jobs (for use in generate_graph)
* Fri Mar 1 11:53:39 2013 +0000 | Matthieu Muffato | Fixed a memory leak in data_dbc()
* Thu Feb 28 15:41:46 2013 +0000 | Leo Gordon | cosmetic: renamed README.txt back to README to retain an unbroken history in CVS
* Thu Feb 28 15:37:42 2013 +0000 | Leo Gordon | cosmetic: added new commits to README and renamed it Changelog; split out the old README.txt (non-Changelog part)
---------------------[before and during EnsEMBL rel.71]----------------------------------------------------------------
* Thu Feb 28 10:12:41 2013 +0000 | Leo Gordon | avoid having beekeeper run in submitted-to-the-farm state - detect it, report and quit
* Thu Feb 28 09:47:40 2013 +0000 | Leo Gordon | param_substitution is now default everywhere, no need to call it explicitly
* Thu Feb 28 09:42:33 2013 +0000 | Leo Gordon | added param_required() and param_is_defined() interfaces to Process
* Wed Feb 27 21:34:47 2013 +0000 | Leo Gordon | bugfix: updated examples of how to use JobFactory without and with input_id_template
* Wed Feb 27 19:08:40 2013 +0000 | Leo Gordon | bugfix: changed implementation of data_dbc() to correctly compare things before caching
* Wed Feb 27 14:00:42 2013 +0000 | Leo Gordon | Clone::clone is no longer used, so dependency has been removed
* Fri Feb 22 16:55:12 2013 +0000 | Matthieu Muffato | It is more efficient to give MySQL a LIMIT clause
* Sat Feb 23 00:52:57 2013 +0000 | Leo Gordon | JobFactory uses $overriding_hash to create jobs/rows from input_id_template; 'input_id' parameter deprecated; standaloneJob supports templates.
* Sat Feb 23 00:49:15 2013 +0000 | Leo Gordon | Substitution machinery now supports an extra $overriding_hash that contains parameters with higher precedence than the whole of param() structure
* Fri Feb 22 16:36:19 2013 +0000 | Leo Gordon | fixed several problems with parameter substitution and detection of undefs; added param_required() and param_is_defined()
* Fri Feb 22 10:42:51 2013 +0000 | Leo Gordon | reload the cached data_dbc() value on change of param('db_conn')
* Thu Feb 21 16:14:35 2013 +0000 | emepyc | The modules of the analyses must be accessible
* Fri Feb 15 17:05:20 2013 +0000 | Matthieu Muffato | New runnable to check the size of the resultset of any SQL query
* Tue Feb 19 17:18:06 2013 +0000 | Leo Gordon | removed param_substitute() call from Runnables -- no longer needed, as substitution is automatic
* Tue Feb 19 16:46:05 2013 +0000 | Leo Gordon | a "total" (anything-to-anything) substitution mechanism has been implemented in Hive::Params
* Fri Feb 15 17:04:36 2013 +0000 | Matthieu Muffato | The preferred meadow type must be registered
* Fri Feb 15 17:03:24 2013 +0000 | Matthieu Muffato | The modules of the analysis must be loadable
* Fri Feb 15 17:01:27 2013 +0000 | Matthieu Muffato | In dataflow rules within the same database, the destination analysis must exist
* Fri Feb 15 17:00:38 2013 +0000 | Matthieu Muffato | In control rules within the same database, the condition analysis must exist
* Fri Feb 15 22:25:39 2013 +0000 | Leo Gordon | make sure all LSF pids are quoted, to protect them from tcsh interpretation of square brackets
* Thu Feb 14 16:41:49 2013 +0000 | Leo Gordon | a new script to remove old "DONE" jobs and associated job_file and log_message entries
* Thu Feb 14 10:45:26 2013 +0000 | Leo Gordon | seed_pipeline.pl now shows examples of input_ids of seedable analyses
* Thu Feb 14 09:54:00 2013 +0000 | Leo Gordon | Made $final_clause a parameter of _generic_fetch() & removed default ORDER-BY; hopefully faster
* Wed Feb 13 17:20:14 2013 +0000 | Leo Gordon | In case no -logic_name/-analysis_id was supplied, show the list of analyses that have no incoming dataflow (and so are candidates for seeding)
* Wed Feb 13 13:52:56 2013 +0000 | Leo Gordon | Added perldoc to seed_pipeline.pl script
* Wed Feb 13 13:35:55 2013 +0000 | Leo Gordon | A new script to quickly seed any analysis of any pipeline.
* Wed Feb 13 10:19:08 2013 +0000 | Leo Gordon | hide the calls to URLFactory into the DBAdaptor's constructor
* Tue Feb 12 10:22:02 2013 +0000 | Leo Gordon | hash of resources no longer depends on default_meadow (bugfix)
* Thu Feb 7 11:42:11 2013 +0000 | Kathryn Beal | Updated to release 71
* Wed Feb 6 17:43:21 2013 +0000 | Matthieu Muffato | Tables must be in the right order. Otherwise, the foreign key checks complain
* Fri Jan 25 19:42:28 2013 +0000 | Leo Gordon | resolving conflict: using mine
|\
| * Tue Jan 15 11:03:26 2013 +0000 | Matthieu Muffato | Table dataflows are now included into semaphore boxes (bugfix: wrong internal name)
* | Fri Jan 25 19:26:36 2013 +0000 | Leo Gordon | diagram improvement: (1) no more "empty boxes" and (2) tables dataflown from a box are shown in their boxes
|/
* Mon Jan 14 13:23:52 2013 +0000 | Leo Gordon | Added a new presentation, moved presentations into a separate folder.
* Fri Jan 11 11:19:11 2013 +0000 | Leo Gordon | cosmetic fix: commented back the debug output that was left uncommented by mistake
* Fri Jan 11 11:07:47 2013 +0000 | Leo Gordon | Added coloured barchart display option and jobs/data display option (no big data checks, use with care on small examples). 'Pad' is now configurable from JSON. Beware: JSON config options have moved around!
* Thu Jan 10 16:14:06 2013 +0000 | Leo Gordon | injected a padding around the pipeline diagram
* Fri Jan 4 17:03:14 2013 +0000 | Leo Gordon | send the fatal "COULDNT CREATE WORKER" message to stderr instead of stdout
* Fri Jan 4 15:10:47 2013 +0000 | Leo Gordon | added command line options -submit_stdout_file and -submit_stderr_file to peek into submission output/error streams
* Fri Jan 4 14:51:35 2013 +0000 | Leo Gordon | using PERLs File::Path::make_path instead of mkdir-p to create hive_log_dir
* Fri Jan 4 11:03:31 2013 +0000 | Leo Gordon | added a LongMult pipeline diagram in completed state (for easier reference)
* Mon Dec 17 12:13:43 2012 +0000 | Leo Gordon | fixed a bug in computing num_required_workers according to the new rules (thanks Matthieu for reporting)
* Wed Dec 12 14:41:16 2012 +0000 | Leo Gordon | bugfix: correctly checking analysis_capacity (thanks Andy for reporting)
* Wed Dec 12 10:44:01 2012 +0000 | Leo Gordon | bugfix: do not proceed with negative numbers of workers for submission (thanks to Matthieu for reporting)
* Thu Dec 6 11:18:59 2012 +0000 | Leo Gordon | bugfix: -job_limit now works correctly also with respecializing workers
* Thu Dec 6 10:47:07 2012 +0000 | Leo Gordon | fix: 'msg' view now displays the analysis of the job (which is fixed), not that of worker (which may change with time)
* Wed Dec 5 22:25:35 2012 +0000 | Leo Gordon | experimental feature: re-specialization of workers instead of dying from NO_WORK
* Sat Dec 1 19:11:56 2012 +0000 | Leo Gordon | switched to using Limiter class for job_limit and made some related structural changes
* Fri Nov 30 13:47:42 2012 +0000 | Leo Gordon | changed the meaning of 'num_required_workers' to "total estimated number of workers needed for this analysis" ( 'num_running_workers' is now included in it )
* Thu Nov 29 12:21:22 2012 +0000 | Leo Gordon | fresh schema diagram
* Thu Nov 29 11:46:47 2012 +0000 | Leo Gordon | renamed 'job_message' table to 'log_message' and JobMessageAdaptor to LogMessageAdaptor everywhere
* Wed Nov 28 21:40:45 2012 +0000 | Leo Gordon | swapped hive_capacity for analysis_capacity in example PipeConfig files
* Wed Nov 28 21:30:44 2012 +0000 | Leo Gordon | change of default behaviour: hive_capacity is now off by default (=NULL); setting hive_capacity=0 or analysis_capacity=0 stops scheduling AND specialization to a particular analysis
* Wed Nov 28 13:23:48 2012 +0000 | Leo Gordon | cleanup: removed runnable(), output() and parameters() subroutines from Process as no longer used by Compara
* Wed Nov 28 12:21:37 2012 +0000 | Leo Gordon | removed the "compile_module_once" option as the only way to compile modules now is once after specialization
* Tue Nov 27 11:31:00 2012 +0000 | Leo Gordon | secutiry: make sure stringify() always produces perl-parsable structures, so that global settings of Data::Dumper do not affect its results (thanks to Uma and Matthieu for reporting)
---------------------[during EnsEMBL rel.70]----------------------------------------------------------------
* Fri Nov 23 14:26:53 2012 +0000 | Leo Gordon | bugifx: create meadow_capacity limiters whether or not there is a limit
* Thu Nov 22 21:26:37 2012 +0000 | Leo Gordon | added a new per-analysis "analysis_capacity" limiter for cases where users want to limit analyses independently
* Thu Nov 22 16:56:36 2012 +0000 | Leo Gordon | switch the Scheduler to using universal Limiter objects (cleaner code, more precise computation and should allow for expansion)
* Thu Nov 22 14:07:21 2012 +0000 | Leo Gordon | moved pending adjustment out of the main scheduling subroutine, which simplified the logic and improved readability
* Thu Nov 22 17:21:22 2012 +0000 | Leo Gordon | Introduced a new 'NO_ROLE' cause_of_death for failures during specialization (not so much of an error, really!)
* Fri Nov 23 11:16:12 2012 +0000 | Leo Gordon | bugfix: avoid specializing in an otherwise BLOCKED analysis that is temporarily in SYNCHING state (thanks to Kathryn for reporting)
* Wed Nov 21 12:23:11 2012 +0000 | Leo Gordon | (multi-meadow scheduler) restrict the set of analyses that a worker with a given meadow_type can specialize into
* Tue Nov 20 15:35:44 2012 +0000 | Leo Gordon | separated the Scheduler's code into a separate module (not an object yet)
* Tue Nov 20 16:57:23 2012 +0000 | Matthieu Muffato | Merge branch 'master' of git.internal.sanger.ac.uk:/repos/git/ensembl/compara/ensembl-hive
|\
| * Tue Nov 20 12:35:30 2012 +0000 | Leo Gordon | bugfix: if re-running a job that creates a semaphored group, we no longer die (thanks Miguel for reporting)
| * Mon Nov 19 16:25:14 2012 +0000 | Leo Gordon | Added API and schema support for analysis_base.meadow_type / Analysis->meadow_type(), which will be NULL/undef by default
| * Mon Nov 19 15:22:44 2012 +0000 | Leo Gordon | proof of concept: all structures passed into calls and back are now meadow-aware
| * Fri Nov 16 13:44:01 2012 +0000 | Leo Gordon | pass complete valley-wide stats into schedule_workers without filtering
| * Fri Nov 16 10:36:49 2012 +0000 | Leo Gordon | aggregate meadow stats collection in the Valley
* | Mon Nov 19 22:16:26 2012 +0000 | Matthieu Muffato | Merge branch 'master' of git.internal.sanger.ac.uk:/repos/git/ensembl/compara/ensembl-hive
|\ \
| |/
| * Fri Nov 16 23:27:58 2012 +0000 | Leo Gordon | turn Utils::Graph into Configurable and use the same interface to config as Meadow and Valley
* | Sun Nov 18 11:59:06 2012 +0000 | Matthieu Muffato | All the combinations of parameters are tested and cover all possible cases
|/
* Fri Nov 16 15:03:19 2012 +0000 | Leo Gordon | bugfix: no longer leaves CLAIMED jobs after compilation error during specific -job_id execution
* Fri Nov 16 14:29:48 2012 +0000 | Leo Gordon | bugfix: min_batch_time moved to prevent infinite loop in -compile_module_once 0 mode
* Fri Nov 16 12:11:01 2012 +0000 | Leo Gordon | make Valley into Configurable and move SubmitWorkersMax into Valley's context, because it is more "global" than a Meadow
* Fri Nov 16 11:52:51 2012 +0000 | Leo Gordon | concentrate the "Configurable" functionality in one class with the intention to use it wider
* Fri Nov 16 10:48:01 2012 +0000 | Leo Gordon | meadow->signature() is slightly more useful than meadow->toString()
* Thu Nov 15 12:08:11 2012 +0000 | Leo Gordon | removed PendingAdjust option from beekeeper and config file as it never really needs to be unset
* Thu Nov 15 10:37:01 2012 +0000 | Leo Gordon | simplification of the interface: scripts no longer understand --user/--password/--host/--port/--database and require --url instead
* Tue Nov 13 15:19:29 2012 +0000 | Leo Gordon | capture Worker's death message during the new 'SPECIALIZATION' status in job_message/msg (thanks, Thomas!)
* Tue Nov 13 13:07:26 2012 +0000 | Leo Gordon | bugfix: msg view should behave when analysis_id is still NULL
* Tue Nov 13 11:06:01 2012 +0000 | Leo Gordon | feature: jobless workers will now leave module compilation errors in the job_message table (thanks, Kathryn!)
---------------------[before EnsEMBL rel.70]----------------------------------------------------------------
* Mon Nov 12 14:15:40 2012 +0000 | Leo Gordon | updated the release number to 70 in the schema
* Fri Nov 9 13:59:24 2012 +0000 | Leo Gordon | bugfix: worker.log_dir varchar(80) was too limiting, now extended to varchar(255); (thanks, Kathryn!)
* Fri Nov 9 12:05:28 2012 +0000 | Leo Gordon | bugfix: make sure we release claimed jobs from a manually-run worker whose Runnable fails at compilation (thanks, Miguel!)
* Thu Nov 8 10:50:51 2012 +0000 | Leo Gordon | job_count_breakout now also returns the components that go into the breakout_label
* Tue Nov 6 12:55:26 2012 +0000 | Leo Gordon | bugfix: now works on patched schema too
* Tue Nov 6 12:52:34 2012 +0000 | Leo Gordon | substituted fetch_all_failed_jobs() by a more versatile fetch_all_by_analysis_id_status()
* Tue Nov 6 12:23:45 2012 +0000 | Leo Gordon | move job_count_breakout code into AnalysisStats to be called centrally
* Fri Nov 2 14:23:13 2012 +0000 | Leo Gordon | quote and env-substitute runWorker.pl's -url commandline parameter
* Fri Nov 2 15:14:57 2012 +0000 | Leo Gordon | parametrically slow down the LongMult test pipeline using -take_time global parameter
* Fri Nov 2 10:03:39 2012 +0000 | Leo Gordon | cosmetic: removed CVS magic $_Revision and $_Author variables that cause CVS out of sync with Git
* Fri Nov 2 09:59:09 2012 +0000 | Leo Gordon | cosmetic: added a short summary of Git commits to Changelog for CVS-only users
* Thu Nov 1 15:59:55 2012 +0000 | Leo Gordon | bugfix: query in Q::fetch_all_dead_workers_with_jobs() has to reference worker table by its full name
* Thu Nov 1 15:31:36 2012 +0000 | Leo Gordon | clearer display of job_counters in beekeeper's output
* Thu Nov 1 15:16:08 2012 +0000 | Leo Gordon | clearer display of job_counters on the graph; removed misleading and unused remaining_job_count() and cpu_minutes_remaining()
* Thu Nov 1 14:33:42 2012 +0000 | Leo Gordon | Merge branch 'bugfix_greedy_grep'
|\
| * Thu Nov 1 12:05:35 2012 +0000 | Leo Gordon | avoid grepping out lines by patterns potentially present in job_name_prefix
* | Thu Nov 1 12:00:00 2012 +0000 | Leo Gordon | bugfix: only limit buried-in-haste workers to really dead ones
|/
* Wed Oct 31 13:22:46 2012 +0000 | Leo Gordon | fixing permissions of all files in one go
* Wed Oct 31 13:19:14 2012 +0000 | Leo Gordon | Do not expose the password in workers' url by storing it in an environment variable
---------------------[after EnsEMBL rel.69]----------------------------------------------------------------
2012-10-19 15:45 lg4
* sql/tables.sql: better match heavy queries with indices on job
table
2012-10-19 15:43 lg4
* modules/Bio/EnsEMBL/Hive/: Queen.pm, DBSQL/AnalysisJobAdaptor.pm:
merge reset_and_grab into one subroutine; pre-increment dependent
semaphore if re-running a DONE job; use -force flag for
force-running an individual job
2012-10-19 15:40 lg4
* scripts/beekeeper.pl: propagation of -force flag through
beekeeper.pl
2012-10-17 12:55 lg4
* modules/Bio/EnsEMBL/Hive/: Queen.pm, Worker.pm,
DBSQL/AnalysisJobAdaptor.pm: moved special-job-reset and
special-job-reclaim into the same call, removed the unnecessary
fetch in between
2012-10-16 12:37 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm: cosmetic
changes
2012-10-16 10:42 lg4
* modules/Bio/EnsEMBL/Hive/Queen.pm, scripts/beekeeper.pl: try not
to shock the Q::register_worker_death() code with inexistent
W->analysis_id
2012-10-16 10:26 lg4
* modules/Bio/EnsEMBL/Hive/Worker.pm, scripts/runWorker.pl: moved
specializaton call into W::run, so that death messages during
specialization could be recorded in W->log_dir
2012-10-15 16:06 lg4
* modules/Bio/EnsEMBL/Hive/Worker.pm: print the resource_class_id
of the worker
2012-10-15 16:04 lg4
* scripts/beekeeper.pl: pass either rc_name or logic_name or job_id
from beekeeper.pl to runWorker.pl
2012-10-15 10:44 mm14
* modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm: bugfix:
$analysis instead of $self->analysis
2012-10-15 10:42 lg4
* modules/Bio/EnsEMBL/Hive/Worker.pm: set compile_module_once=1 as
default
2012-10-13 12:31 lg4
* modules/Bio/EnsEMBL/Hive/: URLFactory.pm,
PipeConfig/HiveGeneric_conf.pm: allow database names to contain
dashes
2012-10-13 11:02 lg4
* modules/Bio/EnsEMBL/Hive/Queen.pm, scripts/runWorker.pl,
sql/patch_2012-10-13.sql, sql/tables.sql, sql/tables.sqlite: if
runWorker.pl is run manually, rc_name may stay NULL in the
database
2012-10-12 21:24 lg4
* docs/: hive_schema.mwb, hive_schema.png: updated schema diagram
with worker.resource_class_id
2012-10-12 17:15 lg4
* modules/Bio/EnsEMBL/Hive/Queen.pm,
modules/Bio/EnsEMBL/Hive/Worker.pm, scripts/runWorker.pl,
sql/foreign_keys.mysql, sql/patch_2012-10-12.sql, sql/tables.sql,
sql/tables.sqlite: separating create_new_worker() from
specialize_new_worker()
2012-10-11 12:37 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm,
sql/triggers.mysql, sql/triggers.sqlite: proper counting of
semaphored jobs by triggers and in constructor
2012-10-10 14:45 lg4
* modules/Bio/EnsEMBL/Hive/Queen.pm, scripts/runWorker.pl:
refactoring of the Q::create_new_worker() and introduction of
-force flag
2012-10-10 14:36 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisStatsAdaptor.pm: we should
not leave SYNCHING analysis out (especially if there are not too
many READY analyses)
2012-10-10 14:34 lg4
* modules/Bio/EnsEMBL/Hive/RunnableDB/LongMult/PartMultiply.pm:
slow things down a little
2012-10-09 10:48 lg4
* docs/hive_schema.mwb, docs/hive_schema.png,
sql/foreign_keys.mysql: added a DF-to-DF foreign key and
refreshed the diagram
2012-10-09 10:25 lg4
* sql/tables.sqlite: bugfix: forgot to add semaphored_job_count to
SQLite schema, now included
2012-10-09 10:22 lg4
* sql/: patch_2012-10-08.sql, tables.sql, tables.sqlite: turned two
unique keys into primary keys (needed by BaseAdaptor)
2012-10-08 16:06 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisStatsAdaptor.pm: allow the
batch_size to be updated via
$analysis_stats_adaptor->update($stats);
2012-10-08 12:17 lg4
* modules/Bio/EnsEMBL/Hive/Queen.pm, scripts/runWorker.pl: removed
the input_id functionality from runWorker as both redundant
(standaloneJob) and probably not working
2012-10-08 12:13 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm: those
"return" statements would have never worked anyway, so I removed
them
2012-10-05 16:14 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm: extend the
param_init() of the garbage-collected jobs to include
analysis->parameters() for template substitution (still limited!)
2012-10-05 14:14 lg4
* modules/Bio/EnsEMBL/Hive/Queen.pm, sql/patch_2012-10-06.sql,
sql/tables.sql, sql/tables.sqlite: cause_of_death="" no longer
used for decision making, cause_of_death IS NULL by default and
FATALITY renamed UNKNOWN for clarity
2012-10-05 10:09 lg4
* modules/Bio/EnsEMBL/Hive/Queen.pm: fetch_failed_workers() is
dropped as no longer used, get_hive_current_load() cosmetically
touched
2012-10-04 16:47 lg4
* modules/Bio/EnsEMBL/Hive/AnalysisStats.pm,
modules/Bio/EnsEMBL/Hive/PipeConfig/HiveGeneric_conf.pm,
sql/patch_2012-10-05.sql, sql/tables.sql, sql/tables.sqlite:
EMPTY state added and definitions of READY and WORKING made more
intuitive
2012-10-04 15:45 lg4
* modules/Bio/EnsEMBL/Hive/AnalysisStats.pm: bugfix: a typo
2012-10-04 15:39 lg4
* modules/Bio/EnsEMBL/Hive/AnalysisStats.pm,
modules/Bio/EnsEMBL/Hive/Queen.pm,
modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm,
modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisStatsAdaptor.pm,
scripts/beekeeper.pl, sql/patch_2012-10-04.sql, sql/tables.sql,
sql/tables.sqlite, sql/triggers.mysql, sql/triggers.sqlite:
introduced semaphored_job_count, renamed
unclaimed_job_count-->ready_job_count, changed reporting, fixed
hive_capacity=0
2012-10-03 14:55 lg4
* modules/Bio/EnsEMBL/Hive/: Queen.pm,
DBSQL/AnalysisStatsAdaptor.pm: common denominator for
schedule_workers and specialize_new_worker
2012-10-03 14:11 lg4
* modules/Bio/EnsEMBL/Hive/: Queen.pm, DBSQL/AnalysisAdaptor.pm:
Fetching data via AnalysisAdaptor allows to print logic_names of
failed analyses
2012-10-03 11:09 lg4
* scripts/runWorker.pl: print stats if could not create worker
anyway, but do not sync in the end (too cryptic)
2012-10-03 10:51 lg4
* modules/Bio/EnsEMBL/Hive/: DBSQL/AnalysisJobAdaptor.pm,
PipeConfig/HiveGeneric_conf.pm: renamed -input_job_id to
-prev_job_id to be in sync with other names
2012-10-02 16:47 lg4
* docs/: hive_schema.mwb, hive_schema.png: updated schema diagrams
2012-10-02 16:18 lg4
* modules/Bio/EnsEMBL/Hive/Analysis.pm,
modules/Bio/EnsEMBL/Hive/AnalysisStats.pm,
modules/Bio/EnsEMBL/Hive/Worker.pm,
modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm,
modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisStatsAdaptor.pm,
modules/Bio/EnsEMBL/Hive/PipeConfig/HiveGeneric_conf.pm,
modules/Bio/EnsEMBL/Hive/Utils/Graph.pm,
sql/patch_2012-10-02.sql, sql/tables.sql, sql/tables.sqlite:
moved failed_job_tolerance, max_retry_count, can_be_empty and
priority columns from analysis_stats to analysis_base
2012-10-02 14:56 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm: bugfix: do
not forget PRE_- and POST_CLEANUP states
2012-10-02 13:00 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm: bugfix:
changed the order of the atomic SEMAPHORED->READY state&counter
UPDATE so that it works as intended in SQLite as well
2012-10-02 12:17 lg4
* sql/tables.sql: added a fake default to last_update field
(required by stricter MySQL setup of Vega)
2012-10-02 11:48 lg4
* modules/Bio/EnsEMBL/Hive/AnalysisStats.pm,
modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisStatsAdaptor.pm,
sql/tables.sql, sql/tables.sqlite: added specific defaults into
analysis_stats and analysis_stats_monitor; re-ordered the fields
for easier navigation
2012-10-01 15:58 lg4
* modules/Bio/EnsEMBL/Hive/: Worker.pm, DBSQL/BaseAdaptor.pm:
bugfix: fetch_by_dbID should work now (thanks to ChuangKee and
Miguel)
2012-10-01 12:53 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/BaseAdaptor.pm: bugfix:
primary_key_constraint now works (thanks to Miguel!)
2012-09-28 11:01 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm: bugfix:
typo fixed, thanks to Miguel for pointing out!
2012-09-27 16:48 lg4
* modules/Bio/EnsEMBL/Hive/: AnalysisStats.pm,
DBSQL/AnalysisStatsAdaptor.pm, PipeConfig/HiveGeneric_conf.pm:
make AnalysisStats a rearrangeable EnsEMBL-style constructor, a
proper store method and other preparations
2012-09-27 15:29 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisStatsAdaptor.pm:
simplification of DYNAMIC hive_capacity update code
2012-09-27 12:03 lg4
* scripts/cmd_hive.pl: retired the cmd_hive.pl script; likely not
working and duplicating functionality of more flexible PipeConfig
2012-09-27 10:50 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm: bugfix:
also release jobs that were in PRE_CLEANUP or POST_CLEANUP states
2012-09-26 15:03 lg4
* modules/Bio/EnsEMBL/Hive/: Queen.pm, DBSQL/BaseAdaptor.pm:
switched Queen to become descendent of Hive::DBSQL::ObjectAdaptor
and removed _generic_fetch from it
2012-09-26 12:31 lg4
* modules/Bio/EnsEMBL/Hive/: Queen.pm, Worker.pm: rearranged
Worker's storable getters/setters, introduced and used a proper
rearranging new() method
2012-09-26 11:27 lg4
* modules/Bio/EnsEMBL/Hive/: Queen.pm, Worker.pm: Worker doesnt
really need its own reference to db (can go via adaptor)
2012-09-25 16:20 lg4
* modules/Bio/EnsEMBL/Hive/Queen.pm: bugfix: GROUP BY now includes
a proper prefix of the index
2012-09-25 16:04 lg4
* modules/Bio/EnsEMBL/Hive/Queen.pm,
modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm,
modules/Bio/EnsEMBL/Hive/PipeConfig/LongMult_conf.pm,
sql/patch_2012-09-25.sql, sql/tables.sql, sql/tables.sqlite:
Dropped 'BLOCKED' job status and introduced 'SEMAPHORED' status
that is maintained in sync with semaphore_counts; less confusing
and more efficient (with new 3-part index)
2012-09-25 12:32 lg4
* modules/Bio/EnsEMBL/Hive/Queen.pm,
modules/Bio/EnsEMBL/Hive/Worker.pm, scripts/beekeeper.pl,
scripts/runWorker.pl, sql/patch_2012-09-24.sql, sql/tables.sql,
sql/tables.sqlite: record each Workers log_dir in the database;
simplified the log_dir code and renamed cmdline options
accordingly
2012-09-21 22:16 lg4
* docs/hive_schema.mwb, docs/hive_schema.png,
modules/Bio/EnsEMBL/Hive/Analysis.pm,
modules/Bio/EnsEMBL/Hive/AnalysisStats.pm,
modules/Bio/EnsEMBL/Hive/Queen.pm,
modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisStatsAdaptor.pm,
modules/Bio/EnsEMBL/Hive/PipeConfig/HiveGeneric_conf.pm,
scripts/lsf_report.pl, sql/foreign_keys.mysql,
sql/patch_2012-09-21.sql, sql/tables.sql, sql/tables.sqlite:
moved resource_class_id from analysis_stats and
analysis_stats_monitor to analysis_base
2012-09-21 14:46 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/: AnalysisJobAdaptor.pm,
AnalysisStatsAdaptor.pm: fetch_all never seems to be executed for
these adaptors
2012-09-21 09:34 lg4
* modules/Bio/EnsEMBL/Hive/Meadow/LSF.pm: bugfix: better parsing of
the LSF-job-name
2012-09-20 15:56 lg4
* modules/Bio/EnsEMBL/Hive/: Queen.pm,
DBSQL/AnalysisStatsAdaptor.pm: optimization: worker should not
sync analyses it is not ready to run
2012-09-20 11:51 lg4
* modules/Bio/EnsEMBL/Hive/Meadow.pm,
modules/Bio/EnsEMBL/Hive/Queen.pm,
modules/Bio/EnsEMBL/Hive/Meadow/LOCAL.pm,
modules/Bio/EnsEMBL/Hive/Meadow/LSF.pm, scripts/beekeeper.pl:
replaced internal rc_id by rc_name in the Meadow code and in most
of the Scheduler; needs testing
2012-09-20 11:44 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/BaseAdaptor.pm: allow JOIN to
appear in the constraint and act wisely - so we do not need extra
complicated syntax for joining
2012-09-07 11:20 lg4
* modules/Bio/EnsEMBL/Hive/Extensions.pm: not ready yet to scrap
the "Runnable" support
2012-09-07 10:29 lg4
* modules/Bio/EnsEMBL/Hive/Worker.pm: (patch offered by Matthieu)
Allow the Job to kill the Worker even on succecss
2012-09-05 15:07 mm14
* modules/Bio/EnsEMBL/Hive/RunnableDB/DatabaseDumper.pm: Updated
the list of eHive tables
2012-09-05 15:00 lg4
* modules/Bio/EnsEMBL/Hive/: Queen.pm, DBSQL/AnalysisJobAdaptor.pm,
DBSQL/AnalysisStatsAdaptor.pm: these methods are already defined
in the parent class
2012-09-05 10:33 lg4
* modules/Bio/EnsEMBL/Hive/: AnalysisStats.pm,
DBSQL/AnalysisJobAdaptor.pm, DBSQL/AnalysisStatsAdaptor.pm:
trimmed the commented-out 'use' statements
2012-09-04 17:07 lg4
* modules/Bio/EnsEMBL/Hive/Extensions.pm: slimmed down the
Extensions module a bit (valuable code already moved into
Hive::Analysis)
2012-09-04 17:02 lg4
* docs/hive_schema.mwb, docs/hive_schema.png,
modules/Bio/EnsEMBL/Hive.pm,
modules/Bio/EnsEMBL/Hive/Analysis.pm,
modules/Bio/EnsEMBL/Hive/AnalysisCtrlRule.pm,
modules/Bio/EnsEMBL/Hive/AnalysisStats.pm,
modules/Bio/EnsEMBL/Hive/DataflowRule.pm,
modules/Bio/EnsEMBL/Hive/Process.pm,
modules/Bio/EnsEMBL/Hive/Queen.pm,
modules/Bio/EnsEMBL/Hive/Worker.pm,
modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisAdaptor.pm,
modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm,
modules/Bio/EnsEMBL/Hive/DBSQL/BaseAdaptor.pm,
modules/Bio/EnsEMBL/Hive/PipeConfig/HiveGeneric_conf.pm,
modules/Bio/EnsEMBL/Hive/Utils/Graph.pm, scripts/cmd_hive.pl,
scripts/lsf_report.pl, sql/foreign_keys.mysql,
sql/patch_2012-09-04.sql, sql/procedures.mysql,
sql/procedures.sqlite, sql/tables.sql, sql/tables.sqlite:
substituted the overloaded legacy 'analysis' table by a slimmer
'analysis_base'
2012-09-04 10:09 lg4
* scripts/beekeeper.pl: actually switch to using rc_name in the
workers commandline
2012-09-03 12:26 lg4
* scripts/beekeeper.pl: make sure beekeeper reports the same
scheduling plans both when it is actually scheduling and in
"reporting" mode
2012-09-03 12:23 lg4
* scripts/lsf_report.pl: adding rc_name to the lsf_report
2012-09-03 12:21 lg4
* modules/Bio/EnsEMBL/Hive/Meadow/LOCAL.pm,
modules/Bio/EnsEMBL/Hive/Meadow/LSF.pm, scripts/beekeeper.pl:
rc_name support in the beekeeper
2012-09-03 12:20 lg4
* modules/Bio/EnsEMBL/Hive/Queen.pm, scripts/runWorker.pl: rc_name
support in the Worker
2012-08-29 09:51 lg4
* modules/Bio/EnsEMBL/Hive/Params.pm: a typo in perldoc
2012-08-28 10:17 lg4
* modules/Bio/EnsEMBL/Hive/: DBSQL/BaseAdaptor.pm, Queen.pm: this
diagnostic information is no longer needed
2012-08-28 10:05 lg4
* modules/Bio/EnsEMBL/Hive/Queen.pm: Scheduler should explain that
workers are not added because of the pending ones
2012-08-27 14:10 mm14
* modules/Bio/EnsEMBL/Hive/RunnableDB/DatabaseDumper.pm: eHive
tables are always included unless exclude_ehive is defined
2012-08-25 21:09 lg4
* modules/Bio/EnsEMBL/Hive/PipeConfig/HiveGeneric_conf.pm: make
sure default is in the beginning of the list
2012-08-25 10:58 mm14
* modules/Bio/EnsEMBL/Hive/RunnableDB/DatabaseDumper.pm: Updated
the eHive table list + fixed typo
2012-08-24 15:49 lg4
* modules/Bio/EnsEMBL/Hive/: AnalysisJob.pm, DataflowRule.pm,
Process.pm, Queen.pm, ResourceClass.pm, Worker.pm: inherit
Job,Worker,DFR,RC from Bio::EnsEMBL::Storable, reuse some code
2012-08-24 15:44 lg4
* sql/tables.sql: starting the rel69...
2012-08-24 14:38 lg4
* modules/Bio/EnsEMBL/Hive/Worker.pm: bugfix: make sure there is at
least a number in the query (reported by Matthieu)
2012-08-23 12:01 lg4
* modules/Bio/EnsEMBL/Hive/: ResourceClass.pm,
ResourceDescription.pm: renamed to_string into toString for
uniformity
2012-08-23 10:45 mm14
* modules/Bio/EnsEMBL/Hive/RunnableDB/DatabaseDumper.pm: Added a
"skip_dump" parameter to ease the restoration of a dump
2012-08-17 15:52 lg4
* modules/Bio/EnsEMBL/Hive/Worker.pm: trying to be more careful
with references; release jobs' parameters earlier
2012-08-16 12:16 mm14
* modules/Bio/EnsEMBL/Hive/RunnableDB/DatabaseDumper.pm: Can copy a
database to another database
2012-08-16 12:12 lg4
* modules/Bio/EnsEMBL/Hive/: Process.pm, Worker.pm: change
suggested by Matthieu to avoid crashing if the temp_directory has
already been deleted by Runnable
2012-08-14 11:57 lg4
* modules/Bio/EnsEMBL/Hive/Queen.pm,
modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisStatsAdaptor.pm,
modules/Bio/EnsEMBL/Hive/PipeConfig/HiveGeneric_conf.pm,
scripts/beekeeper.pl, scripts/cmd_hive.pl,
scripts/generate_graph.pl, scripts/runWorker.pl: switch to
module->new() notation everywhere, to simplify text searches
2012-08-03 16:31 lg4
* scripts/ehive_unblock.pl: no longer used as individual jobs are
no longer specifically blocked
2012-08-03 16:22 lg4
* modules/Bio/EnsEMBL/Hive/Process.pm: removed honeycomb support
because it is no longer used by Compara modules
2012-08-03 10:36 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/JobMessageAdaptor.pm: hopefully
will fix the "was not locked with LOCK TABLES" error message
---------------------[before EnsEMBL rel.69]----------------------------------------------------------------
2012-08-01 14:23 lg4
* scripts/: runWorker.pl, standaloneJob.pl: removed the alternative
"nowrite" spelling to simplify interface
2012-07-31 17:01 lg4
* modules/Bio/EnsEMBL/Hive/Process.pm,
modules/Bio/EnsEMBL/Hive/Worker.pm, scripts/standaloneJob.pl:
made it possible for a standaloneJob to provide Runnables with a
functional worker_temp_directory()
2012-07-31 16:15 lg4
* modules/Bio/EnsEMBL/Hive/Worker.pm: moved life_cycle() from
Worker.pm into Process.pm and now also calling it from
standaloneJob.pl (actually removed from Worker)
2012-07-31 16:13 lg4
* modules/Bio/EnsEMBL/Hive/Process.pm,
modules/Bio/EnsEMBL/Hive/Worker.pm, scripts/standaloneJob.pl:
moved life_cycle() from Worker.pm into Process.pm and now also
calling it from standaloneJob.pl
2012-07-25 16:30 lg4
* modules/Bio/EnsEMBL/Hive/Worker.pm: only add partial timers'
measurement if the job completed successfully
2012-07-24 16:48 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm: if running
a worker with a specific job_id, the status is set to READY, but
the retry_count is set depending on whether PRE_CLEANUP is needed
or not
2012-07-24 16:17 lg4
* modules/Bio/EnsEMBL/Hive/Process.pm,
modules/Bio/EnsEMBL/Hive/Worker.pm,
modules/Bio/EnsEMBL/Hive/RunnableDB/FailureTest.pm,
sql/patch_2012-07-23.sql, sql/tables.sql: added two states,
PRE_CLEANUP (conditional) and POST_CLEANUP (unconditional) to the
life cycle of the Job
2012-07-23 16:49 lg4
* modules/Bio/EnsEMBL/Hive/Worker.pm,
modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm,
modules/Bio/EnsEMBL/Hive/RunnableDB/FailureTest.pm,
scripts/standaloneJob.pl, sql/patch_2012-07-22.sql,
sql/tables.sql, sql/tables.sqlite: At last rename GET_INPUT into
FETCH_INPUT for consistency between the schema and the code (it
seems to be harder to patch all the accumulated code)
2012-07-23 12:13 lg4
* modules/Bio/EnsEMBL/Hive/Queen.pm,
modules/Bio/EnsEMBL/Hive/Worker.pm, scripts/beekeeper.pl,
scripts/runWorker.pl: added -compile_modules_once flag to test
the new (slightly faster and more logical) approach
2012-07-16 17:54 mm14
* scripts/lsf_report.pl: rc_id renamed to resource_class_id
2012-07-03 12:06 lg4
* modules/Bio/EnsEMBL/Hive/DBSQL/AnalysisJobAdaptor.pm,
scripts/beekeeper.pl: fixed reset_failed_jobs/reset_all_jobs and
removed remove_analysis_id