-
Notifications
You must be signed in to change notification settings - Fork 0
/
RELEASE_NOTES
4148 lines (3124 loc) · 157 KB
/
RELEASE_NOTES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
RELEASE NOTES
DIABLO NEWS TRANSIT AND READER SYSTEM
http://www.openusenet.org/diablo/
news.software.nntp
Known Issues:
- OLD: dreaderd: problem with corrupt header files caused
by what seems to be a lock race condition with 'dexpireover -R'.
This problem has existed for years.
Experimental (features/utilities that are new and/or not properly tested
and hence should be used with care on production systems):
- Compressed spools
- Compressed feeds
- IPv6 support
- dnewsfeeds %include option
- dexpirecache
- dcancel
Summary of changes likely to impact an upgrade from 5.x
- new dreader header index format. Cannot go back to
older versions as they will not be able to read
the new version.
Summary of changes likely to impact an upgrade from 4.x
- history version 2 (offsets converted to indexes)
- new history expire (dhisexpire)
- fixed bug in dexpire.ctl inheritance handling
Summary of changes likely to impact an upgrade from 3.x
- New spool storage format. Each article is stored with a binary
header. This new format is not usable with older versions of
diablo, although 4.x can read prior storage formats.
Diablo 6-current
2017-09-01
* Fixed broken spool article purging in dcancel
On behalf of: Robert J Adams
Submitted by: Joe Greco
2014-04-21
* Add IPv6 support for vserverlist
Submitted by: Francois Petillon <fantec @ proxad.net>
2014-04-21
* Update error message for newsgroup listing in dreaderd
Submitted by: julien @ trigofacile.com
2014-04-13
* Merge the third of a number of patches from
Miquel van Smoorenburg <mikevs @ xs4all.net>
- 30_diablo-CUR-diablo-allocstrat-weighted.patch
New allocstrat: "weighted", to spread the articles over
different spools relative to the spool size
- 31_diablo-CUR-dreaderd-prealloc.patch
Use disk preallocation for the overview database,
prevents fragmentation. The implementation is just done
for Linux, but can easily be added for other OS's if
they offer this feature.
- 32_diablo-CUR-nostatic.patch
Don't compile diablo and dnewslink statically .. it's 2010
- 33_diablo-CUR-ipv6-enable.patch
Enable IPv6 by default .. it's 2010
2014-04-13
* Merge the second of a number of patches from
Miquel van Smoorenburg <mikevs @ xs4all.net>
- 20_diablo-CUR-large-fdset.patch
Increase the number of filedescriptors we can select()
on with glibc
- 21_diablo-CUR-dreaderd-limit-pipelining.patch
Dreaderd allows pipeling with a large limit (one read
buffer, 64K bytes). If a client sends "BODY \r\nBODY
\r\n" with hundreds of articles, and the connection to
the client is slower than the connection to the backend
(which it always is), dreaderd will download and buffer
all of those articles in memory. I've seen dreaderd
swell up to gigabytes because of this. This patch makes
dreaderd process the next command from the input buffer
only after the output buffer gets emptied.
2014-04-13
* Merge the first of a number of patches from
Miquel van Smoorenburg <mikevs @ xs4all.net>
- 01_diablo-CUR-64bit-fixes.patch
Fixes remaining 64 bit issues - mostly debug printf's
- 02_diablo-CUR-large-artno-fixes.patch
Fixes dgrpctrl and dreaderover output for long article
numbers.
- 03_diablo-CUR-ipv46-dns.patch
- 04_diablo-CUR-dreaderd-msg.patch
If SendMsg() is used without filedescriptor-passing,
do not attach a zero-length control message.
- 05_diablo-CUR-zlib-lfs.patch
zlib.h doesn't want to compile if _FILE_OFFSET_BITS ==
- 06_diablo-CUR-glibc-strptime.patch
Add strptime() prototype - not standard in glibc headers
- 07_diablo-CUR-dreaderd-linux-db185.patch
Linux needs a compatibility include if the db 1.85 API
is used
- 08_diablo-CUR-listen-v4-v6.patch
On current (e)glibc versions getaddrinfo() returns the
wildcard IPv4 address before the wildcard IPv6 address,
which means that we don't get a shared IPv6/IPv4 socket
anymore. It never worked on *BSD anyways (where IPV6_V6ONLY
is the default). Fix it up correctly.
- 09_diablo-CUR-zalloc-fixes.patch
Fix a memory leak in nzalloc(). When adding an extra
mempool to the list of pools, the original mempool was
lost subsequently never freed by freePool(). o Fix a
wrong assumption in zallocStrTrim2(): you cannot shorten
a string from zallocStr by inserting a '\0', because
strlen() is used by zfreeStr() to find out the number
of bytes to free.
- 10_diablo-CUR-dreaderd-groupmap-memleak.patch
ReadAccessCache() forgot to set accessmap.GroupMapSize,
so that xunmap() failed on it - it never got unmapped.
- 11_diablo-CUR-dreader-sr_cconn-null.patch
When a spoolserver dies, don't requeue pending requests
for clients that have already disconnected. This makes
dreaderd forks much more longlived, so do not apply
this without the memory leak fixes (zalloc-fixes and
dreaderd-groupmap-memleak)
- 12_diablo-CUR-dreaderd-fast-connlost.patch
When the remote side closes the connection, write()
returns a write error (which we mostly ignore) but after
that select() never tells us that the socket is writable.
There is a check for this condition in the reader.c
mainloop but it might take long before it's noticed.
Explicitly check for write() errors and kill the session
if so.
Diablo 5-current
2009-05-25
* dreaderd: Don't know why no one ever noticed that tcp
socket options per user weren't getting set right.
2009-04-20
* dcancel: Add the ability to cancel articles by article
number in the reader header index. (from Michael Dorrington)
2009-04-18
* Oops, never committed the stunnel patch for useproxied.
2009-02-01
* parsedate wouldn't deal well with certain days of the
week that weren't capitalized
2009-01-28
* Cleanup of various code problems, including new expire
cleanup, spaces-in-Newsgroups cleanup, etc.
2009-01-15
* Implement alternate article numbering views for clients
(this version still needs some cleanup). Clients may
indicate their capabilities via the XNUMBERING extension.
RFC977 imposes no real limits on article numbering. This
may cause some issues with non-64-bit-clean clients.
RFC3977 imposes a 2^31-1 limit AND places restrictions
on the resetting of the low-water mark, which is excessively
retarded because it effectively means that once a group hits
the limit, it's just dead, but it's the behaviour that is
defined in the RFC.
WINDOW guarantees compatibility with non-64-bit-clean
clients by restricting the article numbers to a window
between 1 and 2^31-1; further, the number of articles in
a group is limited to guarantee that most clients should
detect it as a "group reset" along the various lines of
ill-defined logic often used by clients. It is not a
frequent event, but when it happens, this has the best
possibility of being successful without being too
disruptive.
Another patch shortly will clean this code up, and will
allow for selection of which view to present by default
to clients who do not explicitly request a particular
view.
2008-12-24
* Absolutely huge changes that have been underway for more
than half a year.
There are a set of interrelated but different problems
going on in the system.
The first is that the legacy overview code was not 64-bit
safe; this effectively limited overviews for a group to
around 60 million articles and then wackiness happened.
The code has been walked through and updated so that on
64-bit platforms, larger overviews are possible (we've
exercised it to 500 mil). 32-bit platforms currently lose
due to the inability of the operating system to map large
files, but support for this is also possible and a cleanly
coded solution may be committed in the future.
Further, article numbering is a disaster waiting to happen.
The legacy code is organized to handle 31-bit message
numbers. Certain alt.binaries high volume groups are
poised to pass message number 2^31-1. Now, on one hand,
the problem could be solved by simply 64-bit-ifying the
code to handle 63-bit article numbers. On the other hand,
some nimrods decided to specify 2^31-1 as a maximum article
number in RFC3977, which, fortunately, we don't really
claim compliance with. This code revision simply extends
the article numbering to 2^63. In doing so, it will update
your overview files from VERSION 3 to VERSION 4. This is
a harmless change except for groups where article numbering
exceeds 2^31-1. Some effort was put into keeping the files
otherwise-compatible with older code.
Note that if you are using Diablo as an article numbering
master, you will need to update Diablo AND dreaderd.
People running this codebase will be able to have larger
overviews and won't run into a problem at artno 2^31-1.
2008-12-22
* Various compiler warning cleanups
2008-12-19
* Runaway spool thread status flag repurposing, based on a
strategy POC'd by tim@euroaccess (thanks for the headaches,
grin)
2008-12-18
* Various compiler warning cleanups
2008-12-18
* It's 2008. Everyone has zlib. If I don't hear objections, I
will probably be slowly removing conditional support for it.
2008-12-18
* Add error checking for group resizing; many platforms limit
calloc memory availability and application of an "unlimit" or
equivalent in the daily.reader script might be required. This
just adds a reasonable error message
2008-12-18
* Add compressed header transmission to clients (XZVER support)
Newshosting and a few other places have been running this for
almost a year without any complaints, but it is still in
testing.
2008-12-16
* drgpctl masscancel: support cancellation of large numbers of
articles from very large groups (one-pass through the overview
file)
2008-12-15
* Dump threads status from sigUsr1 (can't find who submitted this
patch, apologies, whoever)
2008-12-15
* Report virtual server name in log lines
2008-12-15
* Bump max TCP buffer size to 2048K
2008-12-06
* fwrite patch to allow for proper bytes accounting in Diablo
2008-08-01
* Patch from Avi, reworked, to provide coredumps on Linux
2008-04-02
* I didn't realize this hadn't actually made it into dreaderd. A
patch to use new CMSG macros for file descriptor passing, needed
to make most 64-bit architectures happy.
2008-03-30
* I've thrown support for UseProxied into dreaderd, but it is not
complete or correct at this time. Using it will probably result
in a file descriptor leak. Anyone interested in adding support
for SSL via stunnel, and who's interested in hacking on the code
a bit, can contact me for further information on how this works.
2008-02-28
* Fix Solaris compatibility issue; looks like they now use u_32_t
instead of u_int32_t
2007-11-30
* Tagged 5.1-REL
Diablo 5.1-REL
2007-11-16
* dreaderd: Fix a posting corruption bug that apparently has
always been there (sigh).
2007-11-05
* dreaderd: Add support for Maximum Age Policy
* diablo: Add support for Maximum Age Policy
This feature controls the maximum article age that can be
retrieved from a spool.
2007-09-11
* dreaderd: Add some debugging information to dns.c to log
label failures more clearly
2007-07-11
* dreaderd: Cleanup old references to POST_HIDENNTPPOSTHOST
Cleanup Linux CFLAGS
Thanks to: Michael Dorrington <[email protected]>
2007-01-03
* dreaderd: Minor programming style fix; "improper" use of zfree.
2006-09-21
* diablo: Separate out IFILTER hash based Message-ID refusals;
these were getting lumped in with history refusals.
2006-08-21
* drequeue: Fix five-year-old command line parsing bug
2006-06-29
* diablo: Log something in the incoming log when an IFILTER hash
rule is tripped.
2006-05-18
* diablo: Clean up a spool bug
From: Koos de Haan <[email protected]>
After some investigation i found that the cause lays in the fact we
are using an unwanted metaspool without any spools in it. This to
dump articles in groups we don't want to carry (guess this it rarely
used since i am the first to hit this bug)
2005-12-28
* dreaderd: Introduce BIG_MBUF define for dreaderd.
Large sites with memory to burn may benefit from defining BIG_MBUF
in vendor.h. This will allow more buffering of data and reduced
syscall load.
2005-12-26
* dreaderd: Add an ART_AGE option for dreaderd as well.
STATS_ART_AGE affects the Diablo spooler, and reports the approx.
time since the article was received by the spooler, in seconds.
This is only accurate to within ten minutes due to the data
available from the history.
STATS_ART_AGE_DR affects the dreaderd system, and reports the
seconds between the current time and the time in the Date:
header of retrieved articles coming from spool servers.
2005-11-01
* IMPORTANT: The hashfeed system has changed. The first few
iterations may be painful on non-FreeBSD platforms as I have
few alternative platforms to test on.
Hashfeed has been greatly enhanced to use MD5 based hashing,
which allows a much greater mod value, but also allows multiple
tiers of hash support. More details in hashfeed.c for the
interested. See samples/ for changes.
If you use hashfeed for deterministic article routing and
location, YOU WILL NEED TO CHANGE YOUR CONFIG FILES. The
old hash values are still supported, with a different syntax.
2005-10-23
* dspoolout: For reasons unknown, the nobatch option never made
it from the old dnntpspool.ctl format to the new unified
dnewsfeeds file format.
2005-10-16
* diablo: Add feedertxbuf/feederrxbuf to dnewsfeeds.
Set the TCP buffer sizes for this connection, overriding what
was set globally on the command line. This would seem to be a
useful optimization especially on spool servers, where the
inbound feed ought to have a large rx buffer/small tx buffer,
but the spool connections should be just the opposite.
2005-10-16
* Correct more examples of not checking return values from
system calls. In this case, diablo silently ignores failure
of setsockopt calls to set buffer sizes, something which can
easily fail.
2005-10-10
* Pass remote IP data via NetRemote. JG20050804
2005-02-20
2005-07-30
Patch to line 1092 of diablo.c - turn off spurious reports of
closing the external filter - if in a child process - the child
process does not interact with the filter in that way.
Patch from mohan
2005-02-20 through 2005-06-04
* lots of patches were done and not documented, blame Mike.
2005-02-20
* diablo: Ben O'Hara found a place we didn't use the %include
code, reasons unknown.
2005-02-17
* diablo: #ifndef-wrap MAXFORKS/MAXFEEDS. We've had this little
change for a long time and I'm not sure how it never got
committed. This allows you to override these at the top of
vendor.h, along with FD_SETSIZE, if you want to make this sort
of compile-time change. (It worked to do this without this
patch, but it generated compiler warnings...)
2005-01-16
* dgrpctl: Add an option to specify an absolute new
value for NB, NE or NX. Improvements to usage
description and man page.
Submitted by: Frank Durda IV <uhclem.diablo at nemesis.lonestar.org>
2005-01-12
* diablo: Fix a possible buffer overflow in the handling
of dspoool.ctl time handling when the time type is not
specified.
Reported by: Dave Reader <dave @ undone.org.uk>
2005-01-12
* dreaderd: Add a diablo.config option 'readeridenttimeout'
to specify the default timeout to be used for ident requests
if enabled. Can also use an authdef override. Also fix a
buffer handling problem in the returned user.
Reported by: Kristofer Pettijohn <kristofer+devnull @ cybernetik.net>
2004-12-28
* dreaderd: Don't let rate limits fall below 100cps. If you want
someone to read news on a teletype, buy them a teletype. :-)
2004-12-27
* dreaderd: Allow per-readerdef setting of rxbufsize/txbufsize.
Recent changes seem to have broken the command line options, and
cmdline would not seem to be the best way to do that anyways, so
at some point we should probably retire -R/-T.
2004-12-26
* dreaderd: Add "ratelimitrange", a way to randomly vary the rate
limit of a session. Set a range, like "80-120", and the otherwise
applicable rate limit will be varied from 80% to 120% of that
number. Minimum is 1%, maximum is 200%.
2004-12-25
* dreaderd: Silent capping of TCP buffer sizes is bad policy.
Make it log a warning. Also apply capping policy to spool
server connections.
2004-12-24
* dreaderd: Allow generating Xref: lines for articles with
whitespace in Newsgroups: header.
Submitted by: Cor Bosman <cor @ xs4all.nl>
2004-12-22
* dreaderd: Fix a problem with dreaderd crashing when it
encounters a very large (usually corrupt) header. Also
decrease the maximum header size (and control article
size) that dreaderd accept to 512K by default.
2004-12-22
* dreaderd: Allow the initial numarts for a group index
file to be any size the admin wants. We used to limit it
to 65536 arts
2004-12-22
* diablo: When choosing the first spool object, pick one
randomly so that we don't overallocate the first one
2004-12-14
* Add feedgraph to the contrib/ directrory. This program
creates a set of web pages of RRD graphs for feeds.
Submitted by: Kristofer Pettijohn <kristofer @ cybernetik.net>
2004-12-12
* dexpireover: Fix a bug introduced on 2004-10-15 that
was causing dexpireover to use an incorrect file
size when rewriting data.
Submitted by: Cor Bosman <cor @ xs4all.nl>
2004-12-11
* dreaderd: Add a dexpire.ctl option (l) to set the
oldest articles that dreaderd will fetch from a
spool server.
Submitted by: Miquel van Smoorenburg <miquels @ cistron.net>
2004-10-31
* dreaderd: Improvements to MADVISE code
Submitted by: Francois Petillon <fantec @ proxad.net>
2004-10-19
* dreaderd: Add a lib/vendor.h option 'USE_OVER_MADVISE'
to enable the MADVISE of XOVER output as it has caused
problems for some people.
2004-10-15
* feeder-stats.pl: Fixes and ehancements to feeder-stats
including reader stats and text output option.
Submitted by: FUNET Newsmaster <newsmaster @ news.funet.fi>
and Mike Horwath <drechsau @ gmail.com>
2004-10-15
* dreaderd: Add 2 new options to dserver.hosts:
retention=<time_in_second>
groupdef=<list_of_groupdef>
Submitted by: Francois Petillon <fantec @ proxad.net>
2004-10-15
* dreaderd: Revisit using MADVISE for XOVER output and
do some sanity checking of dataentries.
Submitted by: Francois Petillon <fantec @ proxad.net>
2004-10-15
* dexpireover: Fix a few bugs when using -R option
Submitted by: Francois Petillon <fantec @ proxad.net>
2004-09-26
* dreaderd: Add support for banning a user through the existing
custom error propagation facility. Now you can return an auth
message like "110 600 <#mins> 502 Buzz off."
The custom message will only be displayed to the current session.
Aside from the minutes, the normal readerban setup rules apply,
and you must have a readerban set for per-host bans. It might
be nice to separate this functionality at some point, by creating
an AuthBan structure.
2004-09-25
* dreaderd: Add new dserver.hosts flags to allow sleepy, sluggish,
and/or nonresponsive spool servers to be logged or have
connections reset.
2004-09-19
* dexpire: Add an extension to the '-s' (sync filesystem) option
to specify how many files to delete before syncing.
2004-08-20
* dreaderd/dexpireover: Fix a long running bug that caused dexpire.ctl
entry inheritance to not work. WARNING: This may affect exiting
dexpire.ctl setups - check that your logic is what is expected.
Pointed out by: Miquel van Smoorenburg <miquels @ cistron.net>
2004-08-20
* all: Allow setting of zero for rxbuf and txbuf to use
dynamically sized buffers for any OS that allows it.
Set defaults for dreaderd to zero.
Submitted by: Miquel van Smoorenburg <miquels @ cistron.net>
2004-08-07
* diablo: Structures should be zeroed upon allocation. Failure
to do so results in all sorts of unexpected behaviours when
someone adds a nifty new feature but isn't thorough about it,
thus causing all feeds generated from a legacy dnntpspool.ctl
based system to be started with a nice priority of 20.
2004-07-30
* dreaderd: Undo the XOVER performance patch - it causes dreaderd
to go into a spin loop under certain circumstances.
2004-07-28
* diablo: Clean up some of the various time_t-to-int printf issues
and other similar cleanups.
2004-07-27
* dreaderd: Add new option 'settos' to set the IP_TOS
setsockopt option, if available, to set TOS/QoS data
in the IP packet header.
2004-07-20
* dreaderd: Improve performance of XOVER
Submitted by: Francois Petillon <fantec @ proxad.net>
2004-07-20
* dreaderd: Use internal md5 function so that hashed
NNTP-Posting-Host headers are enabled with the cryptpw
option in dreader.access.
2004-07-20
* diablo: Add support for hsize up to 512m entries. Many
operating systems won't be able to handle more than 256m
due to mmap() limitations.
2004-07-11
* dexpireover: Don't delete corrupt data entries by default.
Use '-RR' to remove corrupt entries.
Submitted by: Francois Petillon <fantec @ proxad.net>
2004-07-11
* dexpireover: Do some sanity checking on the value of
dataentries in case it has become corrupt. This fixes
some of the corrupt overview problems.
2004-07-10
* diablo: Allow activedrop to be on without active on.
Submitted by: Alex Kiernan <alex.kiernan @ gmail.com>
2004-07-10
* doverctl: Fix relative path calculation to prevent valid files
being deleted with the 'clean' option.
2004-04-28
* diablo: Add option in diablo.config to reject articles with
a CR with no corresponding LF.
2004-04-28
* dnewslink: Allow sending to both IPv4 and IPv6 destinations
if both are in the DNS.
Submitted by: FUNET Newsmaster <newsmaster @ news.funet.fi>
2004-04-20
* dcancel: Add the ability to cancel articles by article
number in the reader header index.
2004-04-20
* dexpireover: Don't resize dataentries unless we are
rebuilding the data.
2004-04-05
* Disable AIO by default for now. It has caused too many
reported problems. It is used when dreaderd tries to access
articles from a locally mounted spool.
2004-04-04
* diablo: Unmap some a memory mapped area of history before
closing the history file. This fixes a problems where long
running diablo process were holding old history files open
after they were deleted.
2004-03-03
* diablo: Added a new option 'feedermaxheadersize' to diablo.config
with a default of 64k to limit the size of article headers that we
accept. Also reject articles with no blank line after the header.
Based on a patch from: Frank Durda IV <uhclem.diablo at nemesis.lonestar.org>
2004-02-22
* dreaderd: Add a new dexpire.ctl option 'e' to specify the
number of headers stored in each header data file. See
dexpire.ctl for details.
2004-02-14
* Tagged 5.0-REL
Diablo 5.0-REL
2004-01-30
* dreaderd: Handle custom reject messages from external auth
by returning a readerdef in the format "nnn message" (i.e.
the entire returned string would be "110 502 Buzz off").
2004-01-22
* dreaderd: Fix improper number of parameters bug
Special thanks to: Horwath!
2004-01-18
* dreaderd: Add dreaderd cache scoring to weight caching in favour
of popular groups.
Submitted by: Francois Petillon <fantec at proxad.net>
2003-12-07
* Some minor changes to sample cron scripts, including switching
to use dhisexpire by default.
2003-12-03
* dreaderd: Fix AIO access for local spool under Linux
Submitted by: "Homme R. Bitter" <homme at cambrium.nl>
2003-11-25
* dreaderd: Log article retrieval failures better (i.e. 423's).
2003-11-25
* dreaderd: New option, addrealm, to tack on a realm to the user-
supplied authinfo. Useful with virtual servers and complex
environments.
2003-10-28
* dreaderd: Fix XPAT problems relating to high CPU use by dreaderd.
Submitted by: Branko Lankester <branko at euro.net>
2003-10-08
* dfeedtest: Add an option '-X' to show which spool an article would
be installed into. Also add '-M' to specify the Message-ID.
2003-09-07
* diablo: Read the manner in which the article was stored from the
article header. The previous code made it impossible for a spool
to be compressed and then set to uncompressed, despite the fact
that we ultimately read the data that told us the article had been
stored uncompressed...
2003-08-27
* diablo: check the return value on the kill system call. Failures
may be harmless, but then again they could signal a problem.
2003-08-01
* dreaderd: bump the default THREAD_QSIZE up to 16. Please report
any problems you encounter as a result of this change.
2003-07-31
* dreaderd: post.c: silent dropping of obviously bogus and forged
headers is a *really* bad idea. I don't know why I ever did it.
Furthermore, I missed X-Complaints-To, which resulted in a bunch
of bad stuff getting out on the 'net with multiple XCT's by a
spammer intent on confusing the issue.
Fixed by: JG
2003-06-01
* dhisexpire: Fix bug in pointer use that could cause all entries
to be removed.
2003-05-30
* dnewslink: Fix unmapping of articles bug introduced in the
recent performance enhancing patch.
2003-05-28
* dsyncgroups: Add options to specify a username/password of the
remote system.
Submitted by: "Robert J. Adams (jason)" <radams @ siscom.net>
2003-05-25
* dexpirecache: Fix a missing closedir() and significantly
reduce memory usage by doing a chdir() to the basepath, thus
not storing the basepath in each node.
Submitted by: Michael Haardt <michael @ freenet-ag.de>
2003-03-29
* dspoolout: Don't rename a queue file if its size is zero
2003-03-15
* dreaderd: Add some extra logging to 'info server' log line
Submitted by: Robert Kiessling <Robert.Kiessling @ de.easynet.net>
2003-03-14
* dreaderd: Don't accept bad Message-ID's in TAKETHIS only mode
2003-03-02
* diablo: Switch to history version 2 enabled by default.
The new version allows history files > 4GB (up to approx
60GB). The conversion to the new history file is done
during the next history rebuild.
2003-03-02
* dreaderd: Allow use of more characters in newsgroup
descriptions.
Submitted by: Xander Jansen <x+diablo @ surfnet.nl>
2003-03-02
* dnewslink: Let the kernel handle copying of large files
to the network from disk by using a writev(), rather
than copying the data into buffers first.
Submitted by: FUNET Newsmaster <newsmaster @ news.funet.fi>
2003-03-02
* dnewslink: Add new option 'settos' to set the IP_TOS
option (if supported by the OS). This adds quality of
service option for outgoing feeds on networks that
support it.
Submitted by: FUNET Newsmaster <netmgr @ funet.fi>
2003-03-02
* diablo: Add a new dnewsfeeds options 'offerfilter' and
'noofferfilter' to filter out Message-ID's during the
IHAVE/CHECK processing.
Submitted by: tt @ efnet.com
2003-03-02
* diablo: Add feeddelany for remote feed admin
Submitted by: tt @ efnet.com
2003-03-02
* diablo: Fix minor bug for remote feed admin
Submitted by: tt @ efnet.com
2003-03-02
* feeder-stats: Various updates for Diablo 5.x. See the changelog
in feeder-stats.pl for details.
Submitted by: Xander Jansen <Xander.Jansen @ surfnet.nl>
2003-02-15
* all: Limit the maximum Message-ID size to 250 (was 512) to
be in line with INN and Usefor NNTP article draft.
2003-02-15
* dreadover: Change the format of the output and provide more info
about the article (Message-ID hash, size, and time received).
2003-02-15
* dreaderd: Ignore newsgroups with names that are too long
and don't accept local posts to such newsgroups.
2003-02-09
* dreaderd: Fix bug in user based banlists where the username was
being banned instead of the IP address.
2003-02-04
* dicmd: report status on ps/proctitle
2003-02-02
* dcancel: Add a new utility to allow rewriting of spool files
along with history entry updates to remove articles from a
spool. Compressed spools are not yet supported.
2003-01-25
* diablo: Use the correct variable for the article size when
using the feed and spam filters.
Submitted: by Branko Lankester <branko @ euro.net>
2003-01-23
* diablo/dnewslink: Add 2 new options feederpreloadart and
spoolpreloadart n diablo.config to disable the preloading
of articles by dnewslink and spool servers respectively
(the use of MADV_WILLNEED to the madvise() system call).
2003-01-22
* diablo/dreaderd: Set the default maximum article size accepted
by feeder and reader to be 10MB. This can be disabled by
setting {feeder|reader}maxartsize to 0 in diablo.config.
2003-01-22
* dreaderd: Fix a bug which could cause all headers for a group
to be deleted if the header index file could not be opened.
2003-01-22
* diablo: Allow multiple label options per metaspool object in
dspool.ctl
2003-01-21
* dreaderd: Option to include an X-Comments header on a per-
vserver basis.
2003-01-21
* diloadfromspool: Add two new options (-tb and -te) to specify
the start and end times to limit spool scanning.
2003-01-18
* diablo: Fix a race condition during the expire process that
was causing articles to be expired if they arrived after
a spool scan, before the end of the history update
and within a specific time frame
2003-01-17
* diablo: Fix a file descriptor leak in dnewsfeeds parsing
2003-01-16
* dexpire: Improve the performance of the history update
2003-01-15
* didump: Add an option to only dump msgid hashes
2003-01-11
* dhisctl: Fix bug in expire/unexpire option that could corrupt
history
2003-01-08
* dnewsfeeds: Now has an experimental option for doing %include-
files.
2002-12-27
* diloadfromspool/drequeue: Add support for dealing with
compressed spools.
2002-12-27
* dexpirecache: Add tree balancing code.
Submitted by: Francois Petillon <fantec @ proxad.net>
2002-12-23
* dexpirecache: Fix logic in directory open
Submitted by: Francois Petillon <fantec @ proxad.net>
2002-12-16
* dreaderd: Fix bug that was causing caching to be enabled even
if disabled in diablo.config.
2002-11-04
* dreaderd: Add a new and more flexible form of LDAP authentication
that must be enabled with NEW_LDAP in lib/vendor.h. This may
become the default in the future.
Submitted by: michael @ freenet-ag.de (based on code originally
written by Russell Vincent)
2002-11-03
* dreaderd: Improvements to the reader caching algorithms,
including a new dexpirecache program.
Submitted by: Francois Petillon <fantec @p roxad.net>
2002-11-02
* all: Limit the use of MADV_WILLNEED as this causes some
performance issues with very large files.
2002-10-24
* dreaderd: Using an uninitialized auto struct and not checking
return values makes for bad things when doing a getsockname.
Fix both problems.
2002-10-19
* dreaderd: Don't bind to port 119 if we specify a different port.
Also fixes specifing port 119 as command-line option.
2002-10-14 (merged 4-stable: : 2002-10-14)
* dreaderd: Don't duplicate the body when client retrieves
control messages.
2002-10-07
* diablo: UCAR.EDU is burdening USENET with Yet More Binaries
Volume. Unfortunately, Diablo wasn't picking it up as such.
This looked to be quite a problem for split text/binaries
sites.
http://www.unidata.ucar.edu
2002-10-06
* dreaderd: Fix a bug causing closed spool connections to be
left undetected if an article was being retrieved at the time.
2002-09-26
* diablo: Add a dnewsfeed option 'delayinfeed' to delay an
incoming feed until a number of seconds after diablo has started.
Submitted by: Jake Roersma <jaker @ tsunami.triton.net>
2002-09-25
* dreaderd: Fix bug in the ratelimit per command option that
was causing article fetches to be reset to the 'other' value.
2002-09-19
* dnewslink: Fix an obvious logic error with madvise code
Submitted by: FUNET Newsmaster <newsmaster @ news.funet.fi>
2002-09-08
* all: Enable MADVISE in Linux >= 2.4 for improved mmap
performance (reported 33% gain in IO requests).
2002-09-06
* dnewslink: Add an option to disable the CHECK command for
the realtime process only. See the 'nortcheck' option for
the "check" directive in dnewsfeeds.
Submitted by: Francois Petillon <fantec @ proxad.net>
2002-09-03
* diablo: Detect "bommanews" binary article type.
2002-08-20
* diablo: Create a new spool file when we go over a size limit
to prevent spool files getting larger than the allowed boffset
reference (spool file offset) in dhistory.
Submitted by: Francois Petillon <fantec @ proxad.net>
2002-08-17
* dreaderd: Allow different client ratelimits for various
article retrieval types (XOVER, ARTICLE, HEAD, etc).
2002-08-17
* diablo: Fix bug in external spamfilter handling which meant that
the article file could not be opened when a non-default full
path is specified in a spool entry.
2002-08-14
* Fix recently introduced bug causing master diablo process to
die when a spool disk fills up.
Reported by: Jake Roersma <jaker @ tsunami.triton.net>
2002-08-12
* all: Add Linux support for large files (-D_FILE_OFFSET_BITS=64)
This currently only applies to the dhistory file.
2002-08-12
* dnewslink: Fix bug in handling of attempts at remote connections
for hosts with IPv6 and IPv4 addresses.
2002-08-12
* dnewslink: Fix binding of outgoing interface in INET4 case.
2002-08-10
* diablo: Add metaspool allocation strategy 'single' that
allocates the same spool object to all incoming feeds
over the reallocint time period.
2002-08-10
* diablo: Add diablo.config option 'feederarttypes' which
allows the article type code to be disabled completely.
2002-08-10
* dlockhistory: Add an option to lock the entire history file
into memory.
2002-08-08
* diablo: Fix bug in history lookup offset calculation when
using history version 2.
2002-08-07
* dreaderd: Improvements to PGP verification:
- fix a bug in the handling of lines beginning with
'-' sent to PGP.