💥 Textmode improvements, probably not backwards-compatible.

README.md

@@ -1,12 +1,14 @@
-# lotw-trust
+# lotw-trust -- sign arbitrary files with your LoTW key and verify such signatures
 
 ⚠️ **WARNING** ⚠️
 
-This is highly experimental.
+This is an experimental program.
 
-Do not use this program for anything critical. Right now it's still very much an evening project waiting for feedback from people trying to use it. The text mode format in particular is probably not stable.
+Despite occasional enthusiastic comments, I have yet to see any bug reports, or even any reports from people trying to use it in the wild at all. As such, I reserve the right to change file formats if I decide this is beneficial or convenient, until version 1.0.0 strikes.
 
-Please experiment with it, that's the right word.
+As of this moment, this tool is feature complete. Now it needs to become bug-free, so please stress-test and torture it.
+
+Alternatively it could be completely forgotten as most innovations in this hobby are. It's not like radio didn't work without all that before.
 
 ## What is it?
 
@@ -52,7 +54,7 @@ As a result, if LoTW makes a new layer #2 key after you got your layer #3 key an
 
 To make matters more complicated, the #1 Big Master Key is also not eternal, and has an expiry time measured in decades -- the current one expires in 2025. It isn't signed by the key from the previous decade either, so you definitely will not be able to produce an unbroken chain of keys to known keys past 2025, when the current one expires, unless the new key surfaces in trustworthy data earlier than that.
 
-It would be a lot smoother if I can get LoTW to publish their public keys properly. Otherwise, I anticipate that `lotw-trust` will need to be updated on average no less than once a year to keep working, which will be a hassle for service owners.
+It would be a lot smoother if I can get LoTW to publish their public keys in a central trustworthy place. Otherwise, I anticipate that `lotw-trust` will need to be updated on average no less than once a year to keep working, which will be a hassle for service owners.
 
 ### Certificate revocation
 
@@ -89,7 +91,7 @@ It's possible to save the signature block to a separate file, verify such a sign
 
 If your signing situation results in bundling intermediate certificates, I would very much like to see them. The simplest way to let me do that is to send me the contents of your cache directory -- see the `-c` flag.
 
-`lotw-trust sign -t` and `lotw-trust verify -t` will treat the file as text, resulting in an ASCII-armor style file format that you could, in theory, stick into a pipeline in Winlink to automatically sign messages you send. Trying to sign binary files with this flag will produce ugly results, but will not necessarily fail.
+`lotw-trust sign -t` and `lotw-trust verify -t` will treat the file as text, resulting in an ASCII-armor style message format that you could, in theory, stick into a pipeline in Winlink to automatically sign messages you send and verify messages. Be aware that text mode signing does *not* result in a file identical to source upon verification, and is meant for direct human consumption on both ends. Trying to sign binary files with this flag will produce ugly results, but will not necessarily fail.
 
 ## Installation and compilation
 
@@ -101,12 +103,6 @@ It was written with go 1.20.5 and I currently don't know what's the minimum vers
 
 See `testcases` and `test.sh` for simple test cases and a faux-LoTW certification tree structure to play with.
 
-## Plans for future development
-
-As of this moment, this tool is feature complete. Now it needs to become bug-free and formats need to stabilize. Please stress-test and torture it.
-
-Alternatively it could be completely forgotten as most innovations usually are. It's not like radio didn't work without all that before.
-
 ## License
 
 This program is released under the terms of MIT license. See the full text in [LICENSE](LICENSE)

main.go

@@ -1,3 +1,11 @@
+/*
+Copyright (c) 2023 by Eugene Medvedev (R2AZE)
+
+Use of this source code is governed by an MIT-style
+license that can be found in the LICENSE file or at
+https://opensource.org/licenses/MIT.
+*/
+
 package main
 
 import (
@@ -16,6 +24,7 @@ import (
 	"log"
 	"os"
 	"path/filepath"
+	"runtime"
 	"strings"
 	"time"
 
@@ -31,8 +40,9 @@ var version string
 
 const minSupportedVersion = "0.0.3"
 
-const textModeHeader = "-----LOTW-TRUST MESSAGE-----\n"
-const textModeFooter = "\n-----BEGIN LOTW-TRUST SIG-----\n"
+const textModeHeader = "-----LOTW-TRUST MESSAGE-----"
+const textModeFooter = "-----BEGIN LOTW-TRUST SIG-----"
+const textModeEnding = "-----END LOTW-TRUST SIG-----"
 const textModeSigPem = "LOTW-TRUST SIG"
 
 // Binary mode header is the same with extra newlines.
@@ -179,6 +189,34 @@ func saveFile(filename string, fileData []byte) {
 	}
 }
 
+func detectNewline(text string) string {
+	// Detect a newline character based on the text file.
+	// Textmode needs to introduce newlines into the text,
+	// which might not be in the system-dominant
+	// newlines.
+
+	if strings.Index(text, "\r\n") > 0 {
+		// Contains a single windows newline? Use windows newlines.
+		return "\r\n"
+	} else if strings.Index(text, "\r") > 0 {
+		// Contains an osx newline - use osx newlines.
+		return "\r"
+	} else if strings.Index(text, "\n") > 0 {
+		// Contains a linux newline - use linux newlines.
+		return "\n"
+	}
+
+	// In case the input text contains none of that, return an os-based default.
+	switch runtime.GOOS {
+	case "windows":
+		return "\r\n"
+	case "darwin":
+		return "\r"
+	default:
+		return "\n"
+	}
+}
+
 func normalizeText(text []byte) []byte {
 	output := normalizeTextString(string(text))
 	return []byte(output)
@@ -198,17 +236,30 @@ func normalizeTextString(text string) string {
 		"\r\n", replacement,
 		"\r", replacement,
 		"\n", replacement,
-		"\v", replacement,
-		"\f", replacement,
-		"\u0085", replacement,
-		"\u2028", replacement,
-		"\u2029", replacement,
 	)
-	lines := strings.Split(replacer.Replace(text), replacement)
+
+	var textLines []string
+	for _, l := range strings.Split(replacer.Replace(text), replacement) {
+		textLines = append(textLines, strings.Trim(l, " \t"))
+	}
+
+	// All preceding empty lines and all tailing empty lines must be ignored when signing as well.
+	var tailingLines []string
+	for i, s := range textLines {
+		if s != "" {
+			tailingLines = textLines[i:]
+			break
+		}
+	}
+
 	var outLines []string
-	for _, l := range lines {
-		outLines = append(outLines, strings.Trim(l, " \t"))
+	for i := len(tailingLines) - 1; i >= 0; i-- {
+		if tailingLines[i] != "" {
+			outLines = tailingLines[:i+1]
+			break
+		}
 	}
+
 	return strings.Join(outLines, finalNewline)
 }
 
@@ -369,8 +420,7 @@ func main() {
 			// PEM to the rescue!
 
 			textData := textModeHeader
-			textData += string(fileData)
-			textData += "\n"
+			textData += "\n" + string(fileData) + "\n"
 			savingData = []byte(textData)
 
 			displayTime, _ := sig.SigningTime.UTC().Truncate(time.Second).MarshalText()
@@ -402,13 +452,17 @@ func main() {
 		fileData := slurpFile(inputFile)
 
 		var sigBlock []byte
+		var rawTextHeader string
+		var rawTextFooter string
+		var restText string
+		var found bool
 
 		if sigFile == "" {
 
 			if textMode {
 				// Text mode makes everything more complicated on read, too.
 				textData := string(fileData)
-				_, restText, found := strings.Cut(textData, textModeHeader)
+				rawTextHeader, restText, found = strings.Cut(textData, textModeHeader)
 				if !found {
 					l.Fatal("The file does not appear to be signed in text mode.")
 				}
@@ -419,9 +473,17 @@ func main() {
 				}
 				signedText := restText[:tailEnd]
 
-				fileData = []byte(normalizeTextString(signedText))
+				postSig := strings.LastIndex(restText, textModeEnding)
+				if postSig < 0 {
+					l.Fatal("Signed message appears to be missing parts of the signature.")
+				}
 
-				block, _ := pem.Decode([]byte(restText))
+				rawTextFooter = restText[postSig+len(textModeEnding):]
+
+				fileData = []byte(normalizeTextString(signedText))
+				// Amusingly, OSX line endings can also cause pem to fail to decode.
+				// So we normalize the restText before feeding it into the decoder.
+				block, _ := pem.Decode([]byte(normalizeTextString(restText)))
 				if block == nil || block.Type != textModeSigPem {
 					l.Fatal("Signature not found.")
 				}
@@ -445,7 +507,7 @@ func main() {
 		var sigData SigBlock
 		if !textMode {
 			if !bytes.Equal(sigBlock[:len(sigHeader)], []byte(sigHeader)) {
-				l.Fatal("Missing signature header, file probably isn't signed.")
+				l.Fatal("Could not find signature in file.")
 			}
 			sigBlock = sigBlock[len(sigHeader):]
 			// While we're at it, try to uncompress sig block.
@@ -566,10 +628,16 @@ func main() {
 		displayTime, _ := verificationTime.UTC().MarshalText()
 		l.Println("Signed by:", getCallsign(*cert), "on", string(displayTime))
 		if textMode {
-			textData := []byte(fmt.Sprintf("\n-----VERIFIED BY LOTW-TRUST-----\nSigned by: %s on %s",
+			newLine := detectNewline(string(fileData))
+			textData := []byte(rawTextHeader)
+			textData = append(textData, []byte("-----LOTW-TRUST SIGNED----"+newLine)...)
+			textData = append(textData, fileData...)
+			textData = append(textData, []byte(fmt.Sprintf(
+				"%s-----LOTW-TRUST VERIFIED-----%sSigned by: %s on %s", newLine, newLine,
 				getCallsign(*cert),
-				string(displayTime)))
-			fileData = append(fileData, textData...)
+				string(displayTime)))...)
+			textData = append(textData, []byte(rawTextFooter)...)
+			fileData = textData
 		}
 
 		saveFile(outputFile, fileData)

test.sh

@@ -22,12 +22,16 @@ cmp -l $SRC/sstv.jpg $DST/sstv-unsigned.jpg
 
 echo === Detached signature test: must pass.
 go run *.go sign -c $CACHE -p changeme -s $DST/sstv.jpg.sig -a $KEY $SRC/sstv.jpg 
-go run *.go verify -c $CACHE $DST/sstv-signed-unc.jpg
+go run *.go verify -c $CACHE -s $DST/sstv.jpg.sig $SRC/sstv.jpg
 
 echo === Damaged file: must fail.
 printf "00000c: %02x" $b_dec | xxd -r - $DST/sstv-signed.jpg
 go run *.go verify -c $CACHE $DST/sstv-signed.jpg
 
+echo === Damaged signature: must fail.
+printf "0000cc: %02x" $b_dec | xxd -r - $DST/sstv.jpg.sig
+go run *.go verify -c $CACHE -s $DST/sstv.jpg.sig $SRC/sstv.jpg
+
 echo === Text mode tests -- native line endings: must pass.
 go run *.go sign -t -c $CACHE -p changeme $KEY $SRC/lipsum.txt $DST/lipsum-signed.txt
 go run *.go verify -t -c $CACHE $DST/lipsum-signed.txt $DST/lipsum-unsigned.txt
@@ -39,3 +43,6 @@ go run *.go verify -t -c $CACHE $DST/lipsum-signed-mac.txt $DST/lipsum-unsigned-
 echo === Text mode tests -- dos line endings: must pass.
 unix2dos -n $DST/lipsum-signed.txt $DST/lipsum-signed-dos.txt
 go run *.go verify -t -c $CACHE $DST/lipsum-signed-dos.txt $DST/lipsum-unsigned-dos.txt
+
+echo === Text mode tests -- winlink header: must pass.
+go run *.go verify -t -c $CACHE $SRC/lipsum-winlink.txt $DST/lipsum-unsigned-winlink.txt

testcases/files/lipsum-winlink.txt

@@ -0,0 +1,75 @@
+Mid: LFC0HU68UBAY
+Body: 2138
+Date: 2023/06/22 13:59
+From: SERVICE
+Mbo: SERVICE
+Subject: Test Message
+To: N0CALL
+X-Marsprecedence: Routine
+X-Wl2kprecedence: Routine
+
+This is a signed message imitating a file that came in from Winlink.
+It should decode in a very particular way, producing an undamaged file on the other end. Body length is not handles, and I'm not sure I should..
+
+-----LOTW-TRUST MESSAGE-----
+Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Imperdiet massa tincidunt nunc pulvinar sapien et ligula. Enim facilisis gravida neque convallis a cras semper. Viverra maecenas accumsan lacus vel facilisis volutpat est. Nunc sed velit dignissim sodales ut eu sem. Tellus integer feugiat scelerisque varius morbi. Nec nam aliquam sem et tortor consequat. Urna porttitor rhoncus dolor purus. Ac ut consequat semper viverra. Amet venenatis urna cursus eget nunc scelerisque viverra mauris. Tortor condimentum lacinia quis vel eros. Tellus cras adipiscing enim eu turpis egestas pretium aenean. Duis convallis convallis tellus id interdum. Varius duis at consectetur lorem donec massa sapien faucibus et. Aenean euismod elementum nisi quis eleifend quam adipiscing. Nibh sit amet commodo nulla facilisi nullam. Sit amet purus gravida quis blandit. Nullam vehicula ipsum a arcu cursus vitae congue mauris.
+
+Dui accumsan sit amet nulla. Nunc sed blandit libero volutpat sed cras ornare arcu dui. Tellus at urna condimentum mattis pellentesque id nibh tortor. Magna sit amet purus gravida quis blandit turpis cursus in. Non curabitur gravida arcu ac tortor dignissim convallis aenean. Sed risus ultricies tristique nulla. Varius duis at consectetur lorem donec massa sapien. Lobortis feugiat vivamus at augue eget arcu dictum varius duis. Netus et malesuada fames ac turpis egestas integer. Lectus magna fringilla urna porttitor rhoncus dolor. Dignissim cras tincidunt lobortis feugiat vivamus at. Non odio euismod lacinia at quis. Praesent elementum facilisis leo vel fringilla.
+
+At tempor commodo ullamcorper a lacus. Sit amet massa vitae tortor condimentum lacinia quis vel eros. Non blandit massa enim nec. Tortor at risus viverra adipiscing at. Nulla pellentesque dignissim enim sit amet venenatis urna cursus. Eget lorem dolor sed viverra ipsum nunc. Vitae ultricies leo integer malesuada nunc vel risus commodo viverra. Accumsan in nisl nisi scelerisque eu. At quis risus sed vulputate odio ut enim blandit. Dictum varius duis at consectetur lorem donec massa. Ultrices dui sapien eget mi proin sed. Nisl vel pretium lectus quam id leo. Ipsum a arcu cursus vitae congue mauris rhoncus.
+
+Nisl suscipit adipiscing bibendum est ultricies integer quis auctor elit. Nulla porttitor massa id neque aliquam. Praesent semper feugiat nibh sed pulvinar proin gravida. Interdum velit laoreet id donec. Feugiat in ante metus dictum at tempor. Convallis posuere morbi leo urna molestie at elementum. Netus et malesuada fames ac turpis egestas integer eget aliquet. Gravida cum sociis natoque penatibus et. Amet facilisis magna etiam tempor orci eu lobortis elementum nibh. Tristique nulla aliquet enim tortor. At tellus at urna condimentum mattis pellentesque id nibh tortor. Lectus mauris ultrices eros in. Purus faucibus ornare suspendisse sed nisi lacus sed viverra tellus. Vestibulum lectus mauris ultrices eros in cursus turpis. Pharetra convallis posuere morbi leo urna molestie at elementum.
+
+Bibendum arcu vitae elementum curabitur vitae. Eu facilisis sed odio morbi quis commodo. Tristique et egestas quis ipsum suspendisse. Eu volutpat odio facilisis mauris sit amet massa vitae tortor. Ultricies integer quis auctor elit. Quis ipsum suspendisse ultrices gravida dictum. Integer malesuada nunc vel risus commodo viverra maecenas accumsan lacus. Eget duis at tellus at urna. Egestas egestas fringilla phasellus faucibus scelerisque eleifend donec pretium vulputate. Turpis egestas maecenas pharetra convallis posuere. Nunc sed id semper risus. Nunc scelerisque viverra mauris in aliquam sem fringilla ut morbi. Id consectetur purus ut faucibus pulvinar. Nisi porta lorem mollis aliquam ut porttitor leo a. Nunc aliquet bibendum enim facilisis gravida neque convallis a cras. Enim eu turpis egestas pretium aenean pharetra magna. Vitae nunc sed velit dignissim sodales.
+
+
+-----BEGIN LOTW-TRUST SIG-----
+Date: 2023-06-25T11:06:55Z
+Signer: N0CALL
+
+eNrclWlUU2cexnNvFlkSBUOgCQZSthBBuQkii6KyyKKASKQoiIAhAQxwBUIglM2I
+EVlEQSoIFQWsGJYWWQYEgYZFaEHCVimIQMGKYIs4lk2WOdTpnHbqzHyY+TTf3uec
+5z3/9/zP8/zeLxBsaCe00wTgOEFWFg4O4AkAIRf+uipSTlszTbzjcFOU43KPDE7M
+s//6Nu1y5sUoz/wVR+qzDN4vbyQXR984VCdhLgSMRlS/S3qajvazNPC3vnH6CBRI
+dMm0vdLVrVNO7LResd1am7nefsm9UFWVYvHTMLv82Y7GMc6dxh1FyRktXvcW9Z3U
+WhV0TLOfxzKvRp4fwivNvts0jv9oK1OisGbMNHdtOIUj4S5f3iffemNkwt33ujb4
++vjgvmVBorW31SRCpN1/vGsiP0sU5/P1q+/5urNRhSvpVvPJ1ppNCtb8InOBaoFX
+8kLG3sndsah8OTucuHi/gPt076ipfUX1gZjSe5VH6vofJ3HeET0J1wWeNulP/oIk
++Wapd6NOIKcgIXISEoKdt5AgAIIAAOEwsttFdqIFHADIoxHQuSa6PCSLQbqiMHjQ
+lfmbkMGCVsfoNIi6IeSwFBsfLptiEcQODWD5BFNcfHwDYIoLO9BHQHFg+/iFs+la
+kMaGVR6r6gD7nYZhLgXmUHj+bIobHBroS3GEWdzws3QGBG24kFjaB13OobBvOIsX
+AAdTrCx+u6MOkf/xZFlAWSEQ5kUcYEf6BJ0NZO9kwUGQmgqOYQjtZjDoEIPBMHJX
+wRkaQwyGyd8ldJiuCG15P1bmEOwfTLGG2XQChMfI6mEAFDC2FwCwmPfZ+dOwM7B/
+8B+GCQGN368QQCOQQmALAhICcqAQABDpbcdumsc+7C2UBG8VjPD0mcbbo7NUQiav
+PTKr0dH2dprvFe+yscy5iDpALLXPSdJdI16lQuf0ziaeismJhAhz8UOxMmkrwwet
+3Nz6QT1fLjJ3bqu42L81rxxAD+Ahl8VUWqDIZOYd+RAWN7s9fXHfyEWPaZL05Keq
+B0t/bgCUEq0ashWJqgTXYf7JU8kJWpdkJgri8ixrBQjXOokNJzf7fNPoMu/m/Sn+
+9Awcgepwzp8fcX7VhVJBZfa8Sr7y1lWEnZGqHwDCLPJoc51Ag6bkObimdYPG+VK7
+sGMlYvKkWkWa7jHjYvszm2qmqg6q5XAi7ovPfI/u6ff0M/u6fj+IBBBAgSfkAZEx
+SFfyZpQySsks4catPKd40C1056aTj+ryUjL0p3+NGRmPAiEEpL5x1ER9BCnHK/HV
+Y9Q5L3XprXs66jSIq0LnT4bR0OYNwxYAWEehkCB67J8yjPx175OSpzH15DpJfOO8
+2WE9E7u9rbLSpa+mYkjYzKZx376EQk1FlsrPePTAF1ZHdV78oLXl+t4s85Ri36XK
+ZpYZH6vsHakqO7lDi+o4npeyXkIl0K38wSrrSYO8q7ngJnWfrvwjHecMJCE51bkZ
+xxd0YqNvnnoq2WyOLwKSN0ulzeWKnKz5H/dQZeZkjZmch9s4zWkZj1btNIkrFUWf
+RWhj5BS7TpYd0uFfI1s6ue0snPDXLNS4bk0vlnzq0N567TLhvATyd7ZRO/YtrTh9
+Vy8pSN5Q8bb7blktULpwbWR1Gha+leb+2BkCcYtT20vRZoZjLXfaTTKpgDzq+IKO
+7WLNIvrcCXQYJETDkBDp9i/L3v1vyo6HFDfEJqyccxg73BdmBfAE/3sC6EPb31dR
+84MuFxjm/ZfdN6XTGUa/df//AG//iTi3XKouJXtC/Oa7Yc/edCdkBaQ5b6OF0T/W
+sxdEBj6x2reFdCCBdXd2rajRw8gl9vGFr8Tt5JdHcrRq7Bzkh0ykVgx6jlDm7VzZ
+qH6fYd3dgVRHVmKLkRaf1MVspRU8bEY9iEz2HJidcF7ouduQ3C6n5vHSz+MU1fgz
+qqYcGyyNf8J302OZCj4Kqq0w1Q3FLCuU8ual0oHKuGnWrkSFvGjZoy8fqzWzgWb4
+u+7Z+/KH8muNqOXmt5S6MCb1q+3ntyqAONyRCS3kN7TdpeGpsHrfx68NstRlStaG
+8VE61re5IGfmEfl0I8lm/zfKKsADk6UnKeWGBe0X6i+8DQII0E2/98RhQT6/I84H
+KfJ7zFQ9wkKOhJSo78gTK4Zm9mcT7Gocf/0kyPgNzKAhJACs/5E7gOhP3AEBRGRJ
+TRPmgUGk5ypOdbSZecwcJ59Stt65ddxke99Lka7GUayK43SQyCeESt3ZkehBWiY4
+uhNehTXeacEnisODbLEvsg9wdy2W4F5ozWTH1ixtJjax47hih895K7aHMKYDn6wI
+0xFsTU8GPj7Y2TjwTH0Q9aZGvRKjJ7qBshoRSC4LRuxN70s0ftWk4XlXYIOTMY0a
+TRK73xb89LhvdnA+w+EdSetBh/MW/TjLwlHRti/xOPttLfA4IX/4l4qKOG2i3ps5
++xdRVQ9tlBW/lE/fdoHLK3da/6Htev0eh+/H51N/+k5E5By9ajwy80le/J0ay8Up
+Wug9/KZTeVMNg2oH16ovJl2zOc40aVOZw5rFZiRWDuUw9Jhel3p6mdZJtl7lyvNp
+N8ay3KMrtt3zConJJvELzh1OWelb5qoyBGBvKtwUv8dAtBhbu8xkofSOl4k/rjZg
+UVue49b6K434ZbhiO4/ymFoq12kNN13RLt1+ujXDKXyO3EawaLP/QhybwyoqWRoU
+just+1P3+zPlli+3Dg19Vmyp67XjRFnNPqW8QV9atYUE9NYv6Qhh3rfQlBx1VR6D
+BpPrJ+xaxbrdzLoT2KtKy4UGBurZHVIB8Pz+0JX+HNnYb18b5TZdaeyOVvycJMFX
+Vtou2Xkrpjxrm8nu/Cuxl31+4hJq7HHwpxTvpxmpfwsAAP//+wQE8g==
+-----END LOTW-TRUST SIG-----

version.txt

@@ -1 +1 @@
-0.0.8
+0.0.9