Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Misc fix test #248

Merged
merged 10 commits into from
Apr 4, 2021
4 changes: 2 additions & 2 deletions .jazzy.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module: KituraNet
author: IBM
github_url: https://github.com/IBM-Swift/Kitura-NIO/
author: IBM & Kitura Project Authors
github_url: https://github.com/Kitura/Kitura-NIO/

theme: fullwidth
clean: true
Expand Down
9 changes: 8 additions & 1 deletion .kitura-test.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#!/bin/sh

# Run Kitura-NIO tests
travis_start "swift_test"
echo ">> Executing Kitura-NIO tests"
Expand All @@ -10,11 +12,16 @@ if [ $SWIFT_TEST_STATUS -ne 0 ]; then
return $SWIFT_TEST_STATUS
fi

# TODO: For now, short-circuit kitura tests until those are stabalized.
return 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've create issue #249 to remember that...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest to add a // TODO: statement as well. Most IDEs collect and aggregate them. I for example use CLion, and this way I won't lose track of unkept promises.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.




# Clone Kitura
set -e
echo ">> Building Kitura"
travis_start "swift_build_kitura"
cd .. && git clone https://github.com/IBM-Swift/Kitura && cd Kitura
cd .. && git clone https://github.com/Kitura/Kitura && cd Kitura

# Set KITURA_NIO
export KITURA_NIO=1
Expand Down
27 changes: 19 additions & 8 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,33 +15,44 @@ matrix:
dist: xenial
sudo: required
services: docker
env: DOCKER_IMAGE=swift:5.0.3-xenial SWIFT_SNAPSHOT=5.0.3 DOCKER_PRIVILEGED=true SWIFT_TEST_ARGS="--parallel"
env: DOCKER_IMAGE=docker.kitura.net/kitura/swift-ci:5.0.3 SWIFT_SNAPSHOT=5.0.3 DOCKER_PRIVILEGED=true SWIFT_TEST_ARGS="--parallel"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

- os: linux
dist: xenial
sudo: required
services: docker
env: DOCKER_IMAGE=swift:5.1 DOCKER_PRIVILEGED=true SWIFT_TEST_ARGS="--parallel --sanitize=thread" CUSTOM_TEST_SCRIPT=.kitura-test.sh
env: DOCKER_IMAGE=docker.kitura.net/kitura/swift-ci:5.1 DOCKER_PRIVILEGED=true SWIFT_TEST_ARGS="--parallel --sanitize=thread" CUSTOM_TEST_SCRIPT=.kitura-test.sh
- os: linux
dist: xenial
sudo: required
services: docker
env: DOCKER_IMAGE=docker.kitura.net/kitura/swift-ci:5.3.3 DOCKER_PRIVILEGED=true SWIFT_TEST_ARGS="--parallel --sanitize=thread" CUSTOM_TEST_SCRIPT=.kitura-test.sh
- os: linux
dist: xenial
sudo: required
services: docker
env: DOCKER_IMAGE=swift:5.1 SWIFT_SNAPSHOT=$SWIFT_DEVELOPMENT_SNAPSHOT DOCKER_PRIVILEGED=true SWIFT_TEST_ARGS="--parallel --sanitize=thread"

# Removing support due to issue compiling swift-nio-ssl
# - os: osx
# osx_image: xcode10.3
# sudo: required
# env: SWIFT_SNAPSHOT=5.0.1 JAZZY_ELIGIBLE=true SWIFT_TEST_ARGS="--parallel"
- os: osx
osx_image: xcode10.2
osx_image: xcode11.3
sudo: required
env: SWIFT_SNAPSHOT=5.0.1 JAZZY_ELIGIBLE=true SWIFT_TEST_ARGS="--parallel"
env: SWIFT_SNAPSHOT=5.1.3 JAZZY_ELIGIBLE=true SWIFT_TEST_ARGS="--parallel --sanitize=thread"
- os: osx
osx_image: xcode11
osx_image: xcode12.2
sudo: required
env: SWIFT_TEST_ARGS="--parallel --sanitize=thread"
env: SWIFT_SNAPSHOT=5.3.1 SWIFT_TEST_ARGS="--parallel --sanitize=thread"
- os: osx
osx_image: xcode11
osx_image: xcode11.6
sudo: required
env: SWIFT_SNAPSHOT=$SWIFT_DEVELOPMENT_SNAPSHOT SWIFT_TEST_ARGS="--parallel --sanitize=thread" CUSTOM_TEST_SCRIPT=.kitura-test.sh


before_install:
- git clone https://github.com/IBM-Swift/Package-Builder.git
- git clone https://github.com/Kitura/Package-Builder.git

script:
- ./Package-Builder/build-package.sh -projectDir $TRAVIS_BUILD_DIR
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<p align="center">
<a href="http://kitura.io/">
<img src="https://raw.githubusercontent.com/IBM-Swift/Kitura/master/Sources/Kitura/resources/kitura-bird.svg?sanitize=true" height="100" alt="Kitura">
<img src="https://raw.githubusercontent.com/Kitura/Kitura/master/Sources/Kitura/resources/kitura-bird.svg?sanitize=true" height="100" alt="Kitura">
</a>
</p>

Expand All @@ -10,8 +10,8 @@
<a href="https://ibm-swift.github.io/Kitura-NIO/index.html">
<img src="https://img.shields.io/badge/apidoc-KituraNIO-1FBCE4.svg?style=flat" alt="APIDoc">
</a>
<a href="https://travis-ci.org/IBM-Swift/Kitura-NIO">
<img src="https://travis-ci.org/IBM-Swift/Kitura-NIO.svg?branch=master" alt="Build Status - Master">
<a href="https://travis-ci.org/Kitura/Kitura-NIO">
<img src="https://travis-ci.org/Kitura/Kitura-NIO.svg?branch=master" alt="Build Status - Master">
</a>
<img src="https://img.shields.io/badge/os-macOS-green.svg?style=flat" alt="macOS">
<img src="https://img.shields.io/badge/os-linux-green.svg?style=flat" alt="Linux">
Expand All @@ -23,11 +23,11 @@

# Kitura-NIO

Kitura-NIO is a [SwiftNIO](https://github.com/apple/swift-nio) based networking library for Kitura. Kitura-NIO adopts the same API as [KituraNet](https://github.com/IBM-Swift/Kitura-net), making the transition from KituraNet to Kitura-NIO seamless. While Kitura-NIO shares some code with Kitura-Net, the core comprising of HTTPServer, ClientRequest/ClientResponse and TLS support have been implemented using SwiftNIO. Kitura-NIO uses [NIOSSL](https://github.com/apple/swift-nio-ssl) for TLS support.
Kitura-NIO is a [SwiftNIO](https://github.com/apple/swift-nio) based networking library for Kitura. Kitura-NIO adopts the same API as [KituraNet](https://github.com/Kitura/Kitura-net), making the transition from KituraNet to Kitura-NIO seamless. While Kitura-NIO shares some code with Kitura-Net, the core comprising of HTTPServer, ClientRequest/ClientResponse and TLS support have been implemented using SwiftNIO. Kitura-NIO uses [NIOSSL](https://github.com/apple/swift-nio-ssl) for TLS support.

We expect most of our users to require higher level concepts such as routing, templates and middleware. These are not provided in Kitura-NIO. If you want to use those facilities you should be coding at the Kitura level, for this please see the [Kitura](https://github.com/IBM-Swift/Kitura) project. Kitura-NIO, like [Kitura-net](https://github.com/IBM-Swift/Kitura-net), underpins Kitura which offers a higher abstraction level to users.
We expect most of our users to require higher level concepts such as routing, templates and middleware. These are not provided in Kitura-NIO. If you want to use those facilities you should be coding at the Kitura level, for this please see the [Kitura](https://github.com/Kitura/Kitura) project. Kitura-NIO, like [Kitura-net](https://github.com/Kitura/Kitura-net), underpins Kitura which offers a higher abstraction level to users.

Kitura-NIO 2 has been tested with Swift 5. If you are using Swift 4, please use Kitura-NIO 1. See the [release history](https://github.com/IBM-Swift/Kitura-NIO/releases) for details.
Kitura-NIO 2 has been tested with Swift 5. If you are using Swift 4, please use Kitura-NIO 1. See the [release history](https://github.com/Kitura/Kitura-NIO/releases) for details.

## Features

Expand Down Expand Up @@ -64,7 +64,7 @@ We'd be more than happy to receive bug reports, enhancement requests and pull re

1. Clone this repository.

`$ git clone https://github.com/IBM-Swift/Kitura-NIO && cd Kitura-NIO`
`$ git clone https://github.com/Kitura/Kitura-NIO && cd Kitura-NIO`

2. Build and run tests.

Expand All @@ -73,7 +73,7 @@ We'd be more than happy to receive bug reports, enhancement requests and pull re
You may also want to run the tests in parallel:
`$ swift test --parallel`

In some Linux environments, a low open file limit could cause test failures. See [this](https://github.com/IBM-Swift/Kitura-NIO/issues/1).
In some Linux environments, a low open file limit could cause test failures. See [this](https://github.com/Kitura/Kitura-NIO/issues/1).

## Community

Expand All @@ -83,4 +83,4 @@ Join the [Kitura on Swift Forums](https://forums.swift.org/c/related-projects/ki

## License

This library is licensed under Apache 2.0. The full license text is available in [LICENSE](https://github.com/IBM-Swift/Kitura-NIO/blob/master/LICENSE.txt).
This library is licensed under Apache 2.0. The full license text is available in [LICENSE](https://github.com/Kitura/Kitura-NIO/blob/master/LICENSE.txt).
23 changes: 18 additions & 5 deletions Sources/KituraNet/ClientRequest.swift
Original file line number Diff line number Diff line change
Expand Up @@ -535,11 +535,12 @@ public class ClientRequest {
*/
public func end(close: Bool = false) {
closeConnection = close

let percentEncodedURL = URL(string: self.percentEncodedURL)!
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure about the force unwrapping? It looks dangerously un-recoverable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also a bit leery about it, but I was opting for minimal changes. I think it will take quite a bit more restructuring to better handle these situations.


var isHTTPS = false

let group = MultiThreadedEventLoopGroup(numberOfThreads: 1)
if (URL(string: percentEncodedURL)?.scheme)! == "https" {
if (percentEncodedURL.scheme)! == "https" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, even if it was already forced before.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I don't know how you'd recover from a missing scheme? But I'll take another look... maybe there's a better way.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we'd have a huge concrete wall of tests, I'd be happy with it, but in its current state my lack of understanding and the code I see scares me too.

But I don't have any better approaches right now either.

Personally I think force unwrapping is fine if there are multiple good quality and self validating unit tests which reason about the implementation and is convincing of its safety.

Otherwise I'd take the route of being super-defensive; and log warnings & errors whenever the sad path kicks in.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My goal right now is just to get tests consistently passing. So if you can agree that I at least haven't introduced any new problems, can we save the larger restructuring until later?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curiously the existing code does not use URL() as it's base way of storing URLs. I think one reason for some of the complexity in the code is the way Kitura handles http/https vs Unix domain sockets. If we're keeping this functionality (which I think is nice to have), it might make some of the code easier to follow one of these (non-standard) conventions for naming Unix domain sockets: whatwg/url#577

isHTTPS = true
self.sslConfig = TLSConfiguration.forClient(certificateVerification: .none)
}
Expand All @@ -550,8 +551,20 @@ public class ClientRequest {
initializeClientBootstrap(eventLoopGroup: group)
}

let hostName = URL(string: percentEncodedURL)?.host ?? "" //TODO: what could be the failure path here
let portNumber = URL(string: percentEncodedURL)?.port ?? 8080
let hostName = percentEncodedURL.host! //TODO: what could be the failure path here
let portNumber: Int
if let port = percentEncodedURL.port {
portNumber = port
} else {
if percentEncodedURL.scheme == "https" {
portNumber = 443
} else if percentEncodedURL.scheme == "http" {
portNumber = 80
} else {
portNumber = 8080
Log.error("Unknown port for url: \(url) Assuming 8080. This may change in the future.")
}
}
if self.headers["Host"] == nil {
let isNotDefaultPort = (portNumber != 443 && portNumber != 80) //Check whether port is not 443/80
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This whole play of port literals gave me an idea:

We could introduce a small harmless enum in one of the Kitura base modules which would encapsulate the typical ports and give names to them?

The bigger benefit is that it could open the door for future additional features to port handling without change risks.

@dannys42 && @mbarnach what do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea: it makes it more obvious (like which port is the default for Kitura), but also keep tracks of standard and custom ports in a robust fashion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm.. you mean an enum equivalent to the /etc/services file? Might be interesting, but not sure it belongs in Kitura since Kitura really only deals with web traffic?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way I understand it is: instead of 80, you have something like .http and instead of 443, .https. For 8080, it would be .default (names are poorly chosen!). It could also be extended to use .localhost and .domain(let custom) for the hostnames. Feature wise, the same, but better to add semantic and group the value in a single place. As part of the port enum, the var isDefaultPort: Bool { self == .http || self == .https } could be introduced.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another idea, if we're using URL() as a base type, then adding an extension for "isStandardPort" where we can check that if a port was given that it matches the standard for the given scheme. It might also be interesting to do a run-time check of /etc/services rather than having a hard-coded database.

Anyway I think this is a future project. Okay to punt on this for now?

self.headers["Host"] = hostName + (isNotDefaultPort ? (":" + String(portNumber)) : "")
Expand Down Expand Up @@ -639,7 +652,7 @@ public class ClientRequest {
bootstrap = ClientBootstrap(group: eventLoopGroup)
.channelOption(ChannelOptions.socket(SocketOptionLevel(SOL_SOCKET), SO_REUSEADDR), value: 1)
.channelInitializer { channel in
channel.pipeline.addHandler(try! NIOSSLClientHandler(context: self.sslContext!, serverHostname: nil)).flatMap {
channel.pipeline.addHandler(try! NIOSSLClientHandler(context: self.sslContext!, serverHostname: self.hostName)).flatMap {
channel.pipeline.addHTTPClientHandlers().flatMap {
channel.pipeline.addHandler(HTTPClientHandler(request: self))
}
Expand Down
5 changes: 5 additions & 0 deletions Tests/KituraNetTests/KituraNIOTest.swift
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,11 @@ class KituraNetTest: XCTestCase {
guard self.socketType != .tcp else { return }
let fileURL = URL(fileURLWithPath: socketFilePath)
let fm = FileManager.default

// ignore if file does not exist -- which is normal if Kitura is shutdown properly.
guard fm.fileExists(atPath: fileURL.path) else {
return
}
do {
try fm.removeItem(at: fileURL)
} catch {
Expand Down