Skip to content
This repository was archived by the owner on May 12, 2021. It is now read-only.

Commit b9e668c

Browse files
Matt Andersonchtyim
Matt Anderson
authored andcommitted
(TWILL-100) Fix examples POMs and update documentations
- Updates to examples - Updates to BundleJarExample application and documentation. Signed-off-by: Terence Yim <[email protected]>
1 parent 73ecf8e commit b9e668c

File tree

5 files changed

+85
-6
lines changed

5 files changed

+85
-6
lines changed

.gitignore

+6
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,11 @@ out/
1616
lib/
1717
.idea
1818

19+
# Eclipse Files and Dir #
20+
.project
21+
.classpath
22+
.settings
23+
1924
# Gradle Files & Dir #
2025
build/
2126
.stickyStorage
@@ -26,3 +31,4 @@ target/
2631
logs/
2732
zookeeper.out
2833

34+
/bin

src/site/markdown/GettingStarted.md

+66-1
Original file line numberDiff line numberDiff line change
@@ -123,4 +123,69 @@ You can also stop the application through the `TwillController`:
123123
controller.stop().get();
124124
```
125125

126-
This will shut down the application master and all the containers.
126+
This will shut down the application master and all the containers.
127+
128+
### Hello World
129+
130+
To see Twill in action, you can run the "hello world" example applications located in the twill-examples module.
131+
132+
#### Prerequisites
133+
134+
* Single Node or Cluster installation of Hadoop with YARN (Hadoop >= 2.2.0) set-up and running.
135+
* Single Node or Cluster installation of ZooKeeper set-up and running.
136+
* Build of Twill Library Code (minimum, build of twill-examples module)
137+
138+
#### Running the Examples
139+
140+
There are two example applications you can run: HelloWorld and BundledJarExample.
141+
142+
##### HelloWorld Application
143+
144+
The HelloWorld application creates a simple YARN application that prints a line to the log.
145+
146+
You can run the HelloWorld application from any node of the Hadoop cluster using the below command
147+
(be sure to add your ZooKeeper Host and Port):
148+
149+
```sh
150+
$ CP=twill-examples-yarn-0.5.0-incubating-SNAPSHOT.jar:`hadoop classpath`; java -cp $CP org.apache.twill.example.yarn.HelloWorld {zookeeper_host:port}
151+
```
152+
153+
If successful, you should see logs output to the terminal with details of the running application. Once the application
154+
is finished running, check the YARN logs and you should see output like the following:
155+
156+
```
157+
14:49:45.944 [TwillContainerService] INFO o.a.twill.example.yarn.HelloWorld - Hello World. My first distributed application.
158+
```
159+
160+
##### BundledJarExample Application
161+
162+
The BundledJarExample application demonstrates the Twill functionality that allows you to run any Java application
163+
in Twill without worrying about library version conflicts between your application and Hadoop. The example
164+
calls the main class in a sample application `Echo`, which simply logs the command line argument(s) passed to it.
165+
The `Echo` application uses a different version of Guava from Twill and Hadoop distributions. BundledJarExample
166+
looks for the dependency in a `lib` folder packaged at the root of the `Echo` jar.
167+
168+
You can run the BundleJarExample application from any node of the Hadoop cluster using the below command
169+
(be sure to add your ZooKeeper Host and Port):
170+
171+
```sh
172+
$ CP=twill-examples-yarn-0.5.0-incubating-SNAPSHOT.jar:`hadoop classpath`;
173+
java -cp $CP org.apache.twill.example.yarn.BundledJarExample {zookeeper_host:port} twill-examples-echo-0.5.0-incubating-SNAPSHOT.jar echo.EchoMain arg1
174+
```
175+
176+
Like with the HelloWorld example, you should see logs output to the terminal. Once the application is complete, check
177+
the YARN logs as before and you should see output like the following:
178+
179+
```
180+
[TwillContainerService] INFO echo.EchoMain - Hello from EchoMain: 6
181+
err HELLO from scatch
182+
[TwillContainerService] INFO echo.EchoMain - Got args: [arg1]
183+
184+
...
185+
186+
out HELLO from scatch
187+
Got args: [arg1]
188+
189+
```
190+
191+
Congratulations! You have run your first Twill applications.

twill-examples/echo/pom.xml

+6
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,12 @@ limitations under the License.
4646
<dependency>
4747
<groupId>org.slf4j</groupId>
4848
<artifactId>slf4j-api</artifactId>
49+
<version>1.7.7</version>
50+
</dependency>
51+
<dependency>
52+
<groupId>org.slf4j</groupId>
53+
<artifactId>slf4j-simple</artifactId>
54+
<version>1.7.7</version>
4955
</dependency>
5056
</dependencies>
5157

twill-examples/yarn/pom.xml

+5-3
Original file line numberDiff line numberDiff line change
@@ -30,24 +30,26 @@ limitations under the License.
3030
<name>Apache Twill examples: YARN</name>
3131
<description>Examples demonstrating usage of twill-yarn</description>
3232
<artifactId>twill-examples-yarn</artifactId>
33+
34+
<properties>
35+
<guava.version>13.0.1</guava.version>
36+
</properties>
3337

3438
<dependencies>
3539
<dependency>
3640
<groupId>${project.groupId}</groupId>
3741
<artifactId>twill-yarn</artifactId>
3842
<version>${project.version}</version>
39-
<scope>provided</scope>
4043
</dependency>
4144
<dependency>
4245
<groupId>${project.groupId}</groupId>
4346
<artifactId>twill-ext</artifactId>
4447
<version>${project.version}</version>
45-
<scope>provided</scope>
4648
</dependency>
4749
<dependency>
4850
<groupId>com.google.guava</groupId>
4951
<artifactId>guava</artifactId>
50-
<scope>provided</scope>
52+
<version>${guava.version}</version>
5153
</dependency>
5254
</dependencies>
5355

twill-examples/yarn/src/main/java/org/apache/twill/example/yarn/BundledJarExample.java

+2-2
Original file line numberDiff line numberDiff line change
@@ -79,8 +79,8 @@ public static void main(String[] args) {
7979
}
8080

8181
String zkStr = args[0];
82-
BundledJarRunner.Arguments arguments = BundledJarRunner.Arguments.fromArray(
83-
Arrays.copyOfRange(args, 1, args.length));
82+
BundledJarRunner.Arguments arguments = new BundledJarRunner.Arguments(
83+
args[1], "/lib", args[2], Arrays.copyOfRange(args, 3, args.length));
8484

8585
File jarFile = new File(arguments.getJarFileName());
8686
Preconditions.checkState(jarFile.exists());

0 commit comments

Comments
 (0)