Cloud Tools for Eclipse Technical Design

Cloud Tools for Eclipse (C4E) supports development of Java servlet applications on the Google Cloud Platform (GCP) inside the Eclipse Integrated Development Environment (IDE) version 4.5 and later. It enables you to build web applications and sites that run on top of:

• App Engine Standard
• App Engine Flex
• Google Compute Engine (GCE)
• Google Container Environment (GKE)

Each of these is a separate Eclipse WTP facet:

• com.google.cloud.tools.eclipse.facets.appengine_standard
• com.google.cloud.tools.eclipse.facets.appengine_flex
• com.google.cloud.tools.eclipse.facets.gce
• com.google.cloud.tools.eclipse.facets.gke

FYI, code is not up to date with this and only defines a single appengine facet. We need to modify it along these lines.

Build environment

All code will be hosted in the open on Github. Design docs, issue trackers, and the like will also be hosted on Github.

Build will be based on Maven and tycho.

Eclipse settings will be checked into the repo for easy development and import.

Travis will be used for continuous integration and testing.

Java 7 or later will be assumed.

File encoding is UTF-8.

The Google Java Style Guide is followed.

Official plugin releases will be signed by Google and can be installed from Google's Eclipse download site.

Eclipse integration

The plugin will be based on the Eclipse Web Tools Platform (WTP).

There will be a single feature and and multiple plugins. Note that here "features" and "plugins" are technical terms that have almost the reverse of their normal meanings. A "feature" is a group of bundled functionality installed as a unit. Each feature contains multiple "plugins" (OSGI bundles) that provide specific functionality such as authorization, deployment, problem reports, etc.

GCP integration

For most interactions with the Google Cloud Platform, C4E will rely on the app-tools-lib-for-java library rather than invoking network services directly. This library shells out to the gcloud tool. The details are hidden from the user.

The user needs to manually install gcloud on their local system, and possibly tell Eclipse where to find it (if it's not in one of several common locations). Eventually, we may bundle the latest version of gcloud into C4E; but this will likely not happen for 1.0.

New application wizards

Each wizard will provide a fully configured, ultra-simple Eclipse project.

App Engine Standard

The wizard generates a new project containing these files and directories:

• src directory
  • HelloWorldServlet.java that responds only to GET requests with a constant message
• test directory
  • HelloWorldServletTest.java
• war directory
  • WEB-INF directory
    • appengine-web.xml AppEngine configuration file
    • web.xml servlet configuration file
    • lib directory containing any jars the servlet requires
  • index.html with a link that points to the servlet.
• README file
• Maven pom.xml file

The project will be associated with a WTP runtime that runs devappserver, and supplies general Java libraries, in particular servlet-api.jar and jsp-api.jar, to the classpath of the project.

The project will be tagged with an App Engine Standard facet. This facet supplies AppEngine specific libraries such as appengine-tools-api.jar to the classpath.

The project is set to build with Java 7 (the latest version supported by App Engine Standard).

App Engine Flexible

The sample for the Flexible environment is almost identical to the sample for the standard environment except:

• The project will be tagged with an App Engine Flexible nature.
• app.yaml replaces appengine-web.xml
• The javac source and target are set to Java 8

Load existing projects

If M2Eclipse is installed in Eclipse, then an existing Maven based project can be imported into Eclipse in the usual way ("File > Import > Existing Maven Projects"). The project will be tagged with the matching facet (standard, flex, GCE, GKE).

Run and debug your web application on your local workstation for manual testing

We will build a WTP runtime in the com.google.cloud.tools.eclipse.appengine.localserver package. This runtime relies on dev_appserver via app-tools-lib-for-java ands is launched in the usual way for a WTP server. (Select Project in Project Explorer and then choose Run As > Run On Server from the context menu).

WTP provides functionality such as changing the default port and specifying the install location of gcloud.

Switching from a run to debug environment is also provided by Eclipse, with no additional work on our part.

Code validations for problems specific to the App Engine Standard environment

C4E will reference a white list of Java classes available in the App Engine Standard environment. It checks that only Google App Engine supported JRE types are used by a CompilationUnit, and no classes are imported from com.google.appengine.repackaged.

It will be possible for the user to exclude classes and packages from these checks; e.g. testing packages that are not deployed to the server.

TBD: canonical whitelist location for programmatic use

Configuration files (e.g. appengine-web.xml, web.xml, etc.) will be scanned for assorted problems.

Authentication and Authorization

The ide-login library provides IDE agnostic functionality for authorizing code to deploy, monitor, and manage GCP resources.

C4E delegates this to gcloud auth login. gcloud launches a browser window that requests these credentials:

• Know who you are on Google
• View your email address
• View and manage the files in your Google Drive
• View and manage your applications deployed on Google App Engine

Assuming the user consents, the server sends an oAuth 2.0 access token back. gcloud stores the token on local disk and presents it with future deployments.

TBD: if we deploy before authing, does gcloud run this flow automatically?

TBD: UI. Where do we put the "login" button/menu item/ etc.?

Deploy web application

Prerequisites:

In order to deploy, several things must first be done in the DevConsole outside of Eclipse. In particular,

1. The environment you're deploying to must be enabled in the Google Developer Console.

2. Billing should be turned on. (Required for GCE, GKE, and App Engine Flex).

3. Create a project with name and ID in the Developer console.

Deployment Details

Deployment will be managed via WTP's org.eclipse.jst.server.generic.antpublisher. We supply an ant script named gcp.xml that contains targets to deploy and undeploy the application. WTP executes these targets at user request.

These targets invoke gcloud preview app deploy with the correct arguments to locate the user's authorization credentials and upload the artifacts to Google servers.

These targets will then invoke gcloud preview app deploy. Here we run into a problem. gcloud does not support deployment of either war directories or appengine-web.xml files. It needs an app.yaml file instead. Unless this is fixed, we "stage" the application to generate an app.yaml from our appengine-web.xml file. The maven gcloud plugin has code for doing this we can imitate.

Analytics

When the plugin is first installed, the user is asked to opt in to usage tracking.

If and only if they agree, then the plugin sends non-personally identifiable pings for usage of these features to the C4E Analytics account:

• installations
• uninstalls
• crashes + associated stack trace
• deploys
• local emulation runs
• a random UUID that is uniquely tied to the install

At no time do we include or send the project ID, the project name, the Gaia ID, the username, or any other such personally identifiable information. We merely track the total number of events. The only things we track that are not simple counters or arbitrary UUIds are exception stack traces from unanticipated exceptions.

The Analytics account ID is not stored in the source repository on Github. Instead it is supplied when the official artifact is built.

C4E communicates with Google Analytics via the Analytics Measurement Protocol. This is demonstrated in AnalyticsPingManager in the old plugin.