Resource API Transition Plan

Resource API Transition Plan

The [ResourceStore Design] outlines an API for stream based access to files in the data directory. Transitioning the codebase to use this API will require a bit of planning.

{toc}

Planning

GeoServerResourceLoader

• (/) Update internals of GeoServerResourceLoader
• (/) Add Resource access methods to GeoServerResourceLoader
• (/) Remove use of GeoServerResourceLoader search locations ** WEB_INF/web.xml file access

GeoServerDataDirectory and GeoServerSecurityManager

• (/) Update internals of GeoServerDataDirectory ** Already uses GeoServerResourceLoader ** Update methods to use Resource internally
• (/) Add Resource access methods to GeoServerDataDirectory
• (/) Update internals of GeoServerSecurityManager to use Resource
• Add Resource access methods to GeoServerSecurityManager

File interaction:

• (/) Research ** Review use org.geoserver.platform.FileWatcher ** Review use org.geoserver.security.file.FileWatcher ** Review GWC LockProvider
• (/) Port LockProvider and integrate with ResourceStore ** Bridge from GeoServer LockProvider to GWC LockProvider
• (/) Port FileWatcher implementations to use events ** Update internals of platform FileWatcher so subclasses continue to function unchanged ** Update internals of security FileWatcher so subclasses continue to function unchanged
• (/) Update security to use Resource and Resource events directly rather than FileWatcher

Update GeoServer codebase module by module ** Direct use of ResourceStore/Resource and Resource.out () for “stream access” ** Use of Resource.file () for “file access” (i.e. to pass file or URL reference to external library) ** Replace GeoServerResourceLoader “directory access” with use of Resource

GeoServerResourceLoader

• (/) Update internals of GeoServerResourceLoader ** This “migrates” the codebase to use ResourceStore for lookup
• (/) Introduce methods for Resource access
• (/) Remove use of GeoServerResourceLoader search locations ** GeoServerExtensions.file ( path ) provided for access to servletContext * Fix GeoServerJ2eeRoleService access of WEBINF/web.xml
• Transition checklist ** org.geoserver ** org.geoserver.catalog ** org.geoserver.catalog.impl ** org.geoserver.catalog.rest ** org.geoserver.catalog.util ** org.geoserver.config ** org.geoserver.config.util ** org.geoserver.csw ** org.geoserver.csw.store.internal ** org.geoserver.data.test ** org.geoserver.ftp ** org.geoserver.gwc.config ** org.geoserver.gwc.layer ** org.geoserver.h2 ** org.geoserver.inspire ** org.geoserver.jdbcconfig ** org.geoserver.jdbcconfig.internal ** org.geoserver.logging ** org.geoserver.monitor ** org.geoserver.monitor.auditlog ** org.geoserver.ows ** org.geoserver.security ** org.geoserver.security.impl ** org.geoserver.security.ldap ** org.geoserver.spatialite ** org.geoserver.template ** org.geoserver.test ** org.geoserver.wcs ** org.geoserver.web ** org.geoserver.web.admin ** org.geoserver.web.util ** org.geoserver.web.wicket ** org.geoserver.wfs ** org.geoserver.wfs.response ** org.geoserver.wfs.web ** org.geoserver.wfs.xml * org.geoserver.wfs.xml.v11_0 ** org.geoserver.wms ** org.geoserver.wms.eo ** org.geoserver.wps ** org.vfny.geoserver.global ** org.vfny.geoserver.wms.responses.map.htmlimagemap

web.xml regression

GeoServerJ2eeRoleService makes use of the following:

File webXML = loader.find( "web.xml" );

This is the only example of using search locations to find content outside of the data directory.

Functionality moved to GeoServerExtensions in order to use servlet context:

File webXML = GeoServerExtensions.file( "WEB_INF/web.xml" );

GeoServerDataDirectory

• Update internals of GeoServerDataDirectory
• Add Resource access methods to GeoServerDataDirectory
• Transition to Resource use where possible
• Checklist (106 references) ** org.geoserver.catalog ** org.geoserver.community.css.web ** org.geoserver.config ** org.geoserver.csw.store.simple ** org.geoserver.data ** org.geoserver.ftp ** org.geoserver.gwc ** org.geoserver.kml.icons ** org.geoserver.monitor.hib ** org.geoserver.script ** org.geoserver.script ** org.geoserver.security ** org.geoserver.security.filter ** org.geoserver.security.impl ** org.geoserver.security.ldap ** org.geoserver.template ** org.geoserver.test ** org.geoserver.web.admin ** org.geoserver.wfs.xslt.config ** org.geoserver.wms ** org.geoserver.wps ** org.vfny.geoserver.global

findDataDir / findDataDir regression

The following regressions were found, often due to code taking advantageous of gaps in the implementation of GeoServerDataDirectory.

GWCIntegrationTest:

File h2DefaultStore = dd.findDataFile("gwc/diskquota_page_store_h2");

GetCoverageTest.onSetUp has the following:

File data = getDataDirectory().findDataDir("watertemp");

In these cases I am fixing the test case, but will be more concerned if production code is effected by the dataRoot (false) implementation issues as shown below.

{warning} The code is asking to “data/watertemp” but manages to find “watertemp” due to a strange turn of events:

• findDataDir (“watertemp”) is implemented as:

  return dataDir( false, "watertemp" );

• Where dataDir ends up with:

  return resourceLoader.find( dataRoot(false), "watertemp");

• Which is where things go wrong, dataRoot (false) fails to find the data directory, resulting in null

  return resourceLoader.find( null, "watertemp");

• Allowing resourceLoader.find to locate “watertemp” (rather than “data/watertemp”). {warning}

findStyleFile / findOrCreateStyleFile / findStyleSldFile / findStyleSldFile regression

These methods determine a file location based on a StyleInfo and a filename. A few test cases such as DynamicDimensionsTest made use of a relative path when unpacking class path resources:

testData.addStyle(styleName, "../temperature.sld", getClass(), getCatalog());

This resulted in temperature.sld being unpacked outside the styles directory:

styles/
temperature.sld

And a relative path coming out of StyleInfo:

String path = Paths.path("styles",style.getFilename());

1. (/) Changed testData add style to ignore relative paths when determining the target filename.
2. (/) Added Paths.convert ( path, filename ) to allow for relative filenames (as long as they stay inside the data directory)

org.vfny.geoserver.global.GeoserverDataDirectory

Action: refactor this one out of existence!

It is mostly used to access the data directory:

File dataDir = GeoserverDataDirectory.getGeoserverDataDirectory();
File storeDir = new File(dataDir, "data/wcs/mosaicfordelete");        

It is also used to resolve file references:

if (path.startsWith("file:")) {
   File fixedPath = GeoserverDataDirectory.findDataFile(path);
   entry.setValue(DataUtilities.fileToURL(fixedPath).toExternalForm());
}

• (/) Replace each reference to GeoserverDataDirectory with GeoServerResoruceLoader
• (/) Replace GeoserverDataDirectory.getResourceLoader () with GeoServerExtensions.bean (GeoServerResource)
• (/) Feedback: Make GeoServerExtensions easier to use ** Introduced GeoServerExtensionsHelper

GeoServerSecurityManager

• (/) Update internals of GeoServerSecurityManager to use Resource
• (/) Add Resource access methods to GeoServerSecurityManager
• Transition GeoServerSecurityManager direct file access with Resource use where possible
• Checklist (106 references) ** SecurityManager.getSecurityRoot () - 18 items ** getRoleRoot () - 14 items ** getPasswordPolicyRoot - 3 items ** getUserGroupRoot - 13 items ** getAuthRoot - 3 items ** getFilterRoot - 5 items ** getRoot - 3 items ** getMasterPasswordProviderRoot - 5 items

File Events

1. (/) Platform FileWatcher
2. (/) Security FileWatcher
3. (!) GeoServerTeamplateLoader - looks to be complicated