GEOS-11555: fix installation of extensions in various configurations

BUILD.md

@@ -65,27 +65,27 @@ Override these arguments to make use of build.geoserver.org nightly releases:
 * ``--build-arg STABLE_PLUGIN_URL=https://build.geoserver.org/geoserver/${GS_VERSION}/ext-latest/``
 * ``--build-arg COMMUNITY_PLUGIN_URL=https://build.geoserver.org/geoserver/${GS_VERSION}/community-latest/``
 
-Here is a working example for building 2.25.x nightly build::
+Here is a working example for building 2.27.x nightly build::
 ```
 docker build --no-cache-filter download,install \
-  --build-arg WAR_ZIP_URL=https://build.geoserver.org/geoserver/2.25.x/geoserver-2.25.x-latest-war.zip \
+  --build-arg WAR_ZIP_URL=https://build.geoserver.org/geoserver/2.27.x/geoserver-2.27.x-latest-war.zip \
-  --build-arg STABLE_PLUGIN_URL=https://build.geoserver.org/geoserver/2.25.x/ext-latest/ \
+  --build-arg STABLE_PLUGIN_URL=https://build.geoserver.org/geoserver/2.27.x/ext-latest/ \
-  --build-arg COMMUNITY_PLUGIN_URL=https://build.geoserver.org/geoserver/2.25.x/community-latest/ \
+  --build-arg COMMUNITY_PLUGIN_URL=https://build.geoserver.org/geoserver/2.27.x/community-latest/ \
-  --build-arg GS_VERSION=2.24-SNAPSHOT \
+  --build-arg GS_VERSION=2.27-SNAPSHOT \
-  -t 2.25.x .
+  -t 2.27.x .
 ```
 
 When running both stable extensions and community modules can be included:
 
 ```
 docker run -it -p 80:8080 \
+  --env DOWNLOAD_EXTENSIONS=true \
   --env INSTALL_EXTENSIONS=true \
   --env STABLE_EXTENSIONS="ysld" \
   --env COMMUNITY_EXTENSIONS="ogcapi" \
-  -t 2.25.x
+  -t 2.27.x
 ```
 
-Community modules are only available for nightly builds as they have not yet met the requirements for production use. Developers have shared these to attract participation, feedback and funding.
 
 ## How to build from main snapshot releases?
 
@@ -94,26 +94,29 @@ The build.geoserver.org output for the ``main`` branch requires the following:
 * ``--build-arg WAR_ZIP_URL=https://build.geoserver.org/geoserver/main/geoserver-main-latest-war.zip``
 * ``--build-arg STABLE_PLUGIN_URL=https://build.geoserver.org/geoserver/main/ext-latest/``
 * ``--build-arg COMMUNITY_PLUGIN_URL=https://build.geoserver.org/geoserver/main/community-latest/``
+* ``--build-arg GS_VERSION=2.27-SNAPSHOT``
 
+Since the artifacts include the version number, the ``GS_VERSION`` argument is needed to be set to the SNAPSHOT.
 
-Here is a working example for building main branch as 2.25.x build:
+Here is a working example for building main branch as 2.27.x build:
 
 ```
 docker build --no-cache-filter download,install \
   --build-arg WAR_ZIP_URL=https://build.geoserver.org/geoserver/main/geoserver-main-latest-war.zip \
   --build-arg STABLE_PLUGIN_URL=https://build.geoserver.org/geoserver/main/ext-latest/ \
   --build-arg COMMUNITY_PLUGIN_URL=https://build.geoserver.org/geoserver/main/community-latest/ \
-  --build-arg GS_VERSION=2.24-SNAPSHOT \
+  --build-arg GS_VERSION=2.27-SNAPSHOT \
-  -t 2.25.x .
+  -t 2.27.x .
 ```
 
 When running both [stable extensions](https://build.geoserver.org/geoserver/main/ext-latest/) and [community modules](https://build.geoserver.org/geoserver/main/community-latest/) can be included:
 
 ```
 docker run -it -p 80:8080 \
+  --env DOWNLOAD_EXTENSIONS=true \
   --env INSTALL_EXTENSIONS=true \
   --env STABLE_EXTENSIONS="wps,css" \
   --env COMMUNITY_EXTENSIONS="ogcapi-coverages,ogcapi-dggs,ogcapi-features,ogcapi-images,ogcapi-maps,ogcapi-styles,ogcapi-tiled-features,ogcapi-tiles" \
-  -t 2.25.x
+  -t 2.27.x
 ```
 

Dockerfile

@@ -36,7 +36,8 @@ ENV GEOSERVER_LIB_DIR=$CATALINA_HOME/webapps/geoserver/WEB-INF/lib/
 ENV GEOSERVER_REQUIRE_FILE=$GEOSERVER_DATA_DIR/global.xml
 ENV GEOSERVER_VERSION=$GS_VERSION
 ENV HEALTHCHECK_URL=''
-ENV INSTALL_EXTENSIONS=false
+ENV DOWNLOAD_EXTENSIONS=false
+ENV INSTALL_EXTENSIONS=true
 ENV POSTGRES_JNDI_ENABLED=false
 ENV ROOT_WEBAPP_REDIRECT=false
 ENV SKIP_DEMO_DATA=false

README.md

@@ -98,7 +98,8 @@ The environment variable `ROOT_WEBAPP_REDIRECT` can be set to `true` to issue a
 
 The ``startup.sh`` script allows some customization on startup:
 
-* ``INSTALL_EXTENSIONS`` to ``true`` to download and install extensions
+* ``DOWNLOAD_EXTENSIONS`` to ``true`` to download extensions from build server
+* ``INSTALL_EXTENSIONS`` to ``true`` to install extensions from the ``/opt/additional_libs`` directory
 * ``STABLE_EXTENSIONS`` list of extensions to download and install
 * ``CORS_ENABLED`` to ``true`` to enable CORS support. The following environment variables can be used to customize the CORS configuration.
   * ``CORS_ALLOWED_ORIGINS`` (default ``*``)
@@ -115,7 +116,7 @@ Example installing wps and ysld extensions:
 
 ```shell
 docker run -it -p 80:8080 \
-  --env INSTALL_EXTENSIONS=true --env STABLE_EXTENSIONS="wps,ysld" \
+  --env DOWNLOAD_EXTENSIONS=true --env INSTALL_EXTENSIONS=true --env STABLE_EXTENSIONS="wps,ysld" \
   docker.osgeo.org/geoserver:2.25.3
 ```
 
@@ -199,6 +200,19 @@ variables:
 * ``HTTPS_KEYSTORE_PASSWORD`` (defaults to `changeit`)
 * ``HTTPS_KEY_ALIAS`` (defaults to `server`)
 
+## How to add community extensions?
+
+To add community extensions, set the `COMMUNITY_EXTENSIONS` environment variable to a comma-separated list of extensions to install.
+
+If you add community extensions on an official release you will see the following warning:
+
+`WARNING: Installing community extensions on an official release version. Be sure to check compatibility.`
+
+Community modules have not yet met the requirements for production use. Developers have shared these to attract participation, feedback and funding.
+You an add them on your own risk. Please make sure to check the compatibility of the community modules with the GeoServer version you are using.
+In case of an issue make sure to report the usage of community modules in the issue descriptions.
+
+
 ## How to use the docker-compose demo?
 
 The ``docker-compose-demo.yml`` to build with your own data directory and extensions.
@@ -226,12 +240,14 @@ Following is the list of the all the environment variables that can be passed do
 | CATALINA_OPTS | Catalina options. Check [ref](https://www.baeldung.com/tomcat-catalina_opts-vs-java_opts) | `-Djava.awt.headless=true` |
 | GEOSERVER_DATA_DIR | Geoserver data directory location | `/opt/geoserver_data/` |
 | GEOSERVER_REQUIRE_FILE | Geoserver configuration used interally | `/opt/geoserver_data/global.xml` |
-| INSTALL_EXTENSIONS | Indicates whether additional GeoServer extensions should be installed | `false` |
+| DOWNLOAD_EXTENSIONS | Indicates whether additional GeoServer extensions should be installed | `false` |
+| INSTALL_EXTENSIONS | Indicates whether additional GeoServer extensions should be installed | `true` |
 | WAR_ZIP_URL | Specifies the URL for a GeoServer Web Archive (WAR) file | |
 | STABLE_EXTENSIONS | Specifies stable GeoServer extensions | |
 | STABLE_PLUGIN_URL | Specifies the URL for downloading the latest stable GeoServer plugins | `https://build.geoserver.org/geoserver/2.25.x/ext-latest` |
 | COMMUNITY_EXTENSIONS | Specifies community-contributed GeoServer extensions | |
-| COMMUNITY_PLUGIN_URL | Specifies the URL for downloading the latest community-contributed GeoServer plugins | `https://build.geoserver.org/geoserver/2.25.x/community-latest` |
+| COMMUNITY_PLUGIN_BASE_URL | Specifies the **base** URL for downloading the latest community-contributed GeoServer plugins. | `https://build.geoserver.org/geoserver/` | |
+| COMMUNITY_PLUGIN_URL | Specifies the URL for downloading the latest community-contributed GeoServer plugins | `${COMMUNITY_PLUGIN_BASE_URL}2.25.x/community-latest` |
 | ADDITIONAL_LIBS_DIR | Sets the directory for additional libraries used by GeoServer | `/opt/additional_libs/` |
 | ADDITIONAL_FONTS_DIR | Sets the directory for additional fonts used by GeoServer | `/opt/additional_fonts/` |
 | SKIP_DEMO_DATA | Indicates whether to skip the installation of demo data provided by GeoServer | `false` |
@@ -239,12 +255,9 @@ Following is the list of the all the environment variables that can be passed do
 | HEALTHCHECK_URL | URL to the resource / endpoint used for `docker` health checks | `http://localhost:8080/geoserver/web/wicket/resource/org.geoserver.web.GeoServerBasePage/img/logo.png` |
 | GEOSERVER_ADMIN_USER | Admin username |   |
 | GEOSERVER_ADMIN_PASSWORD | Admin password |  |
-
+| WGET_OPTS | Options for the `wget` command | `--no-check-certificate` |
-The following values cannot really be safely changed (as they are used to download extensions and community modules as the docker image first starts up).
-| VAR NAME | DESCRIPTION | SAMPLE VALUE |
-|--------------|-----------|------------|
 | GEOSERVER_VERSION | Geoserver version (used internally) | `2.24-SNAPSHOT`|
-| GEOSERVER_BUILD | Geoserver build (used internally) | `1628` |
+| GEOSERVER_BUILD | Geoserver build (used internally) must not be changed| `1628` |
 
 ## Troubleshooting
 

docker-compose-demo.yml

@@ -10,6 +10,7 @@ services:
     ports:
       - "80:8080"
     environment:
+      - DOWNLOAD_EXTENSIONS=true
       - INSTALL_EXTENSIONS=true
       - STABLE_EXTENSIONS=wps,csw
       - EXTRA_JAVA_OPTS=-Xms1G -Xmx2G

install-extensions.sh

@@ -4,51 +4,98 @@
 function download_extension() {
   URL=$1
   EXTENSION=$2
-  DOWNLOAD_FILE="${ADDITIONAL_LIBS_DIR}geoserver-${GEOSERVER_VERSION}-${EXTENSION}-plugin.zip"
+  VERSION=$3
+  DOWNLOAD_FILE="${ADDITIONAL_LIBS_DIR}geoserver-${VERSION}-${EXTENSION}-plugin.zip"
 
   if [ -e "$DOWNLOAD_FILE" ]; then
       echo "$DOWNLOAD_FILE already exists. Skipping download."
   else
-    if curl --output /dev/null --silent --head --fail "${URL}"; then
+    echo -e "\nDownloading ${EXTENSION} extension from ${URL} to ${DOWNLOAD_FILE}"
-        echo -e "\nDownloading ${EXTENSION} extension from ${URL} to ${DOWNLOAD_FILE}"
+    wget --progress=bar:force:noscroll -c $WGET_OPTS "${URL}" -O ${DOWNLOAD_FILE}
-        wget --progress=bar:force:noscroll -c --no-check-certificate "${URL}" -O ${DOWNLOAD_FILE}
+    if [ "$?" != 0 ]; then
-      else
+        echo "ERROR downloading: ${URL}"
-        echo "URL does not exist: ${URL}"
     fi
   fi
 }
 
-# Download stable plugins only if INSTALL_EXTENSIONS is true
+function install_lib() {
-if [ "$INSTALL_EXTENSIONS" = "true" ]; then
+    ADDITIONAL_LIB=$1
+    if [ ! -e "$ADDITIONAL_LIB" ]; then
+      echo "Skipping ${ADDITIONAL_LIB}: file not found."
+      return
+    fi
+
+    if [[ $ADDITIONAL_LIB == *.zip ]]; then
+      unzip -q -o -d ${GEOSERVER_LIB_DIR} ${ADDITIONAL_LIB} "*.jar"
+      echo "Installed all jar files from ${ADDITIONAL_LIB}"
+    elif [[ $ADDITIONAL_LIB == *.jar ]]; then
+      cp ${ADDITIONAL_LIB} ${GEOSERVER_LIB_DIR}
+      echo "Installed ${ADDITIONAL_LIB}"
+    else
+      echo "Skipping ${ADDITIONAL_LIB}: unknown file extension."
+    fi
+}
+
+# Download stable plugins only if DOWNLOAD_EXTENSIONS is true
+if [ "$DOWNLOAD_EXTENSIONS" = "true" ]; then
   echo "Starting download of extensions"
   if [ ! -d "$ADDITIONAL_LIBS_DIR" ]; then
     mkdir -p $ADDITIONAL_LIBS_DIR
   fi
   for EXTENSION in $(echo "${STABLE_EXTENSIONS}" | tr ',' ' '); do
     URL="${STABLE_PLUGIN_URL}/geoserver-${GEOSERVER_VERSION}-${EXTENSION}-plugin.zip"
-    download_extension ${URL} ${EXTENSION}
+    download_extension ${URL} ${EXTENSION} ${GEOSERVER_VERSION}
-  done
-  for EXTENSION in $(echo "${COMMUNITY_EXTENSIONS}" | tr ',' ' '); do
-    URL="${COMMUNITY_PLUGIN_URL}/geoserver-${GEOSERVER_VERSION}-${EXTENSION}-plugin.zip"
-    download_extension ${URL} ${EXTENSION}
   done
+  if [ ${#COMMUNITY_EXTENSIONS} -gt 0 ]; then
+    # build community version string from GEOSERVER_VERSION by removing the last part and adding SNAPSHOT
+    if [ -z "$COMMUNITY_EXTENSIONS_VERSION" ]; then
+      COMMUNITY_EXTENSIONS_VERSION="${GEOSERVER_VERSION/-SNAPSHOT/.x}"
+      COMMUNITY_EXTENSIONS_VERSION="${COMMUNITY_EXTENSIONS_VERSION%.*}-SNAPSHOT"
+    fi
+    COMMUNITY_PLUGIN_BASE_URL=${COMMUNITY_PLUGIN_BASE_URL:-"https://build.geoserver.org/geoserver/"}
+    if [ -z "$GEOSERVER_RELEASE_BRANCH" ]; then
+      GEOSERVER_RELEASE_BRANCH="${GEOSERVER_VERSION/-SNAPSHOT/.x}"
+      GEOSERVER_RELEASE_BRANCH="${GEOSERVER_RELEASE_BRANCH%.*}.x"
+
+    fi
+    COMMUNITY_PLUGIN_URL=${COMMUNITY_PLUGIN_URL:-"${COMMUNITY_PLUGIN_BASE_URL}${GEOSERVER_RELEASE_BRANCH}/community-latest/"}
+
+    echo "installing community modules from COMMUNITY_PLUGIN_URL=${COMMUNITY_PLUGIN_URL} with GEOSERVER_RELEASE_BRANCH=${GEOSERVER_RELEASE_BRANCH} and COMMUNITY_EXTENSIONS_VERSION=${COMMUNITY_EXTENSIONS_VERSION} from GEOSERVER_VERSION=${GEOSERVER_VERSION}"
+
+
+
+    for EXTENSION in $(echo "${COMMUNITY_EXTENSIONS}" | tr ',' ' '); do
+      URL="${COMMUNITY_PLUGIN_URL}/geoserver-${COMMUNITY_EXTENSIONS_VERSION}-${EXTENSION}-plugin.zip"
+      download_extension ${URL} ${EXTENSION} ${COMMUNITY_EXTENSIONS_VERSION}
+    done
+  fi
   echo "Finished download of extensions"
+else
+  echo "Skipping download of extensions as DOWNLOAD_EXTENSIONS is false"
 fi
 
-# Install the extensions
+# Install the extensions only if INSTALL_EXTENSIONS is true
-echo "Starting installation of extensions"
+if [ "$INSTALL_EXTENSIONS" = "true" ]; then
-for EXTENSION in $(echo "${STABLE_EXTENSIONS},${COMMUNITY_EXTENSIONS}" | tr ',' ' '); do
+  echo "Starting installation of extensions"
-  ADDITIONAL_LIB=${ADDITIONAL_LIBS_DIR}geoserver-${GEOSERVER_VERSION}-${EXTENSION}-plugin.zip
+  for EXTENSION in $(echo "${STABLE_EXTENSIONS}" | tr ',' ' '); do
-  [ -e "$ADDITIONAL_LIB" ] || continue
+    ADDITIONAL_LIB=${ADDITIONAL_LIBS_DIR}geoserver-${GEOSERVER_VERSION}-${EXTENSION}-plugin.zip
-
+    install_lib $ADDITIONAL_LIB
-  if [[ $ADDITIONAL_LIB == *.zip ]]; then
+  done
-    unzip -q -o -d ${GEOSERVER_LIB_DIR} ${ADDITIONAL_LIB} "*.jar"
+  if [ ${#COMMUNITY_EXTENSIONS} -gt 0 ]; then
-    echo "Installed all jar files from ${ADDITIONAL_LIB}"
+    # print warning if COMMUNITY extensions are installed on an official release (where GEOSERVER_VERSION is not ending with -SNAPSHOT)
-  elif [[ $ADDITIONAL_LIB == *.jar ]]; then
+    if [[ ! $GEOSERVER_VERSION == *-SNAPSHOT ]]; then
-    cp ${ADDITIONAL_LIB} ${GEOSERVER_LIB_DIR}
+      echo "WARNING: Installing community extensions on an official release version. Be sure to check compatibility."
-    echo "Installed ${ADDITIONAL_LIB}"
+    fi
-  else
+    
-    echo "Skipping ${ADDITIONAL_LIB}: unknown file extension."
+    for EXTENSION in $(echo "${COMMUNITY_EXTENSIONS}" | tr ',' ' '); do
+      if [ -z "$COMMUNITY_EXTENSIONS_VERSION" ]; then
+        COMMUNITY_EXTENSIONS_VERSION="${GEOSERVER_VERSION%.*}-SNAPSHOT"
+      fi
+      ADDITIONAL_LIB=${ADDITIONAL_LIBS_DIR}geoserver-${COMMUNITY_EXTENSIONS_VERSION}-${EXTENSION}-plugin.zip
+      install_lib $ADDITIONAL_LIB
+    done
   fi
-done
+  echo "Finished installation of extensions"
-echo "Finished installation of extensions"
+else
+  echo "Skipping installation of extensions as INSTALL_EXTENSIONS is false"
+fi