Using Private Dependencies in Cloud Manager Builds

With the Cloud Manager, Adobe provides a viable tool to self-manage AEM environments in the cloud. In general, the approach is that all AEM customisation code is built from source and then deployed to the Cloud Manager managed environments. While this works well for small to medium-sized projects, larger customers often run into one of the following scenarios:

Previously, allowing a private repository in Cloud Manager was possible but it was a manual task that had to be taken care of on the Adobe side, by your project's CSE. Recently, however, Adobe added a feature to Adobe IO's cloud manager integration to set environment variables to the build in order to make this aspect self-service. With this alone, it’s possible to enforce repository-provided settings.xml by using the file ".mvn/maven.config".

However, duplicating Adobe's settings.xml in the repo is not ideal. For example, if a development team has an optimised settings.xml that they generally use, for the Cloud Manager repo, this would be overwritten with the Adobe defaults. Also, Adobe may change their master version of the settings.xml at any time which would break the Cloud Manager build (e.g. if their repository server is migrated to another domain). To avoid these issues a custom Maven extension comes into play.

Setup a private Maven repository using pipeline variables and the custom extension maven-ext-repos-from-env

Instead of duplicating the settings.xml and adding the credentials for the repository there, simply add the file ".mvn/extensions.xml" with the following content to your repository:

<extensions xmlns="http://maven.apache.org/EXTENSIONS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/EXTENSIONS/1.0.0 http://maven.apache.org/xsd/core-extensions-1.0.0.xsd">
    <extension>
        <groupId>biz.netcentric.maven.extension</groupId>
        <artifactId>maven-ext-repos-from-env</artifactId>
        <version>1.0.1</version>
    </extension>
</extensions>

This Maven extension is open source (https://github.com/Netcentric/maven-ext-repos-from-env) and available via Maven Central. In the case that no environment variables with prefix "MVN_SETTINGS_REPO" are set, this extension will simply do nothing. However, for the Cloud Manager case, the environment variables for the private repo may be set as follows:

aio cloudmanager:set-pipeline-variables \
   <PIPELINE_ID> \
   --programId=<PROGRAM_ID> \
   --variable \
     MVN_SETTINGS_REPO_URL https://artifacts.corp.com/cloud-manager/ \
     MVN_SETTINGS_REPO_USERNAME cloudmanager \
   --secret \
     MVN_SETTINGS_REPO_PASSWORD <REPO_PASSWORD>

Running the build in Cloud Manager with the extension and the environment variables will automatically amend the Maven execution with:

Your Cloud Manager build will show logs as follows:

...
[INFO] Repository added from system environment variables: https://artifacts.corp.com/cloud-manager/ (id: sysEnvRepo user: cloudmanager)
[INFO] Scanning for projects...
[INFO] Downloading from sysEnvRepo: https://artifacts.corp.com/cloud-manager/com/corp/aem/aem-parent/4.3.8/aem-parent-4.3.8.pom
[INFO] Downloaded from sysEnvRepo: https://artifacts.corp.com/cloud-manager/com/corp/aem/aem-parent/4.3.8/aem-parent-4.3.8.pom (51 kB at 39 kB/s)
...

See documentation at https://github.com/Netcentric/maven-ext-repos-from-env/blob/develop/README.md for more details (e.g. on the prerequisites for using aio cloudmanager:set-pipeline-variables).

What about private NPM dependencies?

For npm dependencies, the situation is slightly easier since Cloud Manager does not come with its own npm configuration and npm supports environment variables for both npmrc location and within npmrc itself. But first, in the same way as for Maven, let's set up a secret as pipeline variable to make it available for the build:

aio cloudmanager:set-pipeline-variables \
   <PIPELINE_ID> \
   --programId=<PROGRAM_ID> \
   --secret \
     NPM_PRIVATE_REPO_AUTH <npm-auth>

This authentication information can then be referenced from the .npmrc file in the source repository. In the following example, the scope "corp" is downloaded from a private repository:

​​@corp:registry=https://artifacts.corp.com/cloud-manager/
# https://support.sonatype.com/hc/en-us/articles/115015110067-Using-User-Token-s-with-NPM
//artifacts.corp.com/cloud-manager/:_auth=${NPM_PRIVATE_REPO_AUTH}
//artifacts.corp.com/cloud-manager/:always-auth=true

If the .npmrc is set up in the frontend module's root folder of the source repository, it will become active for Cloud Manager as well as for all developers. If the developers (lie for the Maven use case) already have a slightly different ~/.npmrc file optimised for development, the following snippet can be used in the Maven pom to solely enable this for Cloud Manager.

<profile>
    <id>cloud-manager</id>
    <activation>
        <property>
            <!-- The env variable CM_BUILD is set on any cloud manager build -->
            <name>env.CM_BUILD</name>
        </property>
    </activation>
    <build>
        <plugins>
            <plugin>
                <groupId>com.github.eirslett</groupId>
                <artifactId>frontend-maven-plugin</artifactId>
                <configuration>
                    <environmentVariables>
                        <!-- using name .npmrc-cloudmanager to make it self-explanatory -->
                        <NPM_CONFIG_USERCONFIG>${project.basedir}/.npmrc-cloudmanager</NPM_CONFIG_USERCONFIG>
                    </environmentVariables>
                </configuration>
            </plugin>
        </plugins>
    </build>
</profile>

A note about security

Cloud Manager runs on Azure and IP protection is currently not an option for private repositories. This means you have to make your private repository available on the internet. The following security measures are recommended to lock down the repository:

References

tag
Developer Circle
style
dark-plum