13th November 2018

I have recently had the privilege of joining Pillar, an exciting new project as part of their DevOps team. As a relatively junior developer, this was always going to be a journey of discovery and it was that in spades! ♠️

Our task was to take a React-native project and to create a Continuous Integration/Continuous Delivery (CI/CD) pipeline that could leverage an Automated Testing Framework. This blog deals with the CD part of the process. Another blog will detail the Test Automation part of the pipeline. So our task sounds easy huh. But why set up a CD/CI pipeline I hear you ask? What benefit will that add? Firstly if you’re unsure as to the meaning and difference of these terms, check out this very helpful article (https://www.atlassian.com/continuous-delivery/ci-vs-ci-vs-cd).

How is a CI/CD pipeline useful? In short, it makes developers’ lives easier. It saves developers time, gives them quick feedback and streamlines the whole deployment process including building the application, testing the application, archiving the application in a binary repository and deploying both to the necessary testing platform. All of this can be handled by an automated CI/CD pipeline thus saving developers hours of time. That makes developers happy and frees them up to do what they do best, developing and creating the magic. ✨

So an automated pipeline saves time. That’s nice! However, that is not where it shines. It shines in that it can set up a process that reliably and repeatedly releases code ☀️ As intelligent and as capable as developers are, they are humans. Which means they get tired and can make mistakes and an automated pipeline can help mitigate against that. Here is a great quote describing what CI/CD is. It’s a simple quote really but to the point.

Create a repeatable, reliable process for releasing software

So these were the big picture goals we needed to achieve with our React Native project:

  • Dependency management
  • Build orchestration and publishing
  • Combining the above and automating our pipeline with CircleCI

Dependency management

We started with a React Native project that had many dependencies. Here’s a brief list of the main issues specific to our project:

  • Our application required many unique dependencies such as Signal Server
  • We had our own repositories that needed to be included that we had archived in Artifactory
  • iOS specific dependencies: cocoapods

Dependency management issue #1: Managing our many npm dependencies

To resolve this issue, we used CircleCI and Artifactory. CircleCI was used to bookend the building of our application. We used it to to start off our application builds and as we’ll see later, we used it to close off our application builds as well. We used Artifactory in the following ways:

  • As a private npm registry: we use it to store npm packages that are imported into the application as dependencies, e.g. the various SDKs
  • As a private docker registry: we use this to store assets that are to be deployed into AWS
  • To store generic artefacts: at time of writing we use this to store Signal Server deployables.

This is a little snapshot of our config.ymlfile, the configuration file that CircleCI uses to orchestrate the building of the pipeline. It shows how we used CircleCI to interact with Artifactory to manage our dependencies.

CircleCI config.yml file

Let’s have a look at a few of the steps:

  • Authenticate with Artifactory: this curl command sends your username/password to artifactory, retrieves an authentication token, and stores it in the .npmrc file
  • set Registry to use Artifactory: this line adds the link to Artifactory so that all npm install calls default to our Artifactory registry.
  • The yarn commands were run to install the node dependencies and to validate that the versions in the yarn.lock and the package.json were synced up.

So by using CircleCI and Artifactory, we were able to reliably pull down the many dependencies saved in our Registry and install them on the CircleCI box.

Dependency management issue #2: Cocoapods

Within the world of iOS, dependencies are managed by Cocoapods. A CocoaPod, or “pod” for short, is a general term for either a library or framework that’s added to your project by using CocoaPods.

This is another snapshot of the config.ymlshowing how we installed the various iOS dependencies (pods) required in our application.

Let’s have a look at a few of these steps:

  • Fetch Cocoapods Specs: this is a curl command that downloads and installs the specs for the various libraries and frameworks held in the CocoaPods repository cache(which is updated roughly every 30 mins).
  • Install Cocoapods: on top of the cocoapods specs installed, this installs all the pods from our application on the CircleCI box.
  • restore_cache & save_cache: we used these steps as a time saver. We didn’t want to install the cocoapods each and every time we built the application. So the save_cache command saved our pods in the CircleCI cache and the restore_cache command restored these saved pods from the cache the very next time the application was built.

Build orchestration

OK great so we were able to resolve our dependencies issues. The next step was to orchestrate our build pipeline and publish the application to the testing platforms for iOS and Android. As with dependencies, we had some issues to overcome with our build orchestration as well, introducing the dark world of codesigning…

Build orchestration issue #1: CodeSigning in iOS

In the iOS mobile world, code signing is difficult. CodeSigning is a security measure that allows iOS/Google Play to identify who signed your app and to verify that your app hasn’t been modified since you signed it. It involves dealing with various certificates, provisioning profiles and traditionally each developer on the team would need to manually set these up. But we needed a way of automating the code signing so that we could use it in our CI tool of choice, CircleCI. To resolve this issue, we used Fastlane Match. I will discuss Fastlane Match in more detail in a future blog but for the time being it’s sufficient to say that Fastlane Match was a success and saved us stacks of time. Fastlane Match is an implementation of the Codesigning guide for teams.

This first thing we needed to do was create a private Github repository to hold the Certificates and Provisioning Profiles that would be shared amongst the team. With Fastlane Match, a developer would simply need to install Fastlane(see Getting Started section) , be granted access to the private GitHub repository, and then run the following commands: fastlane match development--readonly.This would download the existing certificates and provisioning profiles for the app to be signed during the development stage. In the same way, the developer would need to run the following command to download the existing certificates and provisioning profiles for the app to be signed during the appstore stage:fastlane match appstore--readonly. Essentially Fastlane Match generated the provisioning profiles and certificates to facilitate the signing of our app for both the development and production stages and saved these in our private github repository. These were used these to create a single signing identity across the team which could be downloaded fairly easily. Success!

Build orchestration issue #2: CodeSigning in Android

Sorting out the Codesigning in Android (known as App Signing in Android) was a whole lot easier than with iOS but it had its own fare share of challenges. Our first step was opting into the signing process that Google recommends for Android app development called App Signing. The next step was to set up a debug.keystore and a release.keystore. It involves using keytool (a JDK cli tool). See this Android article on Signing your app for more information. Here is an example of what that key would look like:

keytool -genkey -v -release.keystore my-release-key.jks
-keyalg RSA -keysize 2048 -validity 10000 -alias my-alias

This command will generate a single private key as a file called release.keystore, saving it in the current directory (which can be moved anywhere you like!) .The private key will be valid for 10,000 days.

Build orchestration issue #3: Automating the build of the app

It is great having the ability to do all of this codesigning with Fastlane Match and with Google’s app signing, now we need to actually build the application. We manage this by using two tools: Fastlane Gym for building the iOS application and Gradle cli tool for building the Android application.

  • Fastlane Gym: This tool is an alternative to the native iOS cli tool, xcodebuild, which can be long and difficult to understand. The gym tool is typically used within a special Fastlane file called Fastfile. The Fastfile contains blocks of code called lanes which represented separate jobs (which can be customised for various tasks such as codesigning, building or both). The following snippet is an example of gym being used (from the fastlane gym documentation page):
build_ios_app(   
  workspace: "MyApp.xcworkspace",
  configuration: "Debug",   
  scheme: "MyApp",
  silent: true,   
  clean: true,
  output_directory: "path/to/dir", # Destination directory  
  output_name: "my-app.ipa",       # specify the name of the .ipa
  sdk: "iOS 11.1"        # SDK to use when building the project
)
  • Gradle CLI tool: this tool allows you to build the android application from the command line using the Gradle wrapper command line tool and is accessible from the root of our Android project. The following snippet of code shows how we used this cli tool to build the staging application.
./gradlew clean assembleProdStaging — no-daemon — stacktrace — max-workers=2 -PBUILD_NUMBER=$CIRCLE_BUILD_NUM -x bundleProdStagingJsAndAssets

Build orchestration issue #4: Building multiple variants from the same source code

So why did we want to build multiple build variants? The goal was to create two separate apps from the same source code (so not having to duplicate code for a whole new app) both for Android and iOS. So four build variants in total. Each build variant pointing to its own separate environment. Success would look like this:

Stage and Prod build variants for Android and iOS

To do this we had to get our hands dirty with the native source code bases. For iOS, we needed to make configurations in the XCode IDE and for Android, we needed to make configurations in the app/build.gradle file:

i0S build variants configuration steps:

  • Created separate bundleID on AppStore Connect. This set a unique ID for a completely separate iOS app called pillarwallet.staging.
  • Added a staging build configuration in XCode called Staging.Release
  • Created a separate build scheme called Staging
  • Configured the Build Settings section in XCode with staging values, particularly the following sections: PackagingSearch Paths, User-defined.
  • Configured the Build phase section in XCode with staging values

Android build configuration steps:

  • We needed a unique ID for a completely separate Android app called pillarwallet.staging Created separate applicationId on Google Play Console.
  • The Android Staging application required a new Firebase project, mostly for the purposes of notifications. This produced an updated version of google-services.json (now including the particulars of both the production and staging apps). We downloaded this and added it to the root of the Android project.
  • To create build variants from Configured 3 main sections within the app/build.gradle file with staging values: signingConfigs, productFlavors, buildTypes.

Build orchestration issue #5: Uploading application toTestFlight & Google Play Store

Separating our source code out to produce independent apps was a milestone. The next step was to upload these app builds to the Beta Testing platforms for iOS and Android. To facilitate this, we used another Fastlane tool, Pilot. This tool allowed us to extend our Fastfile with an additional keyword called upload_to_testflight . It took the packaged app build for both iOS (ipa file) and for Android (apk file) and uploaded it to Testflight and Google Play Console respectively. Below are code snippets showing how this was used:

iOS

desc "Release to production TestFlight"
lane :deploy_prod do
  ...
  upload_to_testflight(changelog: commit[:message],
      skip_waiting_for_build_processing: true,
  )
end

Android

desc "Deploy a new version to the Google Play Staging Test Track"
lane :deploy_staging do
  ...
  upload_to_play_store(
    track: 'internal',
    apk: 'app/build/outputs/apk/app-prod-staging.apk',
    json_key_data: ENV['GOOGLE_JSON_DATA'],
    package_name: 'com.pillarproject.wallet.staging',
    skip_upload_aab: true,
    skip_upload_images: true,
    skip_upload_screenshots: true
  )
end

Wrapping it up in a bow using CircleCI

We used CircleCI to automate the build of our various build artifacts and apps. Below is a summarised snippet(many of the run steps have been ommitted for clarity) of ourstage_iosjob to show how we configured this . You will see the various run steps here performing various tasks already mentioned in this blog. For example the run step named Upload to TestFlight references the Fastlalne lane called deploy_staging . This particular lane was customised to build the staging app and upload it to TestFlight. You will also see that their are steps named prepare to archive ipa file and store_artifacts. These steps save the ipa file and store it in the Artifacts directory of CircleCI in a destination directory called app_build.

stage_ios:
    working_directory: ~/pillar
    macos:
      xcode: "9.4.0"
    environment:
      FL_OUTPUT_DIR: output
    shell: /bin/bash --login -o pipefail
steps:
      - checkout
      ...
      - run:
          name: Upload to TestFlight
          command: cd ios && bundle exec fastlane deploy_staging
      - run:
          name: Announce Deployment
          command: |
            chmod +x .circleci/announceDeployment.sh
            .circleci/announceDeployment.sh "{App Name}"
            "Staging TestFlight"
      - run:
          name: prepare to archive ipa file
          command: |
            mkdir -p ./toArchive
            cp ./ios/pillarwallet-staging.ipa ./toArchive
      - store_artifacts:
          path: ./toArchive
          destination: app_build

CircleCI automatically connects to Github so we used this to orchestrate the triggering of our various app builds. CircleCI has a concept of workflows, which allowed us to wrap up our pipeline so that various jobs were configured together. According to the CircleCI docs…

A workflow is a set of rules for defining a collection of jobs and their run order.

Below is a snippet from the CircleCI/config.yml file, showing how we made use of workflows and customised them to build the app depending on which branch the code was committed from. For example, the jobstage_ios was built provided the job build had successfully run and this would only be built if the code had been committed from master branch.

workflows:
  version: 2
  build_and_deploy:
    jobs:
      - build
      - stage_ios:
          requires:
            - build
          filters:
            branches:
              only:
                  - master
      - stage_android:
          requires:
            - build
          filters:
            branches:
              only:
                  - master
      - release_to_prod:
          type: approval
          requires:
            - stage_ios
            - stage_android
      - prod_ios:
          requires:
            - release_to_prod
      - prod_android:
          requires:
            - release_to_prod

An Expo side note…

When we started, the react-native application was using the Expo wrapper which surrounded our application. Expo describes itself as The Fastest Way to build an app. It was useful for a few reasons:

  • Bootstrapping: Allowed the team to quickly build the foundations of the application, getting them off the ground running.
  • Prototyping: Expo uses various libraries to increase the speed of live reload of the application. This improved feedback allowing the developers to build and make changes more efficiently.

However, we experienced some pain points dealing with Expo. Here is short mention of such issues:

  • Performance: the Expo wrapper contains many libraries and as such, it made the size of our app very large. In hindsight, it made our application approximately 40% larger than it needed to be. The speed of building and running our app was slowed down because of this.
  • Locked into Expo APIs: when using the Expo wrapper your application can be in one of 3 states. 1. Expo locked in state allows developers full access to all the Expo libraries and Expo APIs. However, this prevents the use of certain React Native libraries/Node Modules. 2. Ejected to ExpoKit: this allows the app to be built using the Expo APIs and the workflow is similar to that of a plain React Native app. This gave us greater control but we were still required to use certain Expo APIs and were still unable to access certain 3rd party libraries. 3. Ejected to plain React Native: here you lose access to the Expo APIs and leave the Expo environment but you now have no constraints in accessing 3rd party libraries.
  • Publishing: when building the app using Expo, we were required to use the exp publishcommand to bundle the source code, which made it available from a single URL on the Expo Sever. However, running this command committed all the code to a single Expo server channel. This meant that when we built staging it would commit the staging code to the server, and subsequently building production would override Staging and save Production. The Expo server was serving up the committed code on TestFlight and Google Play Console. As you can imagine, this caused havoc as were unable to control what was being served on the Beta Testing platforms. There was a potential solution for this called Expo Release Channels but for various reasons, we decided against that.

For the above-mentioned reasons, we decided to eject our application from Expo. We initially ejected to ExpoKit which solved certain issues, and then we ejected to plain React Native.

Appendix

To help us achieve the goals mentioned above, we used the following tools:

  • CircleCI: to do the work of building and deployment the application
  • Fastlane Match, Gym & Pilot: to automate the process of signing, building and deploying the application to TestFlight (iOS ) & Google Play Store (Android)
  • Artifactory: binary repository to save various compiled artefacts for re-use in multiple environments

Being involved in this project has been a journey of discoveries and one that has opened me to many new tools and technologies. It’s involved a lot of beard scratching and banging my head against a wall. But no pain no gain! And the gain from all that pain has been progress, learning and experience. To briefly name a few pain points…

  • coming to grips with the Xcode (a learning experience of epic proportions)
  • Understanding how mobile applications are built with React-Native and compiled into their respective code bases for iOS and Android
  • understanding how all mobile applications need to go through something called CodeSigning…a challenge all of its own
  • Understanding how to use a tool like CircleCI to pipeline an application
  • Automating building and deploying of applications using Fastlane

I hope you have found this blog useful and that it might come in handy if you find yourself in a similar situation. Following on from this article, I will be blogging part 2 A journey of the DevOps variety — Part 2. This will focus on the Test Automation part of the pipeline and will include some cool new learning with Cucumber and Appium.

About James Hughes


No Comments

Leave a comment