Skip to content

chenkennt/movie-db-java-on-azure

BranchesTags
 
 

Repository files navigation

Movie Database App using Java on Azure

The purpose of this sample application is to illustrate a modern Java app in the cloud; the result of this project will be to create a movie database simliar to IMDB.

Requirements

In order to create and deploy this sample application, you need to have the following:

An Azure subscription; if you don't already have an Azure subscription, you can activate your MSDN subscriber benefits or sign up for a free Azure account.

In addition, you will need all of the following components before you go through the steps in this README:

| Azure CLI | Java 8 | Maven 3 | Git | Docker |

NOTE: There are additional requirements in the ~/deployment/README.md file which are required in order to setup your development environment; other required components will be installed automatically by the provisioning scripts.

Overview

In the following sections, you will create a development sandbox environment on Azure which uses the following components:

The following diagram illustrates the full topology for this sample application enviroment:

In this basic layout, the following design decisions have been implemented:

  • Internet-facing web apps are running in Linux containers on AAS, which can run across multiple regions worldwide.

  • For better performace, this enviroment uses the following:

  • Container images for the web apps are built using Docker and pushed to a managed private Docker registry in ACR, and deployed to Linux containers on AAS.

  • The web apps communicate with the data apps running in Kubernetes clusters in ACS.

  • Data apps are REST API apps which store and read data from Azure Database for MySQL, which is a fully managed database as a service; data apps store images into and read images from Azure Storage.

  • Another traffic manager is deployed as a load balancer in the front of data apps for routing requests for better performance and availability.

Note: For now, Node.js is being used instead of Java in this sample application for the Azure functions; this will be updated in the near future.

Create and Deploy the Sample Application

Download and customize the sample for your development environment

  1. Follow the steps in the ~/deployment/README.md file of the sample project to clone the project repo and set up your development environment.

  2. Navigate to the configuration directory for your Maven installation; for example: /usr/local/maven/3.5.0/libexec/conf/ or %ProgramFiles%\apache-maven\3.5.0\conf:

    a. Open the settings.xml file with a text editor.

    b. Add parameterized settings for ACR access settings to the <servers> collection in the the settings.xml file, this will enable Maven to use a private registry; for example:

    <servers>
   <server>
      <id>${env.ACR_NAME}</id>
      <username>${env.ACR_USERNAME}</username>
      <password>${env.ACR_PASSWORD}</password>
      <configuration>
         <email>[email protected]</email>
      </configuration>
   </server>
</servers>

    c. Save and close your settings.xml file.

Create the initial build

  1. Open a command prompt and navigate to the ~/deployment/ folder of your local repo.

  2. Login to your Azure account and specify which subscription to use:

    az login
az account set --subscription "<your-azure-subscription>"

    NOTE: You can use either a subscription name or id when specifying which subscription to use; to obtain a list of your subscriptions, type az account list.

  3. Build an initial layout on Azure using an ARM template from using one of the following methods:

    source provision.sh

    NOTE: On Windows, run all shell scripts in Git Bash.

    The provisioning script will create the following resources in Azure, which may take a long time to complete:

    • A MySQL instance
    • A Redis Cache instance
    • A Function app for resizing images
    • Two traffic managers for load balancing
    • An Azure Container Registry for hosting a private Docker image
    • Two Linux-based web apps for front-end websites
    • Two Azure Container Services (Kubernetes Clusters) for REST API services
    • An Azure Container Service (Kubernetes Cluster) for a Jenkins server

Deploy the internal-facing data app into a Kubernetes cluster in ACS

  1. Open a command prompt and navigate to the folder which contains the data app, which is located in the "~/data-app/" folder of your repo; this is a Spring Boot app which:

    • Stores and reads data from Azure Database for MySQL using Spring JDBC

    • Stores images into and reads images from Azure Storage.

  2. Build and dockerize the data app, and push the container into ACR:

    mvn package docker:build -DpushImage

  3. Deploy the data app to a Kubernetes cluster in ACS using the Kubernetes CLI:

    kubectl create secret docker-registry ${ACR_LOGIN_SERVER} \
   --docker-server=${ACR_LOGIN_SERVER} \
   --docker-username=${ACR_USERNAME} \
   --docker-password=${ACR_PASSWORD} \
   [email protected] \
   --namespace=${TARGET_ENV} \
   --save-config
   
envsubst < ../deployment/data-app/deploy.yaml | kubectl apply --namespace=${TARGET_ENV} -f -

  4. Run below command to watch the creation process of your service object in Kubernetes. Wait until column EXTERNAL-IP has a valid IP address, which means your data app is accessible from internet now.

    kubectl get svc --namespace=${TARGET_ENV} --watch

  5. Navigate to the ~/deployment/ folder of your local repo and run the following script, which will configure various variables for your local enviroment:

    cd ../deployment
source dev_setup.sh

    NOTE: Microsoft is currently developing a Maven plugin to deploy to a Kubernetes cluster in Azure Container Service, so in the future you will be able to use mvn deploy.

Test your data app deployment

Run the following command to test whether your data app was successfully deployed:

curl http://${DATA_API_URL}/api/v1

Deploy the Internet-facing web app into Linux containers in AAS

  1. Open the Internet-facing web app, which is located in the "~/web-app/" folder of your repo.

    • This is also a Spring Boot app which talks to the data app that we just deployed.

    • The web app can also use Azure Redis Cache using Spring Data Redis.

  2. Build and dockerize the web app, and push the container into ACR:

    mvn package docker:build -DpushImage

  3. Use Azure CLI to deploy the web app to a Linux container in Azure App Service:

    az webapp config container set -g ${EAST_US_GROUP} \
   -n ${EAST_US_WEBAPP_NAME} \
   --docker-custom-image-name ${ACR_LOGIN_SERVER}/web-app \
   --docker-registry-server-url http://${ACR_LOGIN_SERVER} \
   --docker-registry-server-user ${ACR_USERNAME} \
   --docker-registry-server-password ${ACR_PASSWORD}

az webapp config set -g ${EAST_US_GROUP} \
   -n ${EAST_US_WEBAPP_NAME} \
   --linux-fx-version "DOCKER|${ACR_LOGIN_SERVER}/web-app"

az webapp config appsettings set -g ${EAST_US_GROUP} \
   -n ${EAST_US_WEBAPP_NAME} \
   --settings DATA_API_URL=${DATA_API_URL} \
   PORT=${WEB_APP_CONTAINER_PORT} \
   WEB_APP_CONTAINER_PORT=${WEB_APP_CONTAINER_PORT} \
   STORAGE_CONNECTION_STRING=${STORAGE_CONNECTION_STRING} \
   ORIGINAL_IMAGE_CONTAINER=${ORIGINAL_IMAGE_CONTAINER} \
   THUMBNAIL_IMAGE_CONTAINER=${THUMBNAIL_IMAGE_CONTAINER} \
   REDIS_HOST=${REDIS_HOST} REDIS_PASSWORD=${REDIS_PASSWORD}

az webapp restart -g ${EAST_US_GROUP} -n ${EAST_US_WEBAPP_NAME}

    NOTE: Microsoft is currently developing a Maven plugin to accomplish these steps, So in the future you will be able to use mvn deploy.

Test and diagnose your sample deployment

Test and diagnose using one of the following two methods:

  • Open website in a web browser

    open http://${EAST_US_WEBAPP_NAME}.azurewebsites.net/

  • Run the following command in a console window:

    curl http://${EAST_US_WEBAPP_NAME}.azurewebsites.net/index

OPTIONAL: Enable monitoring and diagnostics using third party services

You can optionally enable New Relic and OverOps monitoring and diagnostics in both the web app and data app by using the steps in the following sections.

Enable New Relic

To enable monitoring using New Relic, use the following steps.

  • Open a console and navigate to the ~/web-app folder in your local repo.

  • Configure the following environment variables:

    export NEW_RELIC_LICENSE_KEY=<your-new-relic-license-key>
export WEBAPP_NEW_RELIC_APP_NAME=<app-name-in-new-relic>

  • Run the following command to build an image with New Relic and push the image to ACR; for example, if your image name is web-app-w-new-relic:

    mvn package docker:build@with-new-relic -DpushImage

  • Run the following commands to deploy the web app to AAS:

    az webapp config container set -g ${EAST_US_GROUP} \
   -n ${EAST_US_WEBAPP_NAME} \
   --docker-custom-image-name ${ACR_LOGIN_SERVER}/web-app-w-new-relic \
   --docker-registry-server-url http://${ACR_LOGIN_SERVER} \
   --docker-registry-server-user ${ACR_USERNAME} \
   --docker-registry-server-password ${ACR_PASSWORD}

az webapp config set -g ${EAST_US_GROUP} \
   -n ${EAST_US_WEBAPP_NAME} \
   --linux-fx-version "DOCKER|${ACR_LOGIN_SERVER}/web-app-w-new-relic"

az webapp restart -g ${EAST_US_GROUP} -n ${EAST_US_WEBAPP_NAME}

  • Browse to your account portal in New Relic to see real-time monitoring data.

Enable OverOps

To enable diagnostics using OverOps, use the following steps.

  • Open a console and navigate to the ~/web-app folder in your local repo.

  • Configure the following environment variables:

    export OVEROPSSK=<your-overops-sk>

  • Run the following command to build an image with OverOps and push the image to ACR; for example, if your image name is web-app-w-overops:

    mvn package docker:build@with-overops -DpushImage

  • Run the following commands to deploy the web app to AAS:

    az webapp config container set -g ${EAST_US_GROUP} \
   -n ${EAST_US_WEBAPP_NAME} \
   --docker-custom-image-name ${ACR_LOGIN_SERVER}/web-app-w-overops \
   --docker-registry-server-url http://${ACR_LOGIN_SERVER} \
   --docker-registry-server-user ${ACR_USERNAME} \
   --docker-registry-server-password ${ACR_PASSWORD}

az webapp config set -g ${EAST_US_GROUP} \
   -n ${EAST_US_WEBAPP_NAME} \
   --linux-fx-version "DOCKER|${ACR_LOGIN_SERVER}/web-app-w-overops"

az webapp restart -g ${EAST_US_GROUP} -n ${EAST_US_WEBAPP_NAME}

  • Browse to your account portal in OverOps to see real-time diagnostic data.

Automate continuous integration and continuous deployment (CI/CD) using Jenkins

  1. Use an existing Jenkins instance, setup continuous delivery - build and configure pipeline using:

    a. A pipeline config file from the cloned repo

    b. The forked repo

    As part of the initial build, a Jenkins cluster with pipelines is setup in a Kubernetes cluster in Azure Container Service. You can see that at:

    http://${JENKINS_URL}

  1. Download the Jenkins CLI JAR from your Jenkins server; for example:

    curl -O http://${JENKINS_URL}/jnlpJars/jenkins-cli.jar

  2. Log into your Jenkins Dashboard, then build and deploy the development, test and production releases for these environments:

    java -jar jenkins-cli.jar -s \
   http://${JENKINS_URL}/ login
java -jar jenkins-cli.jar -s \
   http://${JENKINS_URL}/ \
   build 'movie-db-pipeline-for-dev' -f -v

    NOTE: Job movie-db-pipeline-for-dev is for dev environment you created in previous section.

    If you want to have test and prod environments, you will first need create them using below commands.

    cd ./deployment
source provision.sh --env test
source provision.sh --env prod

    Then you can build the test and prod environments using similar steps; e.g. build 'movie-db-pipeline-for-test' -f -v.

Continue to develop apps and rapidly deploy them

The steps in this section will walk you through the steps to make a simple change to your web app and see those changes reflected on the hom page when you browse to your web app.

  1. Using IntelliJ change the name of the web app in the ~/web-app/src/main/resources/static/index.html page; for example:

    <h1 class="cover-heading">Welcome to My Cool Movie DB on Azure!!!</h1>

  2. Using IntelliJ and Maven, build, deploy and test the web app:

    mvn spring-boot:run
curl http://localhost:8080/

  3. Using IntelliJ and Git, push changes to the forked repo when you are satisfied with the changes:

    git push origin master

  4. Watch Jenkins building and deploying dev and test releases on the Jenkins Dashboard (triggered by GitHub)

    Go to http://${JENKINS_URL}

  5. Trigger a new build of job movie-db-pipeline-for-dev to deploy your latest changes to dev environment.

  6. Once these steps have been completed, you should see your updated title on the home page of your web app.

Scale apps

  1. Scale out Internet facing web apps

    az appservice plan update --number-of-workers 6 \
   --name ${EAST_US_WEBAPP_PLAN} \
   --resource-group ${EAST_US_GROUP}

az appservice plan update --number-of-workers 6 \
   --name ${WEST_EUROPE_WEBAPP_PLAN} \
   --resource-group ${WEST_EUROPE_GROUP}

Sample Application Summary

In review, this sample application utilized all of the following design concepts and technologies.

Web App Design

  • It is a Spring Boot app with an embedded Tomcat server
  • It can run in Linux Containers in Azure App Service
  • App uses Azure Redis Cache and accesses it using Spring Data Redis

NOTE: In the future the app secrets will be stored in an Azure Key Vault.

Data App Design

  • It is a Spring Boot app with an embedded Tomcat server
  • It can run in Kubernetes clusters in Azure Container Service
  • App uses Azure Redis Cache and accesses it using Spring Data Redis
  • App stores and fetches images into and from Azure Storage
  • App stores and reads app data into and from SQL MySQL-as-a-service using Spring JDBC

NOTE: In the future the app secrets will be stored in an Azure Key Vault.

Functions Design

  • For now, Node.js is being used instead of Java in this sample application for the Azure functions; this will be updated in the near future
  • They are used for independent micro computations
  • They are used for linking two disconnected units in a workflow

Troubleshooting

This section will document some of the issues which have been identified when trying to build and deploy this sample application; more will be added as different issues are discovered.

'Permission Denied' error when creating the data app

When you are attempting to build the data app, you might encounter the following error:

[ERROR] Failed to execute goal com.spotify:docker-maven-plugin:0.4.11:build (default-cli) on project data-app: Exception caught: java.util.concurrent.ExecutionException: com.spotify.docker.client.shaded.javax.ws.rs.ProcessingException: java.io.IOException: Permission denied

This error is caused when your user account does not have permissions to create the necessary socket for Docker; more details are available in Solving Docker permission denied while trying to connect to the Docker daemon socket.

'Password Complexity' errors when provisioning the sample application

Some of the Azure services have various password complexity requirements; as a result, you may encounter various errors related to password complexity when you are running the provision scripts. In order to avoid or resolve these issues, you should ensure that the passwords which you choose adhere to standards; for example, at the very least you should choose passwords which have a mixture of uppercase letters, lowercase letters, numbers, and punctuation.

Contributing

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Disclaimer

Note: The sample code, scripts, and documentation in this sample application are not supported under any Microsoft standard support program or service. This sample application is provided AS IS without warranty of any kind. Microsoft disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of this sample application remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use this sample application, even if Microsoft has been advised of the possibility of such damages.

About

Sample movie database app built using Java on Azure

Resources

Readme

License

MIT license
Activity

Stars

0 stars

Watchers

1 watching

Forks

1 fork
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • Java 38.4%
  • Shell 28.3%
  • Groovy 14.4%
  • HTML 12.9%
  • Batchfile 2.8%
  • CSS 2.3%
  • JavaScript 0.9%