Azure Web App is reporting "503: Service unavailable" after Azure DevOps Deployment

Introduction

Picture the scene: You've perfected your code, built an artefact in Azure DevOps, and deployed it to your web app, but now your website is returning "service unavailable." You've restarted the web app, but it is returning this error message far too quickly. There are no errors in Application Insights or any of the logs you would typically check for errors; for example, Umbraco Developers may also check the Umbraco Error Logs.

This blog explores some methods for investigating why the website may return "service unavailable" after a deployment.

App Offline file

Because locked files can not be overwritten during a deployment, the deployment process can be configured to release them using the app offline file (app_offline.htm).

Azur DevOps can be configured to automatically use this file using the "Take App Offline" flag. This flag is set to false by default. The "Take App Offline" flag is configured in YAML. The YAML below is an example of setting TakeAppOfflineFlag:true in the deployment task:

Or ticking the "Take App Offline" Checkbox in the Azure DevOps interface for the deployment task.

Setting "Take App Offline" to true will place an app_offline.htm file in the root directory of the Web App before the sync operation begins. The file will be removed after the sync operation completes successfully.

If the deployment process utilizes this mechanism and fails, the App_Offline.htm file will remain in the root directory of the Web App after the deployment has been completed. The root directory of a Windows-based Web App can be checked by accessing Kudu. Once Kudu has been loaded, select "Debug Console" and "CMD" from the menu.

Once the CMD window is open, type cd site\wwwroot in the Kudo Remote Execution Console, this will list all the files in the website root. The file browser above the Console will list all files alphabetically, allowing a visual check if the file app_offline.htm exists. If you don't want to scan the file list, type dir app_offline.htm /s in the Console.

If the app_offline.htm exists, deleting the file app_offline.htm, will allow the site to start; this can be done by typing del app_offline.htm in the Consoleor by clicking the no entry symbol next to the file. However, doing this without understanding why Azure DevOps failed to remove app_offline.htm as part of the deployment process means the site may not be in a stable.

Further investigation must be done by reviewing any errors reported in the Azure DevOps logs for the current release. The release logs are found on the releases page:

1. The releases page is found by going to https://dev.azure.com/{Organization}/{Project}/_release or by clicking "Releases" in the project menu.
2. Click on the last release to the environment.
3. Click "logs" in the menu.
4. Check for errors. Errors are shown with a red cross next to them and say failed.

A full log can be downloaded by clicking on the three dots next to the deployment time.

If Azure DevOps shows no errors, then the deployment can be reviewed in Kudu by navigating to site\deployments. The folder with its deployment date. There are three files within the folder, each of which will detail if the deployment is successful and if there are any errors.

Always complete post-deployment checks to determine if the deployment was successful.

Eventlog.xml

If the same code works in other environments, the issue may be environmental. However, it may be challenging to determine the exact problem without access to log files. In addition to application-level logging, Web Apps provide an "Eventlog.xml" file that contains details of unhandled errors in the Web App and key events like Microsoft patching.

To access eventlog.xml, logging in an Azure Web App needs to be enabled. For more information on enabling Azure App Service Logging, check out Enable diagnostics logging for apps in Azure App Service.

If there is no app_offline.htm but the site still serves "service unavailable". The next file to check is eventlog.xml. This file is found in the logfiles folder in Kudu To access this file once in the Kudo debug Console type cd \logfiles in the Console. Then click the pencil icon next to the filename eventlog.xml

This will list all recent events in chronological order. Scroll to the end of the file to see the most recent events. Always check that the date and time entry for the event being reviewed is around the deployment time.

An example event entry is below (with key information redacted)

Entries in the log with an error will contain an error trace allowing start up (or any other issues) to be investigated further. An example error entry is shown below:

Azure Front Door

Azure Front Door can also report service unavailable if it does not get a response from the WebApp within a set time. If the Web App takes longer than this time to boot up, Front Door will report the service as unavailable. For more information on changing the origin response timeout and resolving other issues with Azure Front Door, please check out Troubleshoot Azure Front Door's common issues.

Top Tips

• Once any issues have been resolved don't forget to update deployment pipelines and files to ensure they do not reoccur on the next deployment. 
• A quick way to download log files to your local machine is by using:  
  • Windows Apps: https://<mywebapp>.scm.azurewebsites.net/api/dump
  • Linux/ custom containers: https://.scm.azurewebsites.net/api/logs/docker/zip

Summary

A "503: Service unavailable error" after deploying a Web App can be frustrating, especially when the usual logs do not help with diagnosing the issues. By utilizing the app offline file and thoroughly investigating Azure DevOps logs and the eventlog.xml file, developers can systematically identify and resolve underlying problems. Ensuring that the deployment process is properly set up and that post-deployment checks are conducted will help maintain the stability and availability of web applications.

References

• App Offline file (app_offline.htm)
• AzureRmWebAppDeployment@2 - Azure App Service Deploy v2 task
• AzureRmWebAppDeployment@4 - Azure App Service deploy v4 task
• IISWebAppDeploymentOnMachineGroup@0 - IIS web app deploy v0 task
• Accessing the Kudu service