Deploy Meilisearch into Azure Container Apps

Last week I was working on a proof of concept solution which includes a service responsible to provide a simple front-facing search component for a hardware tools website. During the research, I stumbled upon various options and I wanted to try deploying Meilisearch on Azure Container Apps as it meets most of the requirements for the search service within the solution.

Overview of Meilisearch

Meilisearch is a RESTful search API that offers a fast and instant search experience (search as you type), it is designed for a vast majority of needs of small-to-medium businesses with little configuration needed during installation yet with high customization.
Meilisearch is an open-source project built using Rust with more than 29K stars on GitHub and support for various SDKs including dotnet.

Meilisearch on Azure Container Apps

Meilisearch can be deployed on Azure on different options, Meilisearch container image can be deployed on Azure App Services as per the documentation, on this post I will go over the steps needed to prepare the Bicep template needed to deploy a Meilisearch container image into Azure Container Apps and use storage mounts in Azure Container Apps to permanently host Meilisearch database into Azure Files.

The source code used for this post exists on GitHub.

Deploying Meilisearch on Azure Container Apps

In this post, I will go over the Bicep template needed to deploy Meilisearch into Azure Container Apps and any important notes needed for deployment.

You can click on the button below to deploy a Meilisearch instance into Azure Container Apps. This is the final result of the Bicep templates we are going to build:

If you are new to Bicep, you can check my previous post on how to deploy Container Apps using Bicep.

Step 1: Define Azure Storage Resource

Azure storage is needed to create a file share service under it, the file share service will allow us to mount a file share from Azure Files as a volume inside the Meilisearch container. This means that Meilisearch DB and its configuration files are written into the container volume location are persisted in the file share, this is durable storage so if the container is restarted, crashed or a new revision is deployed; the files on the share will not be impacted and the new provisioned container will find the files on the configured volume.

To create a file share service under Azure storage, create a directory named “deploy/modules”, add a new file named “storage.bicep” and use the code below:

@description('The name of your application')

param applicationName string

@description('The Azure region where all resources in this example should be created')

param location string = resourceGroup().location

@description('A list of tags to apply to the resources')

param resourceTags object

@description('The name of the container to create. Defaults to applicationName value.')

param containerName string = applicationName

@description('The name of the Azure file share.')

param shareName string

@description('The name of storage account')

param storageAccountName string

resource storageAccount 'Microsoft.Storage/storageAccounts@2021-09-01' = {

location: location

tags: resourceTags

sku: {

}

kind: 'StorageV2'

}

resource blobServices 'Microsoft.Storage/storageAccounts/blobServices@2021-09-01' = {

parent: storageAccount

}

resource storageContainer 'Microsoft.Storage/storageAccounts/blobServices/containers@2021-09-01' = {

parent: blobServices

}

resource fileServices 'Microsoft.Storage/storageAccounts/fileServices@2021-09-01' = {

parent: storageAccount

}

resource permanentFileShare 'Microsoft.Storage/storageAccounts/fileServices/shares@2022-05-01' = {

parent: fileServices

properties: {

accessTier: 'TransactionOptimized'

enabledProtocols: 'SMB'

shareQuota: 1024

}

var storageKeyValue = storageAccount.listKeys().keys[0].value

output storageAccountName string = storageAccount.name

output id string = storageAccount.id

output apiVersion string = storageAccount.apiVersion

output storageKey string = storageKeyValue

Looking at the code above, notice that we’ve created a file share with the access tier “TransactionOptimized”, you can use “Premium” as it is backed by SSD drives and provides low latency. The size of the file share is set to 1024 gigabytes (1TB).

Step 2: Define Azure Log Analytics Workspace Resource

Add a new file named “logAnalyticsWorkspace.bicep” under the folder “modules”, and use the code below, the log analytics workspace is needed by the Container Apps Environment,

@description('The name of your Log Analytics Workspace')

param logAnalyticsWorkspaceName string

@description('The Azure region where all resources in this example should be created')

param location string = resourceGroup().location

@description('A list of tags to apply to the resources')

param resourceTags object

resource logAnalyticsWorkspace'Microsoft.OperationalInsights/workspaces@2021-12-01-preview' = {

tags: resourceTags

location: location

properties: any({

retentionInDays: 30

features: {

searchVersion: 1

}

sku: {

}

})

}

var sharedKey = listKeys(logAnalyticsWorkspace.id, logAnalyticsWorkspace.apiVersion).primarySharedKey

output workspaceResourceId string = logAnalyticsWorkspace.id

output logAnalyticsWorkspaceCustomerId string = logAnalyticsWorkspace.properties.customerId

output logAnalyticsWorkspacePrimarySharedKey string = sharedKey

Step 3: Define an Azure Container Apps Environment Resource

Add a new file named “acaEnvironment.bicep” under the folder “modules” and use the code below:

@description('The name of Azure Container Apps Environment')

param acaEnvironmentName string

@description('The Azure region where all resources in this example should be created')

param location string = resourceGroup().location

@description('A list of tags to apply to the resources')

param resourceTags object

param logAnalyticsWorkspaceCustomerId string

@secure()

param logAnalyticsWorkspacePrimarySharedKey string

resource environment 'Microsoft.App/managedEnvironments@2022-03-01' = {

location: location

tags: resourceTags

properties: {

appLogsConfiguration: {

destination: 'log-analytics'

logAnalyticsConfiguration: {

customerId: logAnalyticsWorkspaceCustomerId

sharedKey: logAnalyticsWorkspacePrimarySharedKey

}

output acaEnvironmentId string = environment.id

Step 4: Define an Azure Container Apps Environment Storages Resource

Add a new file named “acaEnvironmentStorages.bicep” under the folder “modules” and use the content below:

@description('The name of Azure Container Apps Environment')

param acaEnvironmentName string

@description('The name of your storage account')

param storageAccountResName string

@description('The storage account key')

@secure()

param storageAccountResourceKey string

@description('The ACA env storage name mount')

param storageNameMount string

@description('The name of the Azure file share. Defaults to applicationName value.')

param shareName string

resource environment 'Microsoft.App/managedEnvironments@2022-03-01' existing = {

}

//Environment Storages

resource permanentStorageMount 'Microsoft.App/managedEnvironments/storages@2022-03-01' = {

parent: environment

properties: {

azureFile: {

accountName: storageAccountResName

accountKey: storageAccountResourceKey

shareName: shareName

accessMode: 'ReadWrite'

}

This is a key step to configure a storage definition of type AzureFile in the Container Apps Environment, within this file we have enabled the environment to use the Azure File share service for any Container App under this environment, we are setting the access mode to “ReadWrite” as we need to write and read files from the file share.

Step 5: Define an Azure Container Apps Resource

Add a new file named “containerApp.bicep” under the folder “modules” and use the content below:

param containerAppName string

param location string

param environmentId string

param containerImage string

param targetPort int

param containerRegistry string

param containerRegistryUsername string

param isPrivateRegistry bool

param registryPassName string

param minReplicas int = 0

param maxReplicas int = 1

@secure()

param secListObj object

param envList array = []

param revisionMode string = 'Single'

param storageNameMount string

param volumeName string

param mountPath string

param resourceTags object

param resourceAllocationCPU string

param resourceAllocationMemory string

resource containerApp 'Microsoft.App/containerApps@2022-06-01-preview' = {

location: location

tags: resourceTags

properties: {

managedEnvironmentId: environmentId

configuration: {

activeRevisionsMode: revisionMode

secrets: secListObj.secArray

registries: isPrivateRegistry ? [

{

server: containerRegistry

username: containerRegistryUsername

passwordSecretRef: registryPassName

}

] : null

ingress: {

external: true

targetPort: targetPort

transport: 'auto'

traffic: [

{

latestRevision: true

weight: 100

}

]

}

dapr: null

}

containers: [

{

image: containerImage

env: envList

volumeMounts: [

{

mountPath:mountPath

volumeName:volumeName

}

]

resources:{

cpu: json(resourceAllocationCPU)

memory: resourceAllocationMemory

}

]

volumes: [

{

storageName: storageNameMount

storageType: 'AzureFile'

}

]

scale: {

minReplicas: minReplicas

maxReplicas: maxReplicas

}

output fqdn string = containerApp.properties.configuration.ingress.fqdn

This module is responsible to deploy the actual Meilisearch container image to Container App, what we have done here is the following:

Configuring the ingress of the container app to be external ingress (accepts HTTP requests from the public internet).
Parameterizing the target port of the ingress controller, we will set the value in the next steps.
Parameterizing the Meilisearch container image which will be deployed on this container app, the parameter will hold the Meilisearch image from docker hub.
Parameterizing the compute resources CPU and memory of the container app.
Parameterizing the secrets and environment variables arrays.
Define one single storage volume of type AzureFile for the container app and parameterize the volume name and storage name mount.
Define one single volume mount in the container app and parameterize the mount path and volume name.
Lastly, outputting the FQDN of the provisioned container app as it will be the URL to access Meilisearch API.

Step 6: Define the Main module for the final deployment

Lastly, we need to define the Main Bicep module which will link the modules together, this will be the file that is referenced from AZ CLI command when creating the entire resources, to do so under the folder “deploy”, add a new file named “main.bicep” and use the code below:

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

targetScope = 'subscription'

//Azure Regions which Azure Container Apps available at can be found on this link:

//https://azure.microsoft.com/en-us/explore/global-infrastructure/products-by-region/?products=container-apps&regions=all

@description('The Azure region code for deployment resource group and resources such as westus, eastus, northeurope, etc...')

param location string = 'westus'

@description('The name of your search service. This value should be unique')

param applicationName string = 'meilisearch'

@description('The Container App CPU cores and Memory')

@allowed([

{

cpu: '0.25'

memory: '0.5Gi'

}

{

cpu: '0.5'

memory: '1.0Gi'

}

{

cpu: '0.75'

memory: '1.5Gi'

}

{

cpu: '1.0'

memory: '2.0Gi'

}

{

cpu: '1.25'

memory: '2.50Gi'

}

{

cpu: '1.5'

memory: '3.0Gi'

}

{

cpu: '1.75'

memory: '3.5Gi'

}

{

cpu: '2.0'

memory: '4.0Gi'

}

])

param containerResources object = {

cpu: '1.0'

memory: '2.0Gi'

}

@maxLength(4)

@description('The environment of deployment such as dev, test, stg, prod, etc...')

param deploymentEnvironment string = 'dev'

@secure()

@description('The Master API Key used to connect to Meilisearch instance')

@minLength(32)

param meilisearchMasterKey string = newGuid()

var resourceGroupName = '${applicationName}-${deploymentEnvironment}-rg'

var logAnalyticsWorkspaceResName = '${applicationName}-${deploymentEnvironment}-logs'

var environmentName = '${applicationName}-${deploymentEnvironment}-env'

var storageAccountName = '${take(applicationName,14)}${deploymentEnvironment}strg'

var shareName = 'meilisearch-fileshare'

var storageNameMount = 'permanent-storage-mount'

var meilisearchImageName = 'getmeili/meilisearch:v0.29'

var meilisearchAppPort = 7700

var dbMountPath = '/data/meili'

var volumeName = 'azure-file-volume'

var defaultTags = {

environment: deploymentEnvironment

application: applicationName

}

resource rg 'Microsoft.Resources/resourceGroups@2021-04-01' = {

location: location

tags: defaultTags

}

module storageModule 'modules/storage.bicep' = {

scope: resourceGroup(rg.name)

params: {

storageAccountName: storageAccountName

location: rg.location

applicationName: applicationName

containerName: applicationName

shareName: shareName

resourceTags: defaultTags

}

module logAnalyticsWorkspace 'modules/logAnalyticsWorkspace.bicep' = {

scope: resourceGroup(rg.name)

params: {

logAnalyticsWorkspaceName: logAnalyticsWorkspaceResName

location: rg.location

resourceTags: defaultTags

}

module environment 'modules/acaEnvironment.bicep' = {

scope: resourceGroup(rg.name)

params: {

acaEnvironmentName: environmentName

location: rg.location

logAnalyticsWorkspaceCustomerId: logAnalyticsWorkspace.outputs.logAnalyticsWorkspaceCustomerId

logAnalyticsWorkspacePrimarySharedKey: logAnalyticsWorkspace.outputs.logAnalyticsWorkspacePrimarySharedKey

resourceTags: defaultTags

}

module environmentStorages 'modules/acaEnvironmentStorages.bicep' = {

scope: resourceGroup(rg.name)

dependsOn:[

environment

]

params: {

acaEnvironmentName: environmentName

storageAccountResName: storageModule.outputs.storageAccountName

storageAccountResourceKey: storageModule.outputs.storageKey

storageNameMount: storageNameMount

shareName: shareName

}

module containerApp 'modules/containerApp.bicep' = {

scope: resourceGroup(rg.name)

dependsOn: [

environment

]

params: {

containerAppName: applicationName

location: rg.location

environmentId: environment.outputs.acaEnvironmentId

containerImage: meilisearchImageName

targetPort: meilisearchAppPort

minReplicas: 1

maxReplicas: 1

revisionMode: 'Single'

storageNameMount: storageNameMount

mountPath: dbMountPath

volumeName: volumeName

resourceTags: defaultTags

resourceAllocationCPU: containerResources.cpu

resourceAllocationMemory: containerResources.memory

secListObj: {

secArray: [

{

value: meilisearchMasterKey

}

]

}

envList: [

{

secretRef: 'meili-master-key-value'

}

{

value: dbMountPath

}

]

}

output containerAppUrl string = containerApp.outputs.fqdn

What we have done is the following:

Defined a set of parameters so the end user can control the deployment of Meilisearch instance, parameters defined as the following:
- Location: The Azure region code (“westus”, “northeurope”, “australiacentral”, etc…). This should be a region where Azure container Apps and Azure Storage is available, you can check where Azure Container Apps are available on this link.
- Application Name: the name of the Meilisearch search service, this name will be part of the FQDN and will be used to set resource group name, storage, container app environment, and log analytics workspace.
- Container Resources: Container App CPU and Memory. Read here to understand more about those CPU/Memory combinations. The limits are soft limits and you can request to increase the quota by submitting a support request.
- Deployment Environment: Used to identify deployment resources (“dev”, “stg”, “prod”, etc…) and tag them with the selected environment, this has nothing to do with the capacity or performance of the resources provisioned. This will be useful if you are deploying multiple Meilisearch instances under the same subscription for dev/test scenarios.
- Meilisearch Master Key: This is the Master API Key used with the Meilisearch instance, minimum length is 32 characters. The recommendation is to generate a strong key, if not provided deployment template will generate a guide as the Master API key.
Defined a set of variables to be passed to the child modules to provision needed resources, variables defined are:
- Share Name: the name of the file share which will be created under Azure storage files share, this name will be passed to the Container Apps environment too to configure the storage definition of type AzureFile in the Container Apps environment.
- Storage Name Mount: The name of the storage mount associated with the Meilisearch container.
- Meilisearch Image Name: The Meilisearch docker image name “getmeili/meilisearch:v0.29” hosted in docker hub using tag ‘v0.29’ this is the latest tag at the time of writing this post.
- Meilisearch App Port: This is the port that Container App listens to for incoming requests, Meilisearch uses port 7700, when ingress is enabled on Container App; the ingress endpoint is exposed on port 443.
- DB Mount Path: The path “/data/meili” is used, this path represents the volume inside the Container App mounted to Azure File Share. You can change the path if needed but the path needs to be the same in an environment variable named “MEILI_DB_PATH“
- Volume Name: Logical name used to define the volume used in the Container App.
Notice how we are storing the “Meilisearch Master Key” securely in the Container Apps secrets, and using a “secretRef” in the environment variables to reference the secret. The name of the environment variable which stores the Master API Key must be “MEILI_MASTER_KEY“
Lastly, we are returning the provisioned Container App FQDN as a deployment output.

Step 7: Deploy Meilisearch Resources using Azure CLI

Now we are ready to deploy the resources using Azure CLI, to do so, open a PowerShell console and use the script below, don’t forget to set actual values for the placeholders.

az deployment sub create `

--template-file ./main.bicep `

--location WestUS `

--parameters '{ \"meilisearchMasterKey\": {\"value\":\"YOUR_MASTER_KEY\"}, \"applicationName\": {\"value\":\"YOUR_APP_NAME\"}, \"deploymentEnvironment\": {\"value\":\"dev\"}, \"location\": {\"value\":\"westus\"} }'

Note: You can use the “Deploy to Azure Button” highlighted above to deploy the resources.

If all is completed successfully you should see the resource group and the below 4 resources created under the subscription selected as the image below. To get the Container App FQDN, you can navigate to the Container App or you can get it from the deployment output tab.

Step 8: Test Deployed Meilisearch Instance

To test the deployed Meilisearch Instance, I’ve created a console application that uses the dotnet Meilisearch SDK and it creates an index named “movies” and indexes 40K documents using a JSON file (file is included in source code) or you can download it from this link.

The console application contains the below code:

using System.Text.Json;

namespace Meilisearch.Console

{

public class Movie

{

public int Id { get; set; }

public string Title { get; set; }

public string Poster { get; set; }

public string Overview { get; set; }

public IEnumerable<string> Genres { get; set; }

}

internal class Program

{

static async Task Main(string[] args)

{

MeilisearchClient client = new MeilisearchClient("https://<fqdn>.<location>.azurecontainerapps.io", "<MASTER API KEY>");

var options = new JsonSerializerOptions

{

PropertyNameCaseInsensitive = true

};

string jsonString = await File.ReadAllTextAsync(@"movies.json");

var movies = JsonSerializer.Deserialize<IEnumerable<Movie>>(jsonString, options);

var index = client.Index("movies");

var newSettings = new Settings

{

FilterableAttributes = new string[] { "genres" },

SortableAttributes = new string[] { "title" },

};

await index.UpdateSettingsAsync(newSettings);

await index.AddDocumentsAsync<Movie>(movies,"id");

}

What the console application does is the following:

Instantiating MeilisearchClient by passing the URL of the Container App and the Master API Key.
Reading the JSON content from the file.
Updating the settings and attributes of an index named “movies”.
Lastly, add the documents (movies) into the index named “movies”.

Meilisearch provides a PostMan collection that contains all the endpoints to configure your deployed Meilisearch instance and perform a search as well, Meilisearch PostMan collection can be downloaded from this link.

To verify that documents added successfully to the “movies” index, you can issue the below HTTP Get request and you see a list of movies returned.

GET /indexes/movies/documents

Host: <FQDN>.<Location>.azurecontainerapps.io

Authorization: Bearer <MASTER API KEY>

Follow me on Twitter @tjoudeh

References:

Meilisearch on Azure

Deploy Meilisearch into Azure Container Apps

Overview of Meilisearch

Meilisearch on Azure Container Apps

The source code used for this post exists on GitHub.

Deploying Meilisearch on Azure Container Apps

Step 1: Define Azure Storage Resource

Step 2: Define Azure Log Analytics Workspace Resource

Step 3: Define an Azure Container Apps Environment Resource

Step 4: Define an Azure Container Apps Environment Storages Resource

Step 5: Define an Azure Container Apps Resource

Step 6: Define the Main module for the final deployment

Step 7: Deploy Meilisearch Resources using Azure CLI

Step 8: Test Deployed Meilisearch Instance

Follow me on Twitter @tjoudeh

References:

Like this:

Related Posts

Comments

Leave a Reply Cancel reply

Recent Posts

Search

Share this:

Overview of Meilisearch

Meilisearch on Azure Container Apps

The source code used for this post exists on GitHub.

Deploying Meilisearch on Azure Container Apps

Step 1: Define Azure Storage Resource

Step 2: Define Azure Log Analytics Workspace Resource

Step 3: Define an Azure Container Apps Environment Resource

Step 4: Define an Azure Container Apps Environment Storages Resource

Step 5: Define an Azure Container Apps Resource

Step 6: Define the Main module for the final deployment

Step 7: Deploy Meilisearch Resources using Azure CLI

Step 8: Test Deployed Meilisearch Instance

Follow me on Twitter @tjoudeh

References:

Share this:

Like this:

Related Posts

Comments

Leave a Reply Cancel reply

Recent Posts

Tags

Search