Sign in Subscribe
API Gateway

Running Kong on Microsoft Azure

Learn how you can easily run Kong on Microsoft Azure and configure it to serve traffic from Open Brewery DB secured with API key authentication.

Tom Kerkhove

• 13 min read
Running Kong on Microsoft Azure

TL;DR - Learn how you can easily run Kong on Microsoft Azure and configure it to serve traffic from Open Brewery DB secured with API key authentication.

Recently I wrote about Kong, an open-source API gateway that is gaining a lot of momentum in the API world.

Today, I will show you how easy it is to get a single-node Kong cluster up and running on Microsoft Azure with Azure Container Instances & Azure Database for PostgreSQL.

Running Kong on Azure

Hosting our Kong data store in Azure

Before we can start running our Kong cluster we need to provide a data store for it first. As of today, Kong allows you to use Cassandra or PostgreSQL.

One option would be to spin up VMs or deploy them as containers and host them ourselves, but I'm more of a "Microsoft will handle that for me" type of guy. I try to avoid having to maintain infrastructure while I don't need to.

That's why we can use these two PaaS offerings in Azure - Azure Database for PostgreSQL or Azure Cosmos Db. They both give you a database without having to worry about running it, we can just use it and pay for the service tier that we want.

Creating our data store instance

Unfortunately, Kong does not support using Azure Cosmos Db as a data store due to some missing features in the Cassandra API. If you want to use this, head over and give a :+1:!

However, setting up a new Azure Database for PostgreSQL database is super easy:

  • Go to the Azure Portal
  • Create a new “Azure Database for PostgreSQL” instance
  • Go to “Connection Security” and enable access to Azure services
  • Create a new database called “kong” by using your favorite tool

Running our Kong cluster on Azure Container Instances

Now that our data store is provisioned we are ready to roll! The only thing we need to do first is to run the initial database migrations!

In order to do that, we need to run an instance of the Kong image against our DB as a one-off job.

One possibility to achieve this is to run it locally, but that would be boring and less efficient.

Azure Container Instances is a perfect fit for this where we can create an instance via the CLI and tell it to die afterward:

⚡ tkerkhove@tomkerkhove C:\kong-on-azure
❯  az container create --name kong-migrations \
                       --resource-group kong-sandbox \
                       --image kong:latest \
                       --restart-policy Never \
                       --environment-variables KONG_PG_HOST="<instance-name>.postgres.database.azure.com" \
                                               KONG_PG_USER="<username>" \
                                               KONG_PG_PASSWORD="<password>" \
                       --command-line "kong migrations bootstrap"

Next, we can start our single-node cluster on Azure as well. This can basically be any container platform on Azure - Be it Azure Service Fabric Mesh, Azure Container Instances, Azure Kubernetes Service or Azure Web Apps for Containers.

To keep it simple, we'll stick with Azure Container Instances for now where we can provision it via the CLI:

⚡ tkerkhove@tomkerkhove C:\kong-on-azure
❯  az container create --name kong-gateway /
                       --dns-name-label kong-gateway /
                       --resource-group kong-sandbox /
                       --image kong:latest /
                       --port 8000 8443 8001 8444 /
                       --environment-variables KONG_PG_HOST="<instance-name>.postgres.database.azure.com" /
                                               KONG_PG_USER="<username>" /
                                               KONG_PG_PASSWORD="<password>" /
                                               KONG_PROXY_ACCESS_LOG="/dev/stdout" /
                                               KONG_ADMIN_ACCESS_LOG="/dev/stdout" /
                                               KONG_PROXY_ERROR_LOG="/dev/stderr" /
                                               KONG_ADMIN_ERROR_LOG="/dev/stderr" /
                                               KONG_ADMIN_LISTEN="0.0.0.0:8001, 0.0.0.0:8444 ssl"

This will expose a Kong instance on kong-gateway.westeurope.azurecontainer.io serving the proxy on port 8000 (HTTP) & 8443 (HTTPS), but also management API on 8001 (HTTP) & 8444 (HTTPS).

Careful - You should make sure nobody can access your Admin API on the default ports as this has security implications.

We can get the cluster configuration via the API:

⚡ tkerkhove@tomkerkhove C:\kong-on-azure
❯  curl -i http://kong-gateway.westeurope.azurecontainer.io:8001/

We're ready to roll! We now have a Kong gateway running in Microsoft Azure ready to be consumed.

Serving Open Brewery DB API to our customers via Kong Proxy

Now that our proxy is ready we will configure it to serve data from the Open Brewery DB API to our customers.

In order to do that, we will create a breweries service which can be served for all requests to apis.breweries.com.

Kong Routing Concept

Let's start by creating our breweries service which will get data from https://api.openbrewerydb.org/breweries:

⚡ tkerkhove@tomkerkhove C:\kong-on-azure
❯  curl -i -X POST --url http://kong-gateway.westeurope.azurecontainer.io:8001/services/ --data 'name=breweries' --data 'url=https://api.openbrewerydb.org/breweries'
HTTP/1.1 201 Created
Date: Tue, 12 Feb 2019 17:24:24 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/1.0.3
Content-Length: 273

{"host":"api.openbrewerydb.org","created_at":1549992264,"connect_timeout":60000,"id":"ce43c57c-322e-41a5-99fb-0e59d08dc67b","protocol":"https","name":"breweries","read_timeout":60000,"port":443,"path":"\/breweries","updated_at":1549992264,"retries":5,"write_timeout":60000}

Once the service is there, we can route traffic to it via a route. This will make sure that the consumer his request ends up with the correct service.

We can configure this by adding a route to the service that we've just created:

⚡ tkerkhove@tomkerkhove C:\kong-on-azure
❯  curl -i -X POST --url http://kong-gateway.westeurope.azurecontainer.io:8001/services/breweries/routes --data 'hosts[]=apis.breweries.com'
HTTP/1.1 201 Created
Date: Tue, 12 Feb 2019 17:24:59 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/1.0.3
Content-Length: 355

{"created_at":1549992299,"methods":null,"id":"8233554a-c66e-4411-9f83-85f2b9f15473","service":{"id":"ce43c57c-322e-41a5-99fb-0e59d08dc67b"},"name":null,"hosts":["apis.breweries.com"],"updated_at":1549992299,"preserve_host":false,"regex_priority":0,"paths":null,"sources":null,"destinations":null,"snis":null,"protocols":["http","https"],"strip_path":true}

Note that in this case we are telling it to use the hosts header as a routing mechanism to determine the match, but you can also use routing via uri path or method as documented here.

That's how easy it is to proxy traffic to an upstream system! If we now do a GET request with apis.breweries.com as a Host header we will get a list of breweries:

⚡ tkerkhove@tomkerkhove C:\kong-on-azure
❯  curl -i -X GET --url http://kong-gateway.westeurope.azurecontainer.io:8000/ --header 'Host: apis.breweries.com'
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Date: Tue, 12 Feb 2019 17:25:03 GMT
Set-Cookie: __cfduid=d89d5fb6dde7762abb7e722ecd2ac62bb1549992303; expires=Wed, 12-Feb-20 17:25:03 GMT; path=/; domain=.openbrewerydb.org; HttpOnly; Secure
Cache-Control: max-age=86400, public
Etag: W/"fc69bfa7204e398f373a77c558de1ad8"
X-Request-Id: 0163736f-f302-45fd-aecb-60fbeaaee332
X-Runtime: 0.342859
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
Via: kong/1.0.3
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 4a80c0966d3834a0-LHR
X-Kong-Upstream-Latency: 836
X-Kong-Proxy-Latency: 110

[{"id":2,"name":"Avondale Brewing Co","brewery_type":"micro","street":"201 41st St S","city":"Birmingham","state":"Alabama","postal_code":"35222-1932","country":"United States","longitude":"-86.774322","latitude":"33.524521","phone":"2057775456","website_url":"http://www.avondalebrewing.com","updated_at":"2018-08-23T23:19:57.825Z","tag_list":[]},{"id":4,"name":"Band of Brothers Brewing Company","brewery_type":"micro","street":"1605 23rd Ave","city":"Tuscaloosa","state":"Alabama","postal_code":"35401-4653","country":"United States","longitude":"-87.5621551272424","latitude":"33.1984907123707","phone":"2052665137","website_url":"http://www.bandofbrosbrewing.com","updated_at":"2018-08-23T23:19:59.462Z","tag_list":[]}]

The world loves beer so there is a ton of them so I redacted the response but you get the idea!

As you might have noticed, the response includes not only the standard HTTP headers but also the ones from api.openbrewerydb.org and Kong giving more information.
This is always a hard choice to make - Should we include them or remove them. I'm always leaning towards reducing the responses to what is required as we don't want to leak information about our dependencies, but it's all configurable anyway.

Time to make our API a bit more advanced!

Making our API robust with Kong Plugins

We have a Kong gateway setup that will serve traffic from Open Brewery DB API to everybody who's interested - Great!

Since we want to monetize our API, we'd like to enforce consumers to authenticate first and return a correlation id which can be used to create a support ticket.

Unfortunately, we cannot change the upstream API to do this but luckily Kong provides the capability to deploy plugins.
These are a great way to extend existing systems by deploying logic on the gateway.

In our case we will deploy two of them:

  • correlation-id plugin - It will generate and include a correlation id in the X-Request-ID response header
  • key-auth plugin - It will enforce consumers to authenticate via an API key

As mentioned in my previous article, plugins can be applied on a variety of levels but today we will enforce them on all our services.

Automatically adding correlation ids

Installing plugins are only an API call away:

⚡ tkerkhove@tomkerkhove C:\kong-on-azure
❯  curl -i -X POST --url http://kong-gateway.westeurope.azurecontainer.io:8001/plugins/ --data "name=correlation-id"  --data "config.header_name=X-Request-ID" --data "config.generator=uuid#counter" --data "config.echo_downstream=true"

HTTP/1.1 201 Created
Date: Tue, 12 Feb 2019 17:34:28 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/1.0.3
Content-Length: 259

{"created_at":1549992868,"consumer":null,"id":"39c536b8-6109-4696-80df-958cd7cc708d","service":null,"enabled":true,"run_on":"first","config":{"echo_downstream":true,"header_name":"X-Request-ID","generator":"uuid#counter"},"route":null,"name":"correlation-id"}

As you can see, the only thing we need to do is specify which plugin we want to install, and how it should be configured.

When we call our breweries service again, you'll notice that the X-Request-Id is added automatically:

⚡ tkerkhove@tomkerkhove C:\kong-on-azure
❯  curl -i -X GET --url http://kong-gateway.westeurope.azurecontainer.io:8000/ --header 'Host: apis.breweries.com'
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Tue, 12 Feb 2019 17:36:14 GMT
X-Request-Id: 684ef960-3bac-4b1f-bb49-5ba0d3da31b1#3
X-Runtime: 0.220337
X-Kong-Upstream-Latency: 606
X-Kong-Proxy-Latency: 0

[{"id":2,"name":"Avondale Brewing Co","brewery_type":"micro","street":"201 41st St S","city":"Birmingham","state":"Alabama","postal_code":"35222-1932","country":"United States","longitude":"-86.774322","latitude":"33.524521","phone":"2057775456","website_url":"http://www.avondalebrewing.com","updated_at":"2018-08-23T23:19:57.825Z","tag_list":[]},{"id":4,"name":"Band of Brothers Brewing Company","brewery_type":"micro","street":"1605 23rd Ave","city":"Tuscaloosa","state":"Alabama","postal_code":"35401-4653","country":"United States","longitude":"-87.5621551272424","latitude":"33.1984907123707","phone":"2052665137","website_url":"http://www.bandofbrosbrewing.com","updated_at":"2018-08-23T23:19:59.462Z","tag_list":[]}]

Enforcing API key authentication

Kong comes with a variety of security plugins to enforce authentication & handle authorization.

In our case we will use the key-auth plugin to enforce API key authentication:

⚡ tkerkhove@tomkerkhove C:\kong-on-azure
❯  curl -i -X POST --url http://kong-gateway.westeurope.azurecontainer.io:8001/services/breweries/plugins/ --data 'name=key-auth'
HTTP/1.1 201 Created
Date: Tue, 12 Feb 2019 17:31:14 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/1.0.3
Content-Length: 324

{"created_at":1549992674,"consumer":null,"id":"aa61e0dd-b0bb-4a76-a3c1-dbfac5a38a61","service":{"id":"ce43c57c-322e-41a5-99fb-0e59d08dc67b"},"enabled":true,"run_on":"first","config":{"key_in_body":false,"key_names":["apikey"],"anonymous":null,"hide_credentials":false,"run_on_preflight":true},"route":null,"name":"key-auth"}

Once the plugin is installed, all requests are forced to specify an API key or will receive an HTTP 401 Unauthorized.

⚡ tkerkhove@tomkerkhove C:\kong-on-azure
❯  curl -i -X GET --url http://kong-gateway.westeurope.azurecontainer.io:8000/?by_name=cooper --header 'Host: apis.breweries.com'
HTTP/1.1 401 Unauthorized
Date: Tue, 12 Feb 2019 17:33:10 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
WWW-Authenticate: Key realm="kong"
Content-Length: 41
Server: kong/1.0.3

{"message":"No API key found in request"}

In order to have an API key, we need to create a consumer for the user and generate a key for him.

Let's first create a consumer on the gateway for tom:

⚡ tkerkhove@tomkerkhove C:\kong-on-azure
❯  curl -i -X POST http://kong-gateway.westeurope.azurecontainer.io:8001/consumers/ --data "username=tom"
HTTP/1.1 201 Created
Date: Tue, 12 Feb 2019 17:32:12 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/1.0.3
Content-Length: 103

{"custom_id":null,"created_at":1549992732,"username":"tom","id":"499a5ab5-cae4-42de-af15-d3b476afc34e"}

Once created, we can create a key on the consumer:

⚡ tkerkhove@tomkerkhove C:\kong-on-azure
❯  curl -i -X POST http://kong-gateway.westeurope.azurecontainer.io:8001/consumers/tom/key-auth
HTTP/1.1 201 Created
Date: Tue, 12 Feb 2019 17:32:38 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/1.0.3
Content-Length: 167

{"consumer":{"id":"499a5ab5-cae4-42de-af15-d3b476afc34e"},"created_at":1549992758,"key":"qiKAX9iogSlu6pKxyfbSCp2LjPqzwC3k","id":"ca0c25ea-9051-475a-b602-6ef7b009231c"}

When we call the service again and add apikey=qiKAX9iogSlu6pKxyfbSCp2LjPqzwC3k as a query parameter we receive a list of breweries again:

⚡ tkerkhove@tomkerkhove C:\kong-on-azure
❯  curl -i -X GET --url http://kong-gateway.westeurope.azurecontainer.io:8000/?apikey=qiKAX9iogSlu6pKxyfbSCp2LjPqzwC3k --header 'Host: apis.breweries.com'
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Tue, 12 Feb 2019 17:36:14 GMT
X-Request-Id: 684ef960-3bac-4b1f-bb49-5ba0d3da31b1#3
X-Runtime: 0.220337
X-Kong-Upstream-Latency: 606
X-Kong-Proxy-Latency: 0

[{"id":2,"name":"Avondale Brewing Co","brewery_type":"micro","street":"201 41st St S","city":"Birmingham","state":"Alabama","postal_code":"35222-1932","country":"United States","longitude":"-86.774322","latitude":"33.524521","phone":"2057775456","website_url":"http://www.avondalebrewing.com","updated_at":"2018-08-23T23:19:57.825Z","tag_list":[]},{"id":4,"name":"Band of Brothers Brewing Company","brewery_type":"micro","street":"1605 23rd Ave","city":"Tuscaloosa","state":"Alabama","postal_code":"35401-4653","country":"United States","longitude":"-87.5621551272424","latitude":"33.1984907123707","phone":"2052665137","website_url":"http://www.bandofbrosbrewing.com","updated_at":"2018-08-23T23:19:59.462Z","tag_list":[]}]

Getting an overview of installed plugins

It's always good to keep track of what plugins are installed, certainly if they are applied on different levels.

Kong provides you with the capability to list all of the installed plugins, for example here you can see all of them on a gateway level:

⚡ tkerkhove@tomkerkhove C:\kong-on-azure
❯  curl -i http://kong-gateway.westeurope.azurecontainer.io:8001/plugins
HTTP/1.1 200 OK
Date: Tue, 12 Feb 2019 17:37:22 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/1.0.3
Content-Length: 607

{"next":null,"data":[{"created_at":1549992868,"consumer":null,"id":"39c536b8-6109-4696-80df-958cd7cc708d","service":null,"name":"correlation-id","run_on":"first","enabled":true,"route":null,"config":{"echo_downstream":true,"header_name":"X-Request-ID","generator":"uuid#counter"}},{"created_at":1549992674,"config":{"key_in_body":false,"run_on_preflight":true,"anonymous":null,"hide_credentials":false,"key_names":["apikey"]},"id":"aa61e0dd-b0bb-4a76-a3c1-dbfac5a38a61","service":{"id":"ce43c57c-322e-41a5-99fb-0e59d08dc67b"},"name":"key-auth","run_on":"first","enabled":true,"route":null,"consumer":null}]}

Conclusion

In a matter of minutes, we were able to spin up a single-node Kong cluster that is running on Microsoft Azure and serving traffic from www.openbrewerydb.org!

By using Azure Database for PostgreSQL we could easily spin up a PostgreSQL database to store the metadata for our gateway that is running on Azure Container Instances.

We've also seen how easy it is to create a service, route consumer traffic to it and make it more robust with plugins.

It's good to note that Microsoft Azure is not an officially supported cloud runtime by Kong (yet), but you can find all Azure related docs on Kong Hub - Let me know if you're missing something!

Although this example is not rocket science I think it shows how easy it is to use Kong and get started quickly, however, I've found that the documentation is very API centric and sometimes lacks the high-level picture of how everything ties together - But that can be improved!

Later on, we'll have a look at how Kong & Azure API Management relates to each other and where they both shine.

Thanks for reading,

Tom.

Sign up for more like this.

Enter your email
Subscribe