Azure Talking Azure Data Factory on RunAs Radio A while back I wrote about how you can use Azure Data Factory to build next generation of data integration piplines [https://blog.tomkerkhove.be/2017/12/14/azure-data-factory-2-0-next-generation-data-integration/] and how is it compares to Azure Logic Apps [https://blog.tomkerkhove.be/2017/12/22/azure-data-factory-2-0-how-is-it-different-from-azure-logic-apps/] . Recently I was thrilled
Microsoft Azure Featured Building systems in the cloud > TL;DR - Things change, and so does the cloud. New services are being added and integration between services is being improved, but services also become deprecated. We need to embrace change and design for it. Our industry has shifted quite a lot in the past recent years where we
Integration Azure Data Factory 2.0 - How is it different from Azure Logic Apps? > TL;DR - Azure Logic Apps & Azure Data Factory are both orchestrators, but how do they differ from each other? Well, combining both are the sweet spot. In my previous post [https://blog.tomkerkhove.be/2017/12/14/azure-data-factory-2-0-next-generation-data-integration/] , we've went through the new features of Azure Data Factory 2.
Data Azure Data Factory 2.0 - Next Generation of Data Integration > TL;DR - Microsoft announced Azure Data Factory v2 at Ignite bringing that enables more data integration scenarios and brings SSIS into the cloud. Azure Data Factory is one of those services in Azure that is really great but that doesn't get the attention that it deserves. It is a
Security Azure Key Vault made simple with Azure AD Managed Service Identity (MSI) > TL;DR - Azure Key Vault is hard but that's because you need to understand & implement the authentication with Azure AD. That's why Azure AD Managed Service Identity (MSI) now makes this a lot easier for you. There is no reason anymore not to use Azure Key Vault. Update (29/
Scalability Autoscaling your platform with Azure Monitor > TL;DR - Autoscaling is a great way to not only optimize your costs but also a flexible way of doing asynchronous processing.We will look at how Azure Monitor Autoscale allows you to define autoscaling rules, what the caveats are and what would be good additions to the service
Security Security & Data Announcements at Ignite 2017 A few weeks ago, Microsoft held another edition of its Ignite conference in Orlando, FL. After going through most of the announcements and digesting them I found that there were a couple of interesting ones in the security & data space. Let's have a closer look. Introducing Virtual Network Service Endpoints
Microsoft Azure Exploring Azure Event Grid - Are Service Bus Topics Dead? > TL;DR - In this 3rd and last article on Azure Event Grid, we'll have a look at how this relates to Azure Service Bus Topics and why they are still relevant. Read my introduction on Azure Event Grid here [https://blog.tomkerkhove.be/2017/08/22/exploring-azure-event-grid/]. No, Service
Microsoft Azure Exploring Azure Event Grid - Looking ahead > TL;DR - In this 2nd article on Azure Event Grid, we'll have a look at what I'd like to see being added to Azure Event Grid. Read my introduction on Azure Event Grid here [https://blog.tomkerkhove.be/2017/08/22/exploring-azure-event-grid/]. With a nice foundation of Event Publishers
Microsoft Azure Exploring Azure Event Grid > TL;DR - Azure Event Grid is here - In this first article we'll have a look at what it is, dive into the details and have a look at certain new scenarios. Last week Microsoft announced Azure Event Grid (Preview) [https://azure.microsoft.com/en-us/blog/introducing-azure-event-grid-an-event-service-for-modern-applications/] , an event-driven
Security Using Azure Functions? Don't use Read-Only Resource Locks, for now. It's a good practice to secure your Azure resources. One of the approaches to achieve this is by using Resource Locks [https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-lock-resources] . These allow you to add Read-Only or Delete locks on your resources to prevent somebody from changing or removing it. Unfortunately,
Community Featured Speaking at NDC Sydney 2017 on Azure Data Lake Are you going to NDC Sydney August 14th - 18th 2017? So will I! I have the honor to do a talk on Azure Data Lake & Analtics called "Analyzing StackExchange data with Azure Data Lake". By using a data dump of the well-known StackExchange websites, I will show you how
Logic Apps Using multiple operations in one controller in custom API App & Azure Logic Apps TL;DR - In my Web API I was using one controller with multiple operations that had the same method names which generated invalid Swagger. Here is how I found out what the issue was, how I fixed it and how I could use all my operation in Azure Logic
Conference Mastering Azure Analytics, Integration of Things, Techorama and more! It's been a while since I've written a new post for the simple reason that I didn't have any time left. Behind the scenes I was doing a ton of things that you might like. Here we go! Mastering Azure Analytics by Zoiner Tejada I had the oppertunity and honor
Monitoring Talking about Azure Application Monitoring on .NET Rocks! Recently I was a guest on .NET Rocks to talk about Application Monitoring on Microsoft Azure but we started off by talking about another challenging task in Azure - Keeping track of what resources are depending on each other and how they map to your application architecture. Unfortunately there is
Microsoft Azure Making Secrets, Certificates and Keys Easier with Sentinel for Azure Key Vault One of my favorite services in Azure is Key Vault [https://docs.microsoft.com/en-us/azure/key-vault/key-vault-whatis], of which I have written about a while back [http://tomkerkhove.ghost.io/2015/07/22/securing-sensitive-data-with-azure-key-vault/] . Since then they even added support for creating, managing and monitoring your Azure Key Vault
Automation Handling Azure Alerts with Azure Logic Apps When maintaining dozens, or even hundreds, of Azure resources it is crucial to keep track of what is going on and how they behave. To achieve this, Azure provides a set of functionalities and services to help you monitor them. One of those functionalities is the ability to configure Azure
IoT TechDays NL 2016 - Building your scalable & secure Internet-of-Things on Microsoft Azure A couple of weeks ago I did a talk about designing & building an Internet-of-Things solution on Azure that aims at scalability & security at TechDays NL. Unfortunately there is no silver bullet for designing IoT systems, well there is no silver bullet in general. The talk is aimed to ask the
Data Determining what data storage fits your needs in Azure Microsoft Azure offers a great variaty of data stores in different flavors. While this is great it also makes it a lot harder to determine which one fits your needs best. Today I found this great flow chart that helps you tackle this problem by focusing on data structure, the
Code Quality Improving code quality with code reviews Today I will talk about how we are currently using code reviews to build better solutions and how it gives me more confidence in the quality that we are shipping. Working in teams brings several challenges, one of them is a mixture of coding styles causing code inconsistency across your
Service Fabric "Interface id -XXX is not implemented by object XYZ" in Service Fabric Recently I was working on a Service Fabric project where I was using Service Remoting to communicate from one service to another by using the ServiceProxy. While refactoring my services I got the following exception over and over again: > Interface id -103369040 is not implemented by object Codit.MyOtherService The
Big Data Analyzing StackExchange data with Azure Data Lake - Storing the data Before we can start analysing StackExchange sample data, we'll need to get our data in Azure. There are several options for data storage in Azure, each with a specific goal. For data analytics -espacially with Azure Data Lake Analytics- Azure Data Lake Store is the de facto. In this blog
Big Data Analyzing StackExchange data with Azure Data Lake - Introduction A few weeks ago I wrote about what Microsoft announced on Azure Data Lake [https://tomkerkhove.ghost.io/2015/10/22/exploring-azures-data-lake/] and what you can expect. As of Wednesday 28th of October, Azure Data Lake Store & Analytics are now in public preview allowing you to try it out yourself.
Data Analytics Automated builds for Azure Data Lake Analytics extensibility Recently I have been working on an open-source project [http://github.com/CoditEU/usql] around Azure Data Lake Analytics. It will contain custom U-SQL components that you can use in your own projects and already contains a Xml Attribute Extractor. To build my code I'm using MyGet.org [https://www.
Microsoft Azure Exploring Azure's Data Lake Recently Microsoft revealed their plans for Azure Data Lake and how it groups several big data services in Azure under one umbrella. In this blog post I'll walk you through what it offers and how it enable you to re-use your existing skills to analyse big data. What is Azure
