Start-of-Week Reading List

Mockadillo

The rise of microservices requires more manageable APIs, including testing them. Mockadillo is a new tool built for this exact purpose.

---

Series: How YOU can build a Serverless GraphQL API leveraging microservices

I think this is more like a reference-type article. The topic is so broad and rather complex, but only short read. However, it does have links to more details implementation of GraphQL and anything you need to build a serverless GraphQL microservices API.

---

First World Problems: Very long RMarkdown documents

If you use RMarkdown, this tricks teach you how to manage a very long RMarkdown document. If you haven’t use (or heard) about RMarkdown and you’re doing data visualization and big data stuff, definitely check it out.

---

How I (Accidentally) Made a Game Engine from Scratch with Vanilla JS

A great way to start learning about game programming. This covers basic game programming and written in pure JavaScript.

---

Getting started with Azure App Configuration

The journey of centralized configuration is never ending. Azure App Configuration is another attempt at this, seems very promising however. Today’s configuration is not just in an application, it’s everywhere, the VM, the CI/CD pipeline, etc, and it spans across different cloud provider. Azure App Configuration is built to handle this.

---

SQL Server and CI/CD Reading List

SQL SERVER – 5 Important Steps When Query Runs Slow Occasionally

Obviously, this is generalized and YMMV (your mileage may vary) -kinda experience. But at the least, if you don’t know where to start, this is the right place to look into.

---

HOW TO SAVE MONEY ON AZURE USING AZURE RESERVATIONS

Exactly as the title suggest, it’s using Azure Reservations. You reserve the resources for 1 or 3 year terms. The saving could go up to > 50% in some instances. Only available for few resources however, including Virtual Machine, SQL Data Warehouse, Cosmos DB, SQL Database, Azure Databricks.

---

What are SQL Database instance pools (preview)?

This is not to be confused with Azure SQL Database’s Elastic Pool. Instance pools is a little bit different, in the sense that its main purpose is to migrate (lift-and-shift or otherwise) multiple smaller databases. Instance Pools make it easier to handle this scenario by simplify many things like IP address allocation and deployment time.

---

Nice tests to run in CI before deploying any website

Not all the tests are created equally. Andy Li gives her perspective what to test in your Continuous Integration pipelines before the deployment.

---

Build a CI/CD pipeline for API Management, Part 1

Management of APIs can easily go out of control if it’s not done right. In this video, Miao Jiang talks about CI/CD part of API Management. The result is a clean, manageable CI/CD pipeline that can be deployed in any environment.

---

Free eBooks

Everyone gotta loves free stuff. Here are free eBooks that would improve your technical skills and knowledge.

---

Cognitive Complexity

Fairly short read. SonarQube, a company that makes an automatic code-review tool with the same name, has come up with an algorithm to determine how complex code is and the score to determine easy it’s to understand, called Cognitive Complexity.

---

Azure Security Best Practices

If you are in Azure environment, this is a really good resource to understand what Azure has to offer to secure your application and environment. From identity management, network, monitoring system, VMs and database, it covers pretty much everything security.

---

Learn Azure in a Month of Lunches

You can learn everything Azure in a month of lunches, that’s 30 days x 1 hour = 30 hours! Sounds pretty good deal to me!

---

What to Read?

Stateful Serverless: Long-Running Workflows with Durable Functions

Great post by Jeremy Likness on how to use Durable Functions in Azure Functions to create a long-running workflows. Yes, a serverless service can be used for long-running application. Some of the pattern implemented are: fan-out-fan-in, human interaction, etc. Although the example given is a game, it can definitely be applied in many Enterprise situation.

---

What’s the big deal with IAsyncEnumerable in .NET Core 3.0?

Ever wonder how you can stream data asynchronously? Now you can with IAsyncEnumerable<T>, a new feature in C# 8.0 and .NET Core 3.0.

---

Create dependent pipelines in your Azure Data Factory

One of the very common scenarios in data integration and movement is the dependency of one task to another. Azure Data Factory now offers option to run triggers based on dependency of another trigger. You can even specify offset and dependency size of a trigger.

---

The guide to Visual Studio Code shortcuts, higher productivity and 30 of my favourite shortcuts you need to learn

Want to code faster in Visual Studio Code, this will help you.

---

Solving Cold-Start disturbs serverless’ definition and it’s okay

Serverless has its advantage but cold-start is not one of them. With Azure Premium Functions, you can solve cold-start issue although technically, it’s no longer a serverless. Maxime made an argument when to use serverless or not.

---

IOT Challenge: Build IOT Device and Win Prizes – FREE Azure Sphere Kit

Hackster is hosting an IOT challenge: “Secure Everything with Azure Sphere”, build your IOT device and win prizes.

Head to: https://www.hackster.io/contests/SecureEverything

At the least, you can get free Azure Sphere Kit, worth $75 !!

This Week Reading List

Want to learn any programming language? — Write These 3 Simple Apps!

https://medium.com/@samuel.fare/want-to-learn-any-programming-language-write-these-3-simple-apps-5af8cd119921

Very good read if you want to learn new programming langugage. Sam Fare challenges you to build 3 simple apps with a new programming langauge: Hello World, Anagram and Pizza app. I’ll throw in one more as a bonus: a Todo app.

 

---

How to Become a Better Software Developer by Digging & Climbing

https://dmitripavlutin.com/become-better-software-developer-digging-climbing/

Even if you are an experienced developer, it’s still a very good read. Dmitri shares what he did to become a better developer (and we always have room to become a better developer). It’s a conceptual idea, which is why it’s a very good read, with some practical examples.

 

---

Event-driven analytics with Azure Data Lake Storage Gen2

https://azure.microsoft.com/en-us/blog/event-driven-analytics-with-azure-data-lake-storage-gen2/

If you are dealing with big data, the ingestion, processing, storage, consumption and ETL stuff, add this to your reading list. It outlines a way to use event-driven technique using Azure services to build data for analytics.

 

---

Azure solution architectures

https://azure.microsoft.com/en-us/solutions/architecture/

One of my favorites. This is more of a reference than a reading. Microsoft docs provides cloud architecture solutions (based on Azure services) for different scenarios and needs. Definitely check it out when you need to design and build a new system.

 

---

The Azure Infrastructure Architect Map

https://techcommunity.microsoft.com/t5/Azure-Developer-Community-Blog/The-Azure-Infrastructure-Architect-Map/ba-p/766268

Yet another good resource for a reference. This guides you through way-too-many Azure services based on your needs, sorta help you to decide what Azure service to use for a specific scenario. There’s also Solution Architect map and Security Architect map. Links are in the article.

 

---

Pipeline Pattern Implementations in C# .NET – Part 1

https://michaelscodingspot.com/pipeline-pattern-implementations-csharp/

Part 1 of 2 where Michael explains how to implement pipeline pattern in C#. This is not about `System.IO.Pipelines` library, but rather,
Part 2 – https://michaelscodingspot.com/pipeline-pattern-tpl-dataflow/

Monday Reading List

Learn how you can use GraphQL in .NET Core and C#

https://dev.to/dotnet/learn-how-you-can-use-graphql-in-net-core-and-c-4h96

Don’t worry if you don’t know what GraphQL is, this article introduces the basic of GraphQL and went further on how to use it in .NET Core. The NuGet package used allows you to expose C# class as query-able Graph API.

 

---

After Your Talk 🗣: The Power 💪 of Leverage

https://dev.to/azure/after-your-talk-the-power-of-leverage-2enj

If you’re into giving a talk kinda guy, you definitely should follow Jeremy Likness. He has lots of tips and tricks from how you get started to how to engage your audiences after the talk. This particular one is what you need to do after your talk. It’s part of the whole series of technical presentation guide.

 

---

Do-It-Yourself FAQ Bot

https://medium.com/microsoftazure/diy-faq-bot-707c6ee4d84b

Super cool and fairly easy way to build FAQ chat bot. Chloe also built fake boyfriend app which link you can find in the article. This FAQ chat bot involves Azure Bot Services and Twilio.

 

---

10 OOP & SOLID Design Principles for writing Clean Code

https://dev.to/javinpaul/top-10-object-oriented-design-principles-for-writing-clean-code-4pe1

After you have coded for a long time, it’s easy to forget the basic principles of OOP design. This article talks about how to write a clean code with OOP design pattern. Although example is in Java, the principles can be used in most languages. Some basic principles includes DRY, SOLID, Delegation, Program for Interface, etc.

 

---

Microfrontends: The Benefits of Microservices for Client-Side Development

https://thenewstack.io/microfrontends-the-benefits-of-microservices-for-client-side-development/

The world has not moved on from Microservices yet, now there’s Microfrontends! It’s micro-app but for frontends. You can mesh Angular, Reach, Vue, jQuery and everything else into a single-SPA, isn’t that amazing? If that sounds terrible, there’s a reason why Microfrontends makes sense. Read on.

 

---

5 Steps in Programming to Keep You From Getting Stuck

https://dev.to/ilonacodes/5-steps-in-programming-to-keep-you-from-getting-stuck-dlg

Everyone stuck at a problem every once a while. But who knows talking to a duck can help? Yes!

Azure App Service Auto-Healing

Auto-Healing

Azure App Service provide feature to auto-heal your service whenever it detects anomaly with the service.

Auto-Healing allows you to restart, log or perform custom actions (action) whenever a certain criteria about your App Service is met (trigger), for example, recycle service when slow requests is detected.

You define these triggers and actions to determine what triggers / actions to take.

Azure Web Sites

Before it was renamed to Azure App Service, the service is called Azure Web Sites. To enable auto-heal in Azure Web Sites, you would have to add configuration in the web.config file.

<system.webServer>
  <monitoring>
    <triggers>
      <requests count="100" timeInterval="00:05:00" />
    </triggers>
    <actions value="Recycle" />
  </monitoring>
</system.webServer>

To see different scenarios on how to use this, head over to this link. Some of the scenarios are:

• Recycle based on Request count (code example above).
• Recycle based on slow requests.
• Logging an event or recycle based on HTTP status code(s).
• Take custom actions or recycle / log event based on memory limit.

Azure App Service

In the new Azure App Service, however, auto-heal works little bit differently. Announced in September 2018 (https://azure.github.io/AppService/2018/09/10/Announcing-the-New-Auto-Healing-Experience-in-App-Service-Diagnostics.html), the auto-heal is now available through Azure Portal.

To get to Auto-Healing setting, go to Azure App Service > Diagnose and solve problems > Diagnostic Tools > Auto Healing.

Azure App Service offers two types of auto-heal. First is ProActive Auto-Healing which is enabled by default. And custom Auto-Healing.

You can turn off ProActive Auto-Healing under Azure Portal as well.

To add custom Auto-Healing setting, follow this link.

As the screenshot shows, the first tab (“Configure Mitigation Rules”) is the custom Auto-Healing settings. By default it’s turned off. The second tab (“ProActive Auto-Healing”) is ProActive Auto-Healing and it’s turned on by default.

Reference

https://docs.microsoft.com/en-us/azure/app-service/overview-diagnostics

https://azure.github.io/AppService/2018/09/10/Announcing-the-New-Auto-Healing-Experience-in-App-Service-Diagnostics.html

Azure Resource Manager

Azure Resource Manager Overview

Azure Resource Manager, or ARM, is a way to manage resources in Azure. With ARM, we will be able to define our platform, or infrastructure, as code.

One of the benefits of using code to manage platform / infrastructure is we can check in to source control to see different changes over time. It also allow us to reuse part of the code for other deployment.

Azure Resource Manager also have the following benefits:

• Manage resources. Deploy, add, remove resources at ease.
• Resource grouping. Group resource into logical set that makes sense to you, i.e.: environment, location, etc.
• Resource dependencies. Handle dependencies between different resources.
• Repeatable deployments. ARM template can be used to perform same. repeatable deployments.
• Template. Using template and code to define platform / infrastructure. Template is also reusable.

Architecture

With ARM architecture, there are few components. They are:

• Resource providers. These are the functionalities of the resources, such as compute, storage, etc.
• Resource types. The actual resource of Azure services that will be deployed.
• ARM REST APIs. Allow invocation of ARM command thru its APIs.

In general, ARM template has the following structure:

• Schema (required)
• Content Version (required)
• Parameter
• Variables
• Resources
• Output

How it Works

You can also abstract out parameters for a ARM template to its own template parameter file. ARM template parameter must also have schema and version. The version doesn’t have to be the same with the template file.

In ARM template deployment, Azure automatically detect dependencies between resource types and deploy the dependent resource first. For example, if a template has specified to deploy App Service Plan and App Service Web Site, ARM REST API will deploy App Service Plan first, which Web Site depends on.

You can also specify dependency in ARM template as well as deploy resources simultaneously for services that don’t have dependency, i.e. deploying 4 App Service Websites.

Azure Resource Manager will only deploy resources in the template that are not yet exist. When you the specified resource already exists, it will be skipped.

Within ARM template, you can also specify to deploy the application itself, this is useful, for example, when the App Service Web Site deployment will include populating the website with web application from source control.

Reference

https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-authoring-templates

https://github.com/Azure/azure-resource-manager-schemas

Data Warehouse Solutions in Azure

Date Warehousing Solutions at a Glance

With today’s big data requirements where data could be structured, unstructured, batch, stream and come in many other forms and size, traditional data warehouse is not going to cut it.

Typically, there are 4 types of data stage:

• Ingest
• Store
• Processing
• Consuming

Different technology is required at different stage. This also depends heavily on size and form of data and the 4 Vs: Volume, Variety, Velocity, Veracity.

Consideration for the solutions sometime also depends on:

• Ease of management
• Team skill sets
• Language
• Cost
• Specification / requirements
• Integration with existing / others system.

Azure Services

Azure offers many services for data warehouse solutions. Traditionally, data warehouse has been ETL process + relational database storage like SQL Data Warehouse. Today, that may not always be the case.

Some of Azure services for data warehousing:

• Azure HDInsight
  Azure offers various cluster types that comes with HDInsight, fully managed by Microsoft, but still require management from users. Also supports Data Lake Storage. More about HDInsight. HDInsight sits on “Processing” data stage.
• Azure Databricks
  Its support for machine learning, AI, analytics and stream / graph processing makes it a go-to solution for data processing. It’s also fully integrated with Power BI and other source / destination tools. Notebooks in Databricks allows collaboration between data engineers, data scientist and business users. Compare to HDInsight.
• Azure Data Factory
  The “Ingest” part of data stage. Its function is to bring data in and move them around different system. Azure Data Factory supports different pipelines across Azure services to connect the data and even on-premise data. Azure Data Factory can be used to control the flow of data.
• Azure SQL Data Warehouse
  Typically the end destination of data and to be consumed by business users. SQL DW is platform as a service, require less management from users and great for team who already familiar with TSQL and SSMS (SQL Management Studio). You can also scale it dynamically, pause / resume the compute. SQL DW uses internal storage to store data and include the compute component. SQL Data Warehouse sits on “Consuming” stage.
• Database services (RDBMS, Cosmos, etc)
  SQL database, or other relational database system, Cosmos are part of the storage solutions offered in Azure Services. This is typically more expensive than Azure Storage, but also offer other features. Database services are part of “Storage” stage.
• Azure Data Lake Storage
  Build on top of Azure Storage, ADLS offers unlimited storage and file system based on HDFS, allowing optimization for analytics purpose, like Hadoop or HDInsight. ADLS is part of “Storage” stage.
• Azure Data Lake Analytics
  ADLA is a high-level abstraction of HDInsight. Users will not need to worry about scaling and management of the clusters at all, it’s an instant scale per job. However, this also comes with some limitations. ADLA support USQL, a SQL-like language that allows custom user defined function in C#. The tooling is also what developers are already familiar with, Visual Studio.
• Azure Storage
• Azure Analysis Services
• Power BI

Which one to use?

There’s no right or wrong answer. The right solution depends on many others things, technical and non-technical as well as the considerations mentioned above.

Simon Lidberg and Benjamin Wright Jones have a really good presentation around this topic. See the link at reference for their full talk. But, basically, the flowchart to make decision looks like this:

Reference

https://myignite.techcommunity.microsoft.com/sessions/66581