Reading List

How Microservices Architecture Impacted the Culture of Software Development

Most people do microservice because it’s a new buzzword! But, there’s actually reason behind it and it will improve your team dynamic.


Why I love Trunk Based Development (or pushing straight to master)

Mattia made an argument why it makes sense for the whole team to push straight to `master`. The key is having the right people in the team. He has good points, most I agree, some I disagree with. What’s your take?


Async Await C#

Basic yet comprehensive article on `async` `await` in C#. Worth reading for beginners and expert alike if you want to brush up on `async` `await`.


You don’t need ordered delivery

Particular.net blog posts in general are really good, high quality. This _particular_ post, however, gives a different perspective of how messaging queues are delivered. Often time we think it needs to be ordered. Well, maybe we should think again.


Messaging Practices

I never thought there’s such things as good practices in messaging queue. After reading this, I think it makes sense to have one. When you have to deliver large number of events, good practices make big differences.


Advertisements

Another Free eBook

Microservice database migration guide

Part of breaking up your monolith application to microservices, you will also need to break out your data sets, from relational database to more distributed model. This book provide some guidances around this issue.

Almost-Weekend Reading List

What Is a Pure Function?

That’s exactly what the title says, this article explains what a pure function is. There are 2 things that define a pure function. One of them is the function always returns the same value for the same inputs. I’ll let you read the other from the blog.


The Cloud-native Azure Application Architect Map

If you are confused with Azure services offering because there are too many of them, you’re not alone. The way-too-many-services always lead to analysis paralysis of what to use for your project. This architecture map should help you decide.


A Step-by-Step Guide to Continuous Deployment on Kubernetes

`Dockerfile`, `docker-compose.yaml` file, Kubernete manifest file… ever confused on how to do CI/CD on Kubernetes exactly? Tomas will walk you through his experience.


Research Paper Week: Automatic Indexing in Azure SQL DB

This one is a heavy read, could probably be your help-me-sleep reading list. The research is about how Azure automatically perform indexing in Azure SQL database. Today, Azure already has a lot of automatic recommendation on how to increase your Azure SQL performance. This one takes a step further but automatically applying the recommendation and if it’s not working as intended, roll it back. Pretty cool idea!


Avoid multithreading traps with Roslyn: Lock object selection

What NOT to lock when multithreading? Well, here’s the list.


This Week Reading List

Want to learn any programming language? — Write These 3 Simple Apps!

https://medium.com/@samuel.fare/want-to-learn-any-programming-language-write-these-3-simple-apps-5af8cd119921

Very good read if you want to learn new programming langugage. Sam Fare challenges you to build 3 simple apps with a new programming langauge: Hello World, Anagram and Pizza app. I’ll throw in one more as a bonus: a Todo app.

 


How to Become a Better Software Developer by Digging & Climbing

https://dmitripavlutin.com/become-better-software-developer-digging-climbing/

Even if you are an experienced developer, it’s still a very good read. Dmitri shares what he did to become a better developer (and we always have room to become a better developer). It’s a conceptual idea, which is why it’s a very good read, with some practical examples.

 


Event-driven analytics with Azure Data Lake Storage Gen2

https://azure.microsoft.com/en-us/blog/event-driven-analytics-with-azure-data-lake-storage-gen2/

If you are dealing with big data, the ingestion, processing, storage, consumption and ETL stuff, add this to your reading list. It outlines a way to use event-driven technique using Azure services to build data for analytics.

 


Azure solution architectures

https://azure.microsoft.com/en-us/solutions/architecture/

One of my favorites. This is more of a reference than a reading. Microsoft docs provides cloud architecture solutions (based on Azure services) for different scenarios and needs. Definitely check it out when you need to design and build a new system.

 


The Azure Infrastructure Architect Map

https://techcommunity.microsoft.com/t5/Azure-Developer-Community-Blog/The-Azure-Infrastructure-Architect-Map/ba-p/766268

Yet another good resource for a reference. This guides you through way-too-many Azure services based on your needs, sorta help you to decide what Azure service to use for a specific scenario. There’s also Solution Architect map and Security Architect map. Links are in the article.

 


Pipeline Pattern Implementations in C# .NET – Part 1

https://michaelscodingspot.com/pipeline-pattern-implementations-csharp/

Part 1 of 2 where Michael explains how to implement pipeline pattern in C#. This is not about `System.IO.Pipelines` library, but rather,
Part 2 – https://michaelscodingspot.com/pipeline-pattern-tpl-dataflow/

Transient Fault

Transient Fault

A fault, disconnection or outage, that’s short-term in nature and come back up quickly. For example: loss of network connectivity, timeout, or service unavailable / busy.

Reference
Microsoft Docs

Data Warehouse Solutions in Azure

Date Warehousing Solutions at a Glance

With today’s big data requirements where data could be structured, unstructured, batch, stream and come in many other forms and size, traditional data warehouse is not going to cut it.

Typically, there are 4 types of data stage:

  • Ingest
  • Store
  • Processing
  • Consuming

Different technology is required at different stage. This also depends heavily on size and form of data and the 4 Vs: Volume, Variety, Velocity, Veracity.

Consideration for the solutions sometime also depends on:

  • Ease of management
  • Team skill sets
  • Language
  • Cost
  • Specification / requirements
  • Integration with existing / others system.

Azure Services

Azure offers many services for data warehouse solutions. Traditionally, data warehouse has been ETL process + relational database storage like SQL Data Warehouse. Today, that may not always be the case.

Some of Azure services for data warehousing:

  • Azure HDInsight
    Azure offers various cluster types that comes with HDInsight, fully managed by Microsoft, but still require management from users. Also supports Data Lake Storage. More about HDInsight. HDInsight sits on “Processing” data stage.
  • Azure Databricks
    Its support for machine learning, AI, analytics and stream / graph processing makes it a go-to solution for data processing. It’s also fully integrated with Power BI and other source / destination tools. Notebooks in Databricks allows collaboration between data engineers, data scientist and business users. Compare to HDInsight.
  • Azure Data Factory
    The “Ingest” part of data stage. Its function is to bring data in and move them around different system. Azure Data Factory supports different pipelines across Azure services to connect the data and even on-premise data. Azure Data Factory can be used to control the flow of data.
  • Azure SQL Data Warehouse
    Typically the end destination of data and to be consumed by business users. SQL DW is platform as a service, require less management from users and great for team who already familiar with TSQL and SSMS (SQL Management Studio). You can also scale it dynamically, pause / resume the compute. SQL DW uses internal storage to store data and include the compute component. SQL Data Warehouse sits on “Consuming” stage.
  • Database services (RDBMS, Cosmos, etc)
    SQL database, or other relational database system, Cosmos are part of the storage solutions offered in Azure Services. This is typically more expensive than Azure Storage, but also offer other features. Database services are part of “Storage” stage.
  • Azure Data Lake Storage
    Build on top of Azure Storage, ADLS offers unlimited storage and file system based on HDFS, allowing optimization for analytics purpose, like Hadoop or HDInsight. ADLS is part of “Storage” stage.
  • Azure Data Lake Analytics
    ADLA is a high-level abstraction of HDInsight. Users will not need to worry about scaling and management of the clusters at all, it’s an instant scale per job. However, this also comes with some limitations. ADLA support USQL, a SQL-like language that allows custom user defined function in C#. The tooling is also what developers are already familiar with, Visual Studio.
  • Azure Storage
  • Azure Analysis Services
  • Power BI

Which one to use?

There’s no right or wrong answer. The right solution depends on many others things, technical and non-technical as well as the considerations mentioned above.

Simon Lidberg and Benjamin Wright Jones have a really good presentation around this topic. See the link at reference for their full talk. But, basically, the flowchart to make decision looks like this:

data-warehouse-solutions-in-azure

Reference

https://myignite.techcommunity.microsoft.com/sessions/66581

What is Event Sourcing?

Event Sourcing

Every application has a state, representing current snapshot at the moment. Event sourcing is the idea of persisting all of these states as it changes from one to another.

Imagine a workflow for an invoice goes something like: invoice is created which set the status to pending. Pending invoice then gets approved which sets it approved. Approved invoice can be denied or collected. When invoice is denied, it goes back to pending, but when it’s collected, it becomes closed. The state of an invoice is where it’s at a particular time. However, when event sourcing pattern is implemented, we will be able to see a historical timeline of an invoice, from when it’s created to the current state.

What this allows us to do is, we are able to:

  • Complete rebuild: getting rid of the current state and use events to rebuild the invoice to its current state.
  • Temporal query: we can tell what state an invoice is at particular point in time.
  • Event replay: when the past event is incorrect, we can make the change and compute it from the beginning, thus, replaying the whole series of events.

Some applications of event sourcing are:

  • Source control. Every source control uses time series where commits made change the code. By storing events for every commit, we are able to state of the code at particular point in time.
  • Redo / undo. Applications that utilize undo / redo, such as word processor, will benefit from event sourcing pattern.
  • Document history. Word processor application will also benefit from event sourcing by storing event every time changes are made.

I hope this simple explanation of event sourcing helps you to understand what it is and how we can use it. There’s definitely a lot more to it that I don’t cover here, but there are lots of resources online you can learn from.

Reference

https://martinfowler.com/eaaDev/EventSourcing.html