Transient Fault

Transient Fault

A fault, disconnection or outage, that’s short-term in nature and come back up quickly. For example: loss of network connectivity, timeout, or service unavailable / busy.

Reference
Microsoft Docs

Advertisements

Memoization, huh?

Memoization

To put it simply, it’s caching. In a fancy term.

Memoization is usually being used in function. In other words, memoization is caching in the scope of Function. The purpose of memoization is to avoid doing computation that’s needed to be done repeatedly. This essentially to save time and resources.

The way functions achieve memoization is by caching the result and returning the result if it has been computed previously.

Couple use cases of memoization

  • Pure functions that require intensive computation and called repeatedly.
  • Pure and recursive Function.

See Fibonacci example in the link reference.

Reference

https://medium.com/@chialunwu/wtf-is-memoization-a2979594fb2a

NgRx Entity

NgRx Entity

NgRx Entity is a library to handle entity within NgRx framework. The entity in the context is defined as the application’s domain objects, like User, Employee, etc.

The purposes of NgRx Entity are basically:

  • Reduce boilerplate code.
  • Search and sort entities quickly.

The first thing we need to use NgRx Entity is to create an entity adapter, like so:

const adapter = createEntityAdapter();

NgRx Entity also defines `EntityState<>` interface. The interface is also extensible should the additional properties in the application state are needed,

The shape of EntityState is something like this:

interface EntityState {
  ids: string[];
  entities: { [id: string]: V };
}

What this allows us to do is:

  • Find entity quickly using `entities` dictionary.
  • Maintain order of the list, good for use in sorting.

Some boilerplate codes that are reduced when using NgRx Entity:

  • No need to specify properties of the state interface.
  • Add, remove, update of entities in the state are handled by entity adapter.
  • Entity adapter has most commonly used selectors.

Reference

https://medium.com/ngrx/introducing-ngrx-entity-598176456e15

NgRx State Selectors

NgRx State Selectors

As our application grows, the data structure that represents the application states could get more complex. State data structure in ngRx is tree-like and in the early development, the structure may only have few properties. As the application grows, the structure could have deep nested objects.

When states have deep nested objects, we will have to write more code to select the slice of state. Something like:

// Complex data structure
let state = {
    products: {
        pizza: {
            name: "Pepperoni",
            amount: 4
        },
        topping: {
            name: "mushroom",
            amount: 3
        }
    }
};

// Selecting pizza state
let pizza = state.products.pizza;

Selecting pizza state requires dot notation and it’s also prone to error where products is undefined, that line of code will blow up.

Another issue with this is that we will have to write the same line of code to get pizza state every where we need the state.

This is where state selectors in ngRx comes in. It allows us to define selectors for our states and use it everywhere in our application.

There are mainly 2 state selectors, createFeatureSelector and createSelector. The first one mainly deals with state at the root level while the second one can be used to select a specific object in the state (for example, pizza or topping).

Reference

https://toddmotto.com/ngrx-store-understanding-state-selectors

What is Hadoop?

Hadoop and Distributed System

If you were a basketball player, all you need to is dribble and shooting. With those 2 skills, you can play basketball by yourself really well. But, if you were to play in a team, you are going to be suck. To have a successful basketball team, you will have to team up with other players. But then, you will also need to learn a new skill, passing. And a coach to coordinate everyone.

This is true with monolithic vs distributed system. In monolithic system, all you have is one giant supercomputer with large amount of memory, storage and compute power. In distributed system, you don’t have to have supercomputer, but you will have multiple, maybe less powerful, computers. Just like each player in a basketball team has to learn passing, each computer has to that talk to each other now and they will also have a software to coordinate them.

This is what Hadoop is for. It’s a system to coordinate and orchestrate a cluster of computers, called node, in a distributed system. Hadoop is like the coach for the basketball team.

Hadoop does a lot of heavy lifting, such as:

  • partition data
  • coordinate compute tasks
  • fault tolerance
  • allocate capacity to process / jobs, etc.
  • monitoring
  • security
  • API

The logical components of Hadoop, HDFS, MapReduce and YARN, are what Hadoop uses to do the heavy lifting. These components are essential the storage (HDFS), programming model (MapReduce) and resource manager (YARN).

In big data processing, some crucial requirements are to:

  • store
  • process and
  • scale

These reqs are to allow store, process and analyze data efficiently and in a timely manner. Hadoop is a perfect solution for big data processing.

And because Hadoop is handling most everything in cluster management for developers, we can focus on actually doing the work, building model, processing data, reporting, analyzing, etc. The details of cluster management is abstracted away.

What’s really cool about Hadoop is also its ecosystem. A lot of tools and technology have been created on top of Hadoop. Some of the popular ones are: Hive, HBase, Spark, Pig, Flume/Sqoop, Storm, Oozie, and many more. But, that’s for another day.

NgRx Store Explained

NgRx Store

ngRx Store is a library for Angular 2+, its function is almost like event bus where the store is a central storage for all events and whenever an event is added, the store emits an action. Whoever subscribe to the store‘s action will then get notified. Very similar to pub-sub pattern.

The purpose of ngRx Store is to simplify the state management, that sometime can be simple at first but, as the application grows, becomes unimaginably complicated.

Some example usage of ngRx Store that I can think right now are:

  • Forward execution. When an Angular application sends an API call to update, let’s say, employee data, it doesn’t need to call API anymore to query what the current employee data is. However, things get complicated when there’s a certain logic needs to be applied when updating employee data. For example, when the hire date is set to the future, the employee status will also need to be changed. ngRx Store can be used to manage the employee state so that it remains consistent at every part of the application.
  • Reduce extraneous props. Extraneous prop is when we have a deep component tree and have to pass object around to the leaf component. ngRx Store eliminates the need of passing the object around and just have the leaf component subscribe to the `store` to receive the action. A concrete example of this is notification counter. The counter is normally in its own component and every other event can increase the count. By using ngRx Store, we can avoid other components passing the count to the counter component.
  • Sort of the similar to extraneous props, we can have 2, or more, components that use the same data but display it differently. With ngRx Store, we can manage the state of the data in one place and all other components that use the data just subscribe to it.
  • Step-by-step, wizard-y workflow. Think of this like your tax filing software where it asks you questions and, depend on your answer, take you to different workflow. In Angular, each step the users take can use a separate component and ngRx Store can be used to maintain the state of the whole workflow process.

The problem that ngRx Store trying to solve is really similar to challenges that a large scale enterprise application faces, it just does it in a smaller scale, UI front-end in this case.

There’s definitely a lot more to this concept than what I cover here, but hopefully this gives you the gist of how it works at the high level. There are lots of resources online that cover this very topic.

What is Event Sourcing?

Event Sourcing

Every application has a state, representing current snapshot at the moment. Event sourcing is the idea of persisting all of these states as it changes from one to another.

Imagine a workflow for an invoice goes something like: invoice is created which set the status to pending. Pending invoice then gets approved which sets it approved. Approved invoice can be denied or collected. When invoice is denied, it goes back to pending, but when it’s collected, it becomes closed. The state of an invoice is where it’s at a particular time. However, when event sourcing pattern is implemented, we will be able to see a historical timeline of an invoice, from when it’s created to the current state.

What this allows us to do is, we are able to:

  • Complete rebuild: getting rid of the current state and use events to rebuild the invoice to its current state.
  • Temporal query: we can tell what state an invoice is at particular point in time.
  • Event replay: when the past event is incorrect, we can make the change and compute it from the beginning, thus, replaying the whole series of events.

Some applications of event sourcing are:

  • Source control. Every source control uses time series where commits made change the code. By storing events for every commit, we are able to state of the code at particular point in time.
  • Redo / undo. Applications that utilize undo / redo, such as word processor, will benefit from event sourcing pattern.
  • Document history. Word processor application will also benefit from event sourcing by storing event every time changes are made.

I hope this simple explanation of event sourcing helps you to understand what it is and how we can use it. There’s definitely a lot more to it that I don’t cover here, but there are lots of resources online you can learn from.

Reference

https://martinfowler.com/eaaDev/EventSourcing.html