Dot Collector

Scalable data storage system based on NoSQL, NodeJS and React

Released: Dot Collector

Dot Collector DBMS system

Today I was thinking about the most basic building block of most internet-powered software(s) in the modern world. Deep down in my admiration of how internet and software have changed and continues to change the world, I settled on one thing which forms the backbone. And that basic block is plain and dumb yet immensely important part of all software - meet the DBMS. It stands for Database Management System and provides software with the power to remember things across computers, phones and boundaries!

Technically speaking, there are a ton of things that can be done with DBMS, but today I decided to test my sharpness by building a Dot Collector over a week. Dots, conceptually a text as of this demo, can be sent by a web client and a central server can pull the dots for processing or show insights to end-users. This project aims at creating a simple bootstrap for reading/Write queries based on a Mongo powered Cloud NoSQL Backend, React (without JSX), Express, NodeJS.

This is an amazingly simple yet massively scalable system! That opens opportunities for the following types of applications to name a few:

  1. Acknowledging digital-ads fueled transactions on browsers
  2. IoT based data collection to analyze device health
  3. In an office setting, to analyze the performance of employees in real-time.
  4. Conduct live short-time surveys
  5. Endless more opportunities!

Before I dig deeper let me explain the choice of my tools:

  1. React Website - Extensible and powerful way to build websites. In short, it keeps the code extensible to use cases using limited software engineering resources!
  2. NoSQL Database - I used a free version of a database - MongoDB Atlas. It can be easily paid for and the expanded with more support and/or customizable to needs.
  3. NodeJS Backend - I used Express which can be easily deployed in an AWS backend and automatically scale to millions, then billions of users without much effort.

Few goals I set for this task:

  • Ability to send Dots to a backend and show them on any UI
  • Ability to query Dots and filter them depending on certain conditions, in this case, time

Demo Video

Before I jump over to implementation details, enjoy my test setup’s recording:

Previews

  1. Ability to send Dots to a backend and show them on any UI Dots received on MongoDB backend

  2. Ability to query Dots and filter them depending on certain conditions, in this case, time Filtering and querying dots

Sources

I have made the source openly available and sharable on DotCollector GitHub repo.

My Take

If you can use such a system to reduce the time and/or effort and/or cost of doing business, feel free to consult someone ( or me :-P ). All you need is to estimate how much you could save in terms of $$ and then create a software building, deployment and maintenance strategy around it.

Credits

  1. Mongo API doc for NodeJS

  2. Free version of Mongo Atlas

Dialogue & Discussion