Introducing Eventvisor v0.1

Every organization wants to be data informed, and often data-driven. But very few can confidently tell what data they track, in what shapes, from where, and sent to what destinations.

To tackle this, I am introducing a new Open Source tool called Eventvisor.

For the impatient folks: visit the quick start guide already.

This probably kickstarts the "visor" family of tools after Featurevisor, which Eventvisor is now the newest member of.

Product Engineering challenges#

It will help me introduce the solution to you better if we begin by understanding the challenges first.

If you are working in a medium to large tech organization, you might have faced issues like:

Knowing what logs & analytics events are tracked
In what shapes they are tracked
From where they are tracked (web, mobile, microfrontends, backend, etc)
What destinations they are sent to (GA4, Segment, Datadog, or custom backends)
Who's keeping everyone in sync in realtime as they continuously evolve overlapping cross-team boundaries

Sounds familiar? Then you are not alone.

Ingestion vs. Governance vs. Control#

No shortage of services out there to collect, store, and query data in a highly efficient manner. But we don't have that many that can establish a workflow that:

Brings all parties of an organization (Product, Engineering, Data, Marketing) together since early product development phase to define the data to be tracked
Establishing a single source of truth, ensuring everyone is on the same page as things evolve
Agnostic of whatever logs and/or analytics services you are using
Keeping full control of the data generation, transformation, and routing directly from inside the apps even after product delivery
Instantly affecting all applications without new deployments via remote configuration

This is why Eventvisor was born.

What is Eventvisor?#

To highly summarize, it's a:

Centralized governance tool of all your events and attributes
Runtime validator of tracked data
Transformer of tracked data on the fly based on remote configuration
With advanced filtering & sampling (logs aren't cheap to ingest)
Conditionally routing to destinations (GA4, Segment, Datadog, or custom backends)
Handler of side-effects (broad use case still being explored)

All of it managed via a single Git repository in a language-agnostic way, which remotely controls your tracked events directly from inside your independently deployed applications and services.

Who is it for?#

Data is one of those layers that touches all parts of an organization. Depending on who you are, the benefits are:

Product Managers having clarity and influence over analytics events
Engineering Teams made accountable if they are sending data in wrong shapes
Engineering Managers having clear ownership and boundaries
Principal/Staff Engineers debugging cross-team apps without new deployments
Compliance & Security teams having visibility into all the data being tracked
Marketing teams brought into the same workflow as others (think marketing pixels)
VPs positioning strategically to migrate to/from vendors with ease
CTOs pushing for ingestion cost savings via advanced sampling & filtering
Execs having accurate reports they can rely on from Data teams
Data & Analytics teams recovering their receding hairlines (!)

How does it work?#

If you are an existing user of Featurevisor, the process will feel very similar. The difference is instead of feature flags, you are managing events now which can be either your regular application logs and/or analytics events.

The workflow can be expressed as a three-step process:

Teams send pull requests making changes to event schemas and routing configurations
Upon reviews and merging, CI/CD pipeline builds datafiles (static JSON files) and uploads them to a CDN or your own server
Applications consume the datafiles at runtime using SDKs and track the events

Based on remote configuration, SDK automatically handles filtering, sampling, transforming and routing for you.

What to expect in future?#

It's only the initial v0.1 release, where my primary focus was just to express the idea fully as a working solution. I will tinker with it for a while before I consider it stable enough to tag a v1.0 release.

Things WILL break in v0.x releases, and there will definitely be bugs. I cannot recommend you to use it in production yet unless you are willing to take that risk. But if you are an early adopter type and into GitOps and IaC principles, you will find it exciting to play with.

You can always keep an eye on the roadmap.

Where can I use it?#

At the moment, it works in browser based applications and also in backend applications using Node.js via the same universal JavaScript SDK. More SDKs in other programming languages will follow in future as the project matures.

It has been designed to be programming language agnostic, and is meant to be used anywhere (web apps, native mobile apps, backend services, embedded systems, etc) as long as there's an SDK for it.

Participate!#

You are most welcome to read the docs, try it out, and participate in the Open Source development to help shape it into something great.

I don't expect it to tick with everyone at first glance. And positioning a new bespoke tool like this with broad use cases in early days is always challenging.

But based on personal experience of being in highly chaotic fast moving organizations in cross-team roles, I believe it will find its footing over time. Culture shifts take time, and this workflow is not going to be an overnight yes for everyone.

To help raise awareness, kindly requesting you to:

Share it with your colleagues (especially those in Product, Engineering, Data, and Marketing teams)
Post about it on your social media
Star it on GitHub so more developers discover it
Reach out to me personally for any feedback or questions

You have the power to influence the direction and adoption of this project. Wherever you are.

Cheers!