Talking about BigData

Dear Reader,

It’s been a few months that we can read a lot about no-sql database, columns database, … and most posts dont make the difference between the storage strategy and the language used to exchange with the database engine.

Talking about Business Intelligence, we tried to find an interesting subject to test a BigDatabase engine (Casandra, part of the Hadoop Project) with the Vanilla platform (which is, by nature, sql-oriented)

Purpose is to understand what are the limitation of no-sql language when using Reporting or Etl package, how we can connect Vanilla to no-sql database and what are the performance when compare with a sql-database (PostGreSql), so that we can help our customers and partners when talking about “fashion” BigData projects.

One of the most interesting feature of Vanilla 3.4 is the Maps module, embeded into Dashboard, Report, Kpi and Cubes. So, it was a natural situation to test Vanilla and Cassandra inside a Geolocalisation Projects.

Honest , beginning of project was not trivial, as we had to turn our mind from “sql requests” to adopt another strategy (java class to manipulate data), and we even maintain a PostGreSql database structure in parallel to validate storage content and requests results. Finally, first result is impressive, as Cassandra allow us to mix objects (such as images, maps) and data (such as kpi, location properties) in a common space and provide a single entry for “data” manipulation. Still prototyping the projects, it was very pleasant to be able to add new objects types without running heavy database structure modifications.

In addition, performances are great AND stable : no matter if we have 1 million of data or 20 millions inside Cassandra, result of geo-requests are stable (some milliseconds to get complex calculation that return a set of geodata properties).

We will soon make our development available for the mass, just keep in touch,

Have Fun !

Patrick

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: