Author

I am Joannes Vermorel, founder at Lokad. I am also an engineer from the Corps des Mines who initially graduated from the ENS.

I have been passionate about computer science, software matters and data mining for almost two decades. (RSS - ATOM)

Meta

Entries in practices (34)

Wednesday
Oct282009

No excuse for not disclosing your roadmap

Software is a fast-paced industry. New technologies soon become obsolete ones, and you need keep your mindset in Fire and Motion mode to move forward. Yet, when something really big emerges, say cloud computing, you end up in a crossroad and you need to make a choice about the future of your business.

This future depends on the 3rd party technology you decide to rely on. This is true for software companies buying software components, but it's also true for brick and mortar companies moving to the next generation ERP.

Trying to push my own little company forward, we can't afford reinventing the wheel. Thus, we are relying on loads of 3rd party tools and technologies; most likey any software company I guess - with the notable exceptions of Microsoft and Google that are nearly self-sustained.

There is nothing wrong in itself depending on other business. Specialization has been a driving force behind business growth for the last two centuries. Yet, in order to take good decisions, I need to be informed about future plans for key technologies that we are adopting.

Plainly put: I need business & technology roadmaps.

A roadmap helps to understand:

  • where the company is heading.
  • if it fits your own business vision.
  • if it matches your upcoming requirements.

Looking around me at companies who disclose their roadmaps, I realized that roadmaps are strong drivers to establish trust and commitment. It shows to your customers and partners that you are committed to move forward with them; not just to leverage status quo.

Yet, it's still sad to see that many companies adopt the Absolute Radio Silence strategy of Apple. It might work in the case of Apple, because they have become expert at leveraging the media buzz around their own plans; but it looks to me a total nonsense for B2B markets where relationships last for years if not decades.

The average lifetime of an installed ERP is 8 years.

Hiding behind the argument we don't want to over-promise  to keep your customers and partners in the dark looks to me a lame excuse. The roadmap represents a best effort attempt at summarizing directions, not an exact schedule. Obviously, it comes with a level of uncertainty. In B2B markets, your customers are smart enough to understand that.

Thus, I have decided to publish a public Lokad roadmap for 2010.

Obviously, one can argue that this roadmap is going to benefit to our competitors. Frankly, I don't think so. If disclosing a 3 pages document is sufficient to put your business in trouble then it means that your intellectual property is really weak in the first place.

The roadmap tells what you are going to do, not the fine grained details to make it work. As usual, ideas are dime a dozen, many investors would even offer them for free, execution is everything.

Thursday
Sep172009

Table Storage or the 100x cost factor

Until very recently, I was a bit puzzled by the Table Storage. I couldn't manage to get a clear understanding how the Table Storage could be a killer option against the Blob Storage.

I get it now: Table Storage can cut your storage costs by 100x.

At outlined by other folks already, I/O costs typically represents more than 10x the storage costs if your objects are weighting less than 6kb (the computation has been done for the Amazon S3 pricing, but the Windows Azure pricing happens to be nearly identical).

Thus, if you happen to have loads of fine grained objects to store in your cloud, say less-than-140-characters tweets for example, you're likely to end-up with an insane I/O bill if you happen to store those fine-grained items in the Blob Storage.

But don't lower your hopes, that's precisely the sort of situations the Table Storage has been designed for, as this service lets you insert/update/delete entities by batches of 100 through Entity Group Transactions.

This fine-grained item orientation is reflected in the limitations that apply to entities:

  • A single entity should not weight more than 1MB.

  • A single group transaction should not weight more than 4MB.

  • A single entity property should not weight more than 64kb.

Situations where loads of small items end-ups being processed - threshold being at 60kb - by your cloud apps are likely to be good candidate for the Table Storage.

We will definitively try to reflect this in our favorite O/C mapper.

Monday
Sep142009

Thinking the Table Storage of Windows Azure

Disclaimer: I am not exactly a Table Storage expert. In this post, I am just trying to sort out my own thoughts about this service offered with Windows Azure. Check my follow-up post.

Soon after the release announcement of the release of our new O/C mapper (object to cloud) named Lokad.Cloud, folks on the Azure Forums raised the question of the Table Storage.

Although it might be surprising, Lokad.Cloud does not provide - yet - any support for Table Storage.

At this point, I feel very uncertain about Table Storage, not in the sense that I do not trust Microsoft to end-up with finely tuned product, but rather at the patterns and practices level.

Basically, the Table Storage is an entity storage that features three special system properties:

  • PartitionKey: a grouping criterion - data having the same PartitionKey being kept close.

  • RowKey: the unique identifier for the entity.

  • Timestamp: the equivalent of Blob Storage ETag.

So far, I got the feeling that many developers feel attracted toward the Table Storage for the wrong reasons. In particular, Table Storage is not a substitute of your old plain SQL tables:

  • No support for transactions.

  • No support for keys (let alone foreign keys).

  • No possible refactoring (properties are frozen at setup).

If you are looking for those features, you're most likely betting on the wrong horse. You should be considering SQL Azure instead.

Then, some might argue that SQL Azure won't scale above 10GB (at least considering the current pricing plans offered by Microsoft). Well, the trick is Table Storage won't scale either, at least not unless you're not very cautious with your queries.

AFAIK, the only indexed column of the Table Storage is the RowKey. Thus, any filtering criterion based on custom entity properties is likely to get abyssal performance as soon your Table Storage get large.

Well, sort of, the most probable scenario is like to to be worse as your queries are just going to timeout after exceeding 60s.

Again, my goal here is not to bash the Table Storage, but it must be understood that the Table Storage is clearly not a magically scalable equivalent of the plain old SQL tables.

Back to Lokad.Cloud, we did not consider adding Table Storage because we did not feel the need either although our forecasting back-end is probably very high in the currently complexity spectrum of the cloud apps.

Indeed, the Blob Storage is surprisingly powerful with very predicable performance too:

  • Storing complex objects is a non-issue with a serializer at hand.

  • A blob name prefix is a very efficient substitute to the PartitionKey.

Basically, it seems to me that any Table Storage operation can be executed with the same performance with the Blob Storage for now. Later on, when the Table Storage will start supporting secondary indexes, this situation is likely to evolve, but meantime I still cannot think a single situation that would definitively support Table Storage over Blob Storage.

Monday
Sep142009

O/C mapper - object to cloud

When we started to port our forecasting technology toward the cloud, we decided to create a new open source project called Lokad.Cloud that would isolate all the pieces of our cloud infrastructure that weren't specific of Lokad.

The project has been initially subtitled Lokad.Cloud - .NET execution framework for Windows Azure, as the primary goal of this project was to provide some cloud equivalent of the plain old Windows Services. We did quickly end-up with QueueServices which happens to be quite handy to design horizontally scalable apps.

But more recently, the project has taken a new orientation, becoming more and more an O/C mapper (object to cloud) inspired by the terminology used by O/R mappers. When it comes to horizontal scaling, a key idea is that data and data processing cannot be considered in isolation anymore.

With classic client-server apps, persistence logic is not supposed to invade your core business logic. Yet, when your business logic happens to become so intensive that it must be distributed, you end-up in a very cloudy situation where data and data processing becomes closely coupled in order to achieve horizontal scalability.

That, being said, close coupling between data and data processing isn't doomed to be an ugly mess. We have found that obsessively object-oriented patterns applied to Blob Storage can made the code both elegant and readable.

Lokad.Cloud is entering its beta stage with the release of the 0.2.x series, check it out.

Sunday
Aug302009

Discovering Twitter

I have been hearing a lot about Twitter for a long time. I am still puzzled a bit by the concept, but apparently a significant percentage of the registrants at Lokad do have a Twitter account. So now, I can start wasting time on Twitter too while pretending it's company work :-)

More seriously, it appears that a couple of competitors, prospects, customers are actually discussing of sales forecasts out there, so it might be worth keeping on eye on that.

Thanks to Rinat, a twitter account for Lokad had been setup a while ago. Since this account is intended to be a company account, we need to handle several users here. Sharing passwords isn't such a great method, so I have decided to give a try to CoTweet.

I have also setup a personal twitter account, although, since I am already lagging behind with my various blogs (including the present one), it's not clear if I will be able to keep up with the frequency that appears to be expected by the Twitter community.