Author

I am Joannes Vermorel, founder at Lokad. I am also an engineer from the Corps des Mines who initially graduated from the ENS.

I have been passionate about computer science, software matters and data mining for almost two decades. (RSS - ATOM)

Meta

Friday
Mar212008

## Custom error page in ASP.NET when database connectivity is lost.

A particularly annoying, yet frequent, issue for your ASP.NET is the loss of database connectivity. Indeed, if your database is hosted on a separate machine (as it is generally advised for performance), then your web application is subject to database downtime.

Database downtimes have several particularities

• It generates internal server errors.

• It's not the type of error that can be fixed by be the developer.

• The problem tends to get solved by itself (think: reboot of the database server)

• Errors don't get logged (well, assuming that you are logging errors in the database).

Thus, for my own ASP.NET application, I want to display an error page that invites people to try again at a later time whenever a database downtime occurs. In comparison, if a "real" error is encountered, the error gets logged and the customer is invited to contact the support (although, support is also monitoring server side error logs on its own).

Although, ASP.NET makes it very easy to add a generic error page for all internal errors through the <customErrors/> section in the web.config, it's not that simple to have a dedicated page that is selectively displayed for database connectivity issue. Thus, I have decided to come up with my own HttpModule that catches database connectivity error and performs a custom redirect.

using System;using System.Collections.Generic;using System.Data.SqlClient;using System.Text;using System.Web;namespace Lokad.Web.Modules{    public class DatabaseConnectionErrorModule : IHttpModule    {        public void Init(HttpApplication context)        {            context.Error += new EventHandler(OnError);        }        public void Dispose() { }        protected virtual void OnError(object sender, EventArgs args)        {            HttpApplication application = (HttpApplication) sender;                        // The SQL exception might have been wrapped into other exceptions.            Exception exception = application.Server.GetLastError();            while (exception != null && exception as SqlException == null)            {                exception = exception.InnerException;            }            if (exception as SqlException != null)            {                try                {                    // HACK: no SqlConnection.TryOpen() method.                    // Relying on error numbers seems risky because there are                    // different numbers that can reflect a connectivity problem.                    using (SqlConnection connection = new SqlConnection("foobar"))                    {                        connection.Open();                    }                }                catch (SqlException)                {                    application.Response.Redirect("~/MyError.aspx", true);                }            }        }    }

Ps: I have been suggested to use the Global.asax hook. I have discarded this approach, because no matter how I was looking at the problem, Global.asax just looks legacy to me (zero modularity, zero testability, etc...).

Sunday
Oct212007

## Velib's from a software engineer viewpoint

The Velib's are becoming insanely popular in Paris because of the strikes (strikes in public transportations is a national sport in France, a bit like baseball is in the US). Thus, I have been taking my first Velib ride yesterday, a few months after their initial launch.

The Velib both the name of a public bike renting system in Paris but also the name of the bike itself. There are now 10.000 Velib's in Paris (the figure will increase up to 20.000 at the beginning of 2008). The key idea is that take a Velib from any Velib station and put it back into any other Velib station (it does not have to be the same station).

Velib's are a bit bulky (17kg), but in overall they are quite nicely designed.

In my opinion, there are two main weaknesses in the current Velib's system

• the Velib traffic regulation

• the software interface of the Velib renting system

The idea of taking/letting the Velib wherever you want is quite nice. Yet, in practice, there are very important daily migrations of Velib's within Paris. Basically, in the morning you observe that all the Velib are taken (by the people) toward the inner center of Paris. Then, at the end of the day, there is the opposite flux, and the Velib's get massively migrated to the outer part of Paris again.

For the average user, strong migrations means that that you are having hard-time to actually find a free Velib in your starting area; but also that you are having hard-time again to find a free slot to park your Velib in your arrival. In order to overcome such a situation, the deal with JCDecaux (the company in charge of the Velib system) include some Velib traffic regulation to organize counter-migration of the Velib's (through special trucks). Yet, I suspect that the initial deal was massively under-estimating the strength in the migrations in Paris.

At this point, I can hope for two things: Paris re-negotiates with JCDecaux another agreement to increase the Velib traffic regulation; and/or JCDecaux upgrades its traffic regular software to anticipate the migration and respond more pro-actively to them.

Also, the software interface to rent your Velib is a pain. The first mistake comes from the fact that there is not one but two display devices: a big color digital screen that displays the main interface and below, a small alphanumeric display that displays some informations related to the credit card processing. Together those two display devices are a real pain, because you are never sure were to look at while waiting for the next instruction.

Then, the total number of keys that have to pressed iteratively on the numeric pad to do a rent-for-the-day operation is completely insane. I have quickly lost the track, but it must be around 50 key operations or so; which takes 10 mins no matter how much familiarity you have with the system (I was yesterday assisted by somehow who did dictate to me the instructions in order to speed-up the process).

Not sure that the Velib UI would have succeeded against any hallway testing; yet, during the strikes you got the perfect excuse to be late anyway. You can perfectly afford some 20min struggle to rent your Velib.

Monday
Apr232007

## Minimal back-office for your eBusiness

Basically, there are 3 unavoidable elements for any eBusiness back-office

• Account & User deletion
• Error logs reporting

Account & User deletion: IT'S THE LAW, well, at least in Europe, but I suspect that many other countries provide similar laws, rules or guidelines. Intuitively, if a user who has just created an account asks for a deletion of his account (including all the user-related data), complying is a legal requirement. For example, I have been asking for months the people of StrikeIron to delete my account (mostly because they keep pinging deprecated Web Services URLs of Lokad that do not exist anymore); but they never complied so far. I am not going to sue them for that, but I would suspect that they simply never implemented the delete account feature in their back-office.

Error logs reporting: Running your application on your own web server has a huge advantage: collecting the exceptions that are thrown on your web server is mostly trivial. Many tools (such as ELMAH) can be deployed within hours to provide efficient error logs reports. Exceptions are not the only kind of bugs that your visitors can encounter, yet, it's so easy to keep track of them that you must not disregard such a cheap way of detecting bugs.

Business oriented dashboard: It exists many generic web analytics tools ranging from raw webserver logs to sophisticated user tracking system. Yet, when you design a web application, those elements are most probably going to be only a rough estimate of what your users are actually doing. Indeed, it's often much more interesting to design simple (yet highly specific) indicators that reflects some key aspects of the usage of your application. Those indicators could be critical to quickly estimate impact on a web marketing campaign.

Monday
Feb122007

## What's wrong with PAD files

There are quite a lot of things that are just simply wrong in the IT industry nowadays, I have already discussed the case of the Google Adwords, let's move to the subject of PAD files.

PAD stands for Portable Application Description, it's an XML format designed by the shareware industry to facilitate the submission of software products to software directories. The idea is pretty simple and pretty nice. As a software manufacturer, you create a PAD file for each one of your products; then you publish this PAD file directly on your website. For example, when Lokad did release its first open source product named Lokad Sales Forecasting for ASP.Net, I have created (and submitted) a PAD file for this application.

### Submitting through cut-and-paste

Before PAD, you were just manually submitting your product description to every software directory of the web. Now with PAD, you're still submitting your product description to every single software directory on the web; but the submit operation is now (usually) restricted to a single operation: cut-and-pasting the URL of your PAD file. The support for PAD among the shareware/freeware distributor industry is really impressive. I would guess that over 95% of the freeware / shareware industry now supports PAD files.

But the only thing really impressive about PAD is its absolute lack of design.

### When the XML design makes no sense

As a software producer, you don't need to manually generate your PAD file, you got a free editor for that. Yet, I don't think I have ever seen an XML schema that is so massively adopted while being so poorly designed.

They are so many issues with PAD that it's actually hard to even summarize the topic. Following a quasi-random order, the main PAD issues would be

• You need to specify the size of your software in bytes, kilo-bytes AND mega-bytes (File_Size_Byte, File_Size_K AND File_Size_MB). Don't you think that this information is somehow redundant?

• The requirement description is restricted to OS version. What about required 3rd party software like DirectX or .Net?

• Open source (or source availability) is not part of the fields; furthermore it is not really possible to use PAD to describe open source software.

• Software components / library cannot be described. It does not really "fit" the PAD template.

• The software category field make no sense; a tag based system (think swik.net) would have been some much simpler AND so much more efficient.

• No (X)HTML support for your description fields. Your software description ends up plain text. As a result, big lump of texts (like the 2000 characters description) are almost totally unreadable.

• No consistence in XML tags naming
• some tags are UPPER_CASED

• some tags are Camel_Cased

• some tags are explicit Program_System_Requirements

• some tags are abbreviated Char_Desc_45
• The localization makes no sense (localizing a software ~ translating the software + adapting to the regional settings)
• only the Description tag can be localized.

• not possible to localize the other fields like contact or support emails, like the screenshot.

• no encoding specified upfront in the XML file.
• The company address fields makes no sense for non-US locations (State_Province only apply to USA/Canada).

• Why hard-coding the cost in US Dollars (Program_Cost_Dollars)? There are a lot of currency out-there. Then why not being able to support a price list? (list of currency/value).

• The Download URL section is just moronic. You can specify up to 4 download URLs (why 4?) and the each URL gets its own special tag with no naming consistency

• why not simply providing a URL lists?
• The screenshot section is restricted to a single image URL. Why not a list?

• No extensive mechanism for the affiliate programs (because the list of affiliate programs is hard-coded).

Note that all those suggestions would have made PAD easier to document, to produce and to consume.

Then some high-level criticisms could also be made

• no mechanism to link to other PAD files (especially useful to support software versions).

• no persistent mechanism using Global Identifiers (especially useful to detect replicated PADs).

• no mechanism to retrieve the PAD files by simply crawling the web (think XML feed links in HTML pages).

### Summary: PAD has been designed by junior high kids (probably)

• XML design: tag naming is random, data structures like lists are ignored.

• Web design text readability is not a concern, screenshots are unimportant.

• The world outside of the USA: utterly naive attempt to support internationalization.

• Software industry: operating systems are the only components worth to be mentioned.

Still, I think that a Portable Application Description is based on a good idea, but it would really need to be re-designed from scratch.

Sunday
Jan072007

First, let me clarify what I mean by “adversarial behavior” here. Put very simply, it’s a situation where an actor (can be a company, an individual) does not play by the rules (can be the laws, can be ethics, can be traditions) and attempts to exploit system weaknesses for its own profit with no overall benefits for the society as a whole. I would oppose this behavior to a “constructive behavior” where an actor attempts to make profits by building wealth with a positive overall benefit for the society.

Typical adversarial behaviors includes stealing, cheating, lying. To avoid those issues, most modern societies have laws to fight those behaviors. On the web, the situation gets more complicated. Not because law does not apply to the internet (which is totally untrue) but because laws gets really complicated to enforces. For example, if you’re caught stealing $10000 to your neighbor, you’re in trouble. But what happen if you’re caught stealing$0.001 to 1,000,000 people from 30 countries over the worlds? Well most probably nothing is going to happen, because nobody is going to complain for 0.1cent.

Such situation is precisely what is causing spam. There are a lot of spammers; the impact of each spammer is low on the global scale, but all together the spammers are a real pain for everybody. E-mail as it has been designed several decades ago is super weak against adversarial behaviors. Yet, also the problem is very well known, it’s still quite easy to mess up, see the Coding Horror story about trackbacks .

It has been quite a lot introduction, but the point I will defend here is that AdWords are almost as broken as the e-mail system to fight adversarial behaviors. Let’s review what I get for “open source ecommerce”, I have pasted below the result obtained at 2007-01-07

 Sponsored Links Ecommerce Open Source Open an ecommerce website or an eshop. Try free! storefront.homestead.com Open Source Ecommerce 1000’s of Great Programmers Bidding Fast & Simple Project Outsourcing. www.GetACoder.com Open Source Ecommerce Find the Top Links on Open Source Ecommerce www.linx-best.com Open Source Ecommerce Open Source Ecommerce? Get More Helpful Free Info Here. yoursources.info

Ok, what’s wrong with this listing? Actually the first result is just a blatant lie, HomeStead.com is absolutely not featuring any open source solution (just check their web site if you do not believe me). Well, try to do that on a National TV show, and I am pretty sure that you get a class action the next morning. The second ads, for GetACoder.com, is also a lie but I would say that it’s lesser lie since it can be argued that nothing prevents you to find freelancers to customize an open source ecommerce application. Google can’t be accused of willingly carrying lies, certainly they do not. Yet, they can discard their responsibility. Both Homestead.com and GetACoder.com are lying here, but their behavior is not totally adversarial, they do both provide added-value solutions for their customers.

Just for fun, let’s see what we get for “make money fast” on Google (results obtained 2007-01-05). Note that the results are split in two sections, the first section being displayed before the actual search results, the second section being displayed on the side at the right of the search results.

Make Money Fast
 Sponsored Links www.$3500weekly.com Learn to be a super affiliate Better Quit yr day job. It works! www.3500weekly.com Want to Make a Fortune? Make$55,000 to 1,000,000 a Year! Guaranteed success! - \$49 www.club-fortune.com Fast money work at home Ultimate wealth package try your luck and make1000’s ragstoriches.com NETWORK MARKETERS WANTED JOIN NOW FOR ENDLESS OPPORTUNITIES GROUND FLOOR COMPANY www.trilogyrx.com/essentials Sharemoney Investing up to 385% return for 7 days real short plan investment sharemoney.net Make Money Online Secrets To Make Money Online Revealed In Ebook. Free Download. TurboChargedInternetMarketing.com Make Money Online Today? You can once you know this secret Free audio shows how to get started www.secretofmakingmoney.com