AutoBenching Capacity

When you are building a service it's important to know early on what kind of performance and scalability you can expect to achieve. However. measuring this isn't an easy task and there are lots of variables to take into account - the hardware, the bandwidth, the disparity between production and development platforms, logging while in development, etc.

I've been working on a new project recently using a strict platform model -- interaction between the various pieces is all via restful http services only (so nothing except the api has access to the underlying datastores etc). This means that we should be able to scale out horizontally very easily by throwing some load balancers between the various layers and that we should be able to test each of the interaction layers with a http load testing tool to get some idea of our capacity and performance under load.

Fun with Riak

I started work on a new project recently and we needed a key value store to store some quotes. We don't know how many quotes up front (less than a billion, more than 10 million). We wanted a key value store that would be resilient, easy to expand, and perform well.... I can't think of a better store for us than Riak.

So what is riak and why is it awesome?

Problems with QUOTENAME

So, lets get one thing out of the way before we begin.... Dynamic SQL is generally a bad plan. That being said there are a few scenarios where it is genuinely the best choice. For example in catch all type queries commonly encountered on web forms. (as this excellent post by Gail Shaw illustrates).

A major concern whenever you use dynamic SQL is SQL Injection. To secure yourself from this you have to validate your inputs for any DML code - (If you use parameterised dynamic sql with sp_executeSql then you are secure from a pure parameter standpoint)

Agile Project Management with Trello

We recently started a new project with a smallish development team (of 7 people) scattered through 3 physical locations. After we got everyone together for our inception we decided to give Trello a try.

Data Sizes

Ever wondered just how much space your data is using? If you have many indexes then it may be more than you expect..... Lets find out!

1) Run this:

Table_Rows_DataSize_SQLServer2008.sql

2) Put the data into excel or some other visualization tool.

3) Generate some pretty pictures

4) Realize just how *totally* skewed your data is.

5) Start building a good key value store.......