Getting Real: It’s a problem when it’s a problem Ryan 03 Nov 2005

38 comments Latest by Sam

Don’t waste time on trouble spots that don’t even exist yet

Sure, you need to prepare a bit ahead of time but you should never inflate a problem until it actually is a problem.

For example:

  • Do you really need to worry about scaling to 100,000 users today if it will take you two years to get there?
  • Do you really have to hire eight programmers if you only need three today?
  • Do you really need 12 servers now if you can run on two for a year?

People often spend too much time up front trying to solve problems they don’t even have yet. Sometimes it’s better to wing it. Heck, we actually launched Basecamp without the ability to bill customers! Since the product billed in monthly cycles, we knew we had a 30-day gap to figure it out. We used that time to solve more urgent problems and then, after launch, we tackled billing. It worked out fine (and it forced us into a simple solution).

Don’t sweat stuff until you actually must. Don’t overbuild for scalability. Increase hardware and system software as necessary. If you’re slow for a week or two it’s not the end of the world. Just tell your customers you’re experiencing some growing pains. They’ll appreciate the candor.

Bottom Line: Make decisions just in time, when you have access to the real information you need. In the meanwhile, you’ll be able to lavish attention on the things that require immediate care and feeding.

38 comments (comments are closed)

Ian 03 Nov 05

There’s something to be said for holding off on fixing until there’s something to fix. But taken as an ideology, you’re just embracing firefighting. Planning isn’t dead.

Do you really need four digits for the year, when that won’t be a problem during your lifetime?

Andy 03 Nov 05

Good post JF - I agree for the most part, except:

“If you�re slow for a week or two it�s not the end of the world.”

This may contribute to your customers having a bad first impression of the site - that’s not something you can fix easily.

One of several Steves 03 Nov 05

If it were me, I’d recast that a bit in terms of not bothering with trying to address problems you’re not reasonably going to encounter. You want to make an app scalable, for instance, but not hugely so.

But you do have to consider that things will take longer to get around to than anticipated. I’ve been working with the same client for three and a half years now. When we launched a completely new site two and a half years ago, there were lots of things that we said “they’re not critical now, we’ll get to them later.” It is a good thing, because we wouldn’t have launched if we fixed everything. But, on the other hand, some of those same things still exist. The lower-profile stuff that still makes a difference - but not a show-stopping difference - is too easy to let slip through the cracks as other stuff comes up.

But keeping in mind to get the key stuff done and not plan for every last contingency is definitely a good idea.

Eric 03 Nov 05

it wasn’t a problem. was it?

Jamie Tibbetts 03 Nov 05

#1: Yes, you do need to worry about scalability in the beginning. Not planning for growth can be a huge problem down the road if you don’t build the foundation correctly. I don’t see this as something to avoid, because it really should take very little extra effort to plan for future scalability.

#2 and #3 should be common sense. You would hope that is. ;)

JF 03 Nov 05

This may contribute to your customers having a bad first impression of the site - that�s not something you can fix easily.

I’ll tell you what’s worse… Spending many thousands of dollars and hundreds of hours setting up server clusters with full redundancy and extremely high performance capabilities before you know if you are going to need them.

not my real name 03 Nov 05

I think you’re wrong here. Your examples are sound, but then you king of segue into something that I would consider preaching procrastination.

Maybe not procrastination, but rushing something out the door, unfinished and unprepared sounds like nothing but a folly. While things may work out sometimes, as your basecamp billing example did, it’s hardly a smart practice to just cross your fingers and hope that all the chips will fall into place.

not my real name 03 Nov 05

Typo’d “kind” into “king”. Oops.

Oh, and I agree with Jamie. Scalability is ALWAYS something you should have in mind.

JF 03 Nov 05

See… When you build really simple software (and less of it), you can change quickly when you need to change. Change is your friend.

It’s only when you have messy, elaborate, complex systems that you are scared of not getting it right the first time.

When change is cheap you can put off decisions until you have a lot more information — real information — to make them.

A lot of the decisions you make up front are either 1. wrong, or 2. not necessary or 3. lead to work you didn’t need to do.

One of several Steves 03 Nov 05

When change is cheap you can put off decisions until you have a lot more information � real information � to make them.

Good point. There are definitely systems that make chaging much more painful than it should be.

The problem with the approach is more of a human one than a systems one: sticking to the commitment to change things later. There are lots of little changes that should be made (because the cumulative effect of a lot of little things can be quite big), but it’s easy to let them keep getting pushed off by other things. To use Ian’s example, everyone thought back in the 60s and 70s that there would be plenty of time to fix the two-digit year thing. (Of course, it did end up getting fixed on time, but at great expense, and perhaps at greater expense than if they had just done it right from the start.)

Arun 03 Nov 05

I am interpreting the post this way…
…………………………………………………………
Your friggin architecture and design dont matter. What matters for success of an application are its features.

So dont worry too much (key word is “too much”) about things like scalability etc. Work on great features. Once it is popular, we can go back and look at thing like scalability. If I have the best design in the world but also-ran features, it doesnt help at all.


JF 03 Nov 05

I assumed this was assumed, but of course you need to take reasonable precautions and use your common sense. I’m not suggesting you throw crap out there that will break down when 5 people are using it at the same time.

Dave Tufts 03 Nov 05

In the immortal words of Donald Knuth, “Premature optimization is the root of all evil”

Detroit 03 Nov 05

Scalability may not have been the best example to use here…

My impression:

Jason just walked out of a very long, exhausting meeting after trying to answer every “what if” under the sun.

Just playing the devil’s advocate, they’ll say, just playing the devil’s advocate…

JF 03 Nov 05

I don’t have meetings (thankfully).

Megan Holbrook 03 Nov 05

Do you really need to worry about scaling to 100,000 users today if it will take you two years to get there?

Heh. You mean like not planning for the year 2000 by using a two-digit year designation? Or not assuming anyone would want to add over x# of items to a database…? I think not planning adequately for scalability can bite you in the butt big-time… ;)

Rabbit 03 Nov 05

I�m not suggesting you throw crap out there that will break down when 5 people are using it at the same time.

LOL.

To make it even better, my supervisor was walking by just as I laughed at this, and he spun around. (I didn’t look back at him.)

no name 03 Nov 05

yeah, don’t bother adding a search function to your blog until it gets popular, and then don’t add it after that either. it’s not like this is a “Design and Usability Blog” - ow, whoops!

wdk 03 Nov 05

Just for clarification, how does this fit in with ‘don’t carry debt?’

I mean, in the software itself, you could hack a fix with a couple lines of figurative duct tape. If it works, then it works, and there is no problem. So, to fix the faulty code is in essence solving a problem that isn’t a problem. But, part of not carrying debt (or whatever term you use for it, I forget) is to not have hacked fixes as described here.

Yes, I do know there’s a difference between the two. I’d just like to hear your take on this difference. Heck I’m here to read your take on things, after all. :-)

On a side note, of course not investing the $$ in servers up front helps by not incurring financial debt or at least letting you incur it for more pertinent (to now) reasons.

eek 03 Nov 05

This post reminds me of a quote i keep on my wall:

“A complex system that works is invariably found to have evolved from a simple system that worked….A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system.”
J. Gall

pwb 03 Nov 05

The discussion here is funny.

The biggest insight here is in fact #1. Scalability is one of the easiest things to add as you go. The only major site I can think of that had a scalability issue is Friendster and that was because of increasing the scalability of the existing system, they decided to wast tons of time re-building from scratch. The average developer who thinks scalability is super-important could only dream of having scalability issues!

Then, the Y2K thing is funny. People think they gotcha: “What about two digit years? Heh, heh, heh.”

What about 2 digit years? Total non-issue!!

Tomas Jogin 03 Nov 05

I absolutely agree. I find it a bit ridiculous when, for instance, programmers worry about scaling to high-number-of-users-they-can-only-dream-about years ahead when they should really be concentrating on problems they have right now.

Martin 03 Nov 05

I see it as this: Do no add like 15 layers into the software, so it can interact with everything, and every possible use is just a change in a config file.

The big time and effort needed to create a Does-It-All is not worth it. The vast “just in case” feature set will most likely not be used fully anyway. The ability to say “no problem, I just need to make a small change to a setting” on any request does not pay off.

So I agree with Jason that a flexible foundation (code and data) is key. Flexible in that it allows quick and easy changes, not flexible in that it already has everything under the sun built in already.

Christian Romney 03 Nov 05

I agree with the post, provided the obvious caveat: that the decision you make to day won’t *preclude* scaling to 100,000 later on. In other words, build the cluster when you need it, but make sure you’re not building something unclusterable!

sj 03 Nov 05

I’m blissfully ignorant of the scalability ‘process,’ but I’m curious as to what the line is - when does one divert their attention to infrastructure and away from product development/improvement? We definitely ran into problems with scaling up in time because we’re a bunch of idea people and our sales teams are very good at what they do. Last summer we got absolutely nailed with a deluge of business, and it made it difficult for up - our small-ish team was put into reactionary mode for a long time. As someone who doesn’t make the final decisions related to infrastructure but in the not-to-distant future probably will, what’s the best way to plan for things like this?

Probably not one answer, is there. Probably depends on the situation, doesn’t it. I’m so stupid….

Matt 03 Nov 05

Hate to be a stick in the mud here, but this is just silly.

While I do agree that there’s no point spending lots of cycles/money on something that is highly unlikely (or way off in the future), I really don’t think that’s what is driving this viewpoint. With all due respect, I think this is a simple lack of caring and laziness, articulated as a fancy “philosophy” in order to justify it. I also sense quite a bit of hubris.

Bottom line, there is a grain of truth here: Hyper-planning is unwise and a waste of resources. But making it up as you go is just as unwise. There’s nothing wrong with reasonable planning for likely eventualities. In fact, it’s a smart thing to do.

JF 03 Nov 05

Matt, I wouldn’t call it laziness, I’d call it good time management. Spending time doing things you don’t need to do now instead of spending it on things that need immediate attention is not good time management.

andrew 03 Nov 05

Both Matt and Jason are right: you must reasonably plan for likely eventualities. But you plan. At the highest level you can. You don’t *do* any more than is necessary.

But to operate in this way means you must be agile, responsive, adapative, comfy with change and a little bit of the unknown.

Jamie Thingbox 03 Nov 05

I think it’s a case of launching what you can get away with…

Our site has two pages of bullet points on the to-do list, which works out at about 100 hours of work left to do on it (which is quite a lot when it’s a spare-time project). That’s bug fixes and new features. Our ‘users’ are probably aware of 6 of the 50 or so things that need doing…

Even though the site’s not finished, the site gets busier every day and we get more members and everyone is happy. If we were still working on getting the site finished, we’d be locked in our room, no one would know about us, and we wouldn’t be spreading the love (so to speak).

FWIW, we’re not charging money to use our community site at the moment. People keep asking us how to give us money, and we tell them we’ll charge when it’s finished. There’s even talk of members collecting money from other members, and coming buy to thrust cash in to our hands. So I think we’re doing alright…

JF 03 Nov 05

Thomas, each Basecamp account has its own private URL. It ends with either projectpath.com, clientsection.com, seework.com, grouphub.com, or updatelog.com. This is where you login.

Ian 04 Nov 05

I’m fine with agility. I just think it’s easy to tout the advantages of just-in-time until you march your customers through a minefield.

I just want to know my Backpack account is safe, is all.

Thomas 04 Nov 05

JF: Thanks. I didn’t mean to sound bitter in the last post, your product were great. I just had a really bad hair day.

Jens Meiert 04 Nov 05

Guys, you are right. But it’s critical then that problems are addressed asap, so “one week or two” might be okay, but three or four aren’t anymore, otherwise you corrupt the user experience and drive customers away.

pwb 04 Nov 05

I agree with Thomas’ first post about it being not so simple to figure out how to log in to 37s services. The domain thing is cute but an uncommon approach. The home page for each service should at least provide some direction for how to access the service.

Sam 13 Nov 05

Perhaps the example (scalability vis-�-vis planning) wasn’t the best, but Jason’s philosophy about ‘getting real’ deals with a common problem: healthy perfectionism vs crippling perfectionism.

Crippling perfectionism is where you keep obsessing over the smallest details. It is a kind of obsessive compulsive disorder which creates a lot of anxiety.

To put it in lay man’s term - balance in life. Stop obsessing on the details of the plan and get off your butt and start working on the actual thing. I remember reading somewhere -

It takes 3 times more effort to move from good to best than the time it takes to create something good.

Sam 13 Nov 05

P.S: Please allow an option to preview the comment before posting.

I too had to keep referring to the bookmarks to remember my Basecamp URL. Finally solved the problem by creating a redirecting URL from my domain.