Alan Atlas at Scrumtisch Berlin

2010-02-17 23:35
At the last Berlin Scrumtisch, @AlanAtlas gave a presentation on how he introduced Scrum at Amazon (starting as early as back in 2004). Introducing Scrum at Amazon by that time seemed natural due to a few factors:

  • Amazon was and is always very customer centric. The original methodology of working backwards in time - that is starting with the press release, from there writing the FAQ and manual and finally get to the code - really made people concentrate on the product.
  • Teams were sized according to the two-pizza rule: Each team is supposed to be no larger than can reasonably be fed for lunch with two pizzas. Turns out that is about five to ten people, given regular american pizzas.
  • Management didn't really care exactly *how* the job of software development was done. They only cared *that* it was done. This proved as both - an advantage as it gave quite a bit of freedom - and a disadvantage - as indifference leads to impediments in the process that cannot be easily resolved.

The goal of Alan's team was to build an infinitely scalable, zero downtime storage system that had a web service based interface - it was the S3 team. Given the task at hand, it was clear that the task wasn't going to be anything even close to trivial. Alan's idea: Try Scrum on that.

He himself came to the idea after attending an Agile conference and hearing about Scrum for the first time there. Alan requested a Scrum training from his manager, who approved it, provided it took place in Seattle - close to Amazon HQ that is. Turns out, the Scrum trainer available in Seattle happened to be Ken Schwaber.

The idea of Scrum basically was spread by word of mouth propaganda: In the hallways, in the smokers lounge, close to the coffee machine. People started adopting it - with some teams doing better, some worse. Allan himself got two days a quarter to give people an introduction to this funny new methodology called Scrum.

18 months later, S3 was shipped - as a side effect, the number of subscribers to the internal scrum mailing list had increased from three to 100, there were 150 CSM graduates, still three teams using XM, reputation of scrum was mostly positive.

In January 2007 others started giving their own introductions to Scrum. Developers generally liked it, there were significant failure cases, lots of resistance and misunderstanding mostly on the management side who sometimes confused it with lean production.

In June 2008 Allen became a fulltime internal Scrum trainer. First thing organised was a Scrum gathering - in an open spaces like setup people were invited to discuss their issues and questions on Scrum.

In January 2009, about 50% Scrum usage was reached, their were 600 subscribers on the Scrum mailing list, 450 CSM graduates, reputation of Scrum was as good (or bad) as months before. However it became more and more clear that further restructuring would only be possible against a lot of resistance.

There were a few lessons learned from introducing Scrum in other companies as well:

  • You cannot introduce Scrum if there is no notion of comparably stable teams (as in for at least six months, not five projects at a time per developer).
  • You need permission and ownership of committments to really get going.
  • You need to know at least the basics of Scrum.

There are a few factors that make introduction easier: Community matters. People need to be able to talk to each other, exchange experiences and learn from one another. Coaching matters. More support, immediate success in lots of cases is the result of getting a coach on the team. Credibility for management increases, Scrum implementation is easier understood that way, skepticism and resistance reduced. Management matters. Middle management must be part of the process. They will mess up scrum if not educated correctly. Going bottom-up works up to a point - but to go the whole way, you do need management.

For Amazon, that is the point where implementation got stuck for Allan: Management was neither interested nor really involved in Scrum. However there are some impediments that cannot be fixed w/o. Still Scrum is working to some extend - teams are still trying to get better and improve the process. So it is not really "Scrum, but". Even going only part of the way helped a lot already.

As for the questions: There were a few unusual ones - e.g. on what to do with a team that is itself skeptical about Scrum introduction - especially if that was done by higher management. There are a few ways to remedy that: Point out success to the negative team member. Make that member a Scrum master to let him go through the process and really understand what is going on. And above all make the reasons for going that way clear and transparent. Including promises and wishes from that change.

Another question dealt with how to convince management of Scrum. Clearly better performing teams with happy developers ("you cannot buy developers with money only") are some valid reasons.

Yet another question dealt with convincing customers. Allan's way is to sneak Scrum into the process: Speak the customers language. If he does not like spring planning, call it a project status meeting.

Asked for cases of developers feeling like they work in a hamster wheel when doing Scrum, the general consensus was that it needs to be made sure that people do not only get more responsibilities but get the benefits from Scrum as well. Otherwise Scrum with all its numbers and measurements can be perfectly abused and turned into an amazing micro management tool.

Asked for other bay area companies using Scrum - yes, apple does so, Microsoft at least tries to. Google certainly uses the ideas, without calling it Scrum. Facebook seems not to use it. Xing (not bay area) uses Scrum. There seem to be about 50% of all software development companies worldwide who are using Scrum.

After the talk there was time to gather and have some pizza and pasta, time for drinks and discussions. I really appreciated the comments and ideas exchanged after the talk. Hope to see you all next time around.

Apache brand name - survey

2010-02-16 19:03
Sally Khudairi (VP, ASF Marketing & Publicity) asked for distributing the following survey to people who might be interested in contributing their views to a study on how the brand name Apache is perceived. Me personally, I would be especially interested in finding out more on whether there are any differences in perception inside the ASF vs. outside...

We have been working with PhD candidate Roland Schroll over the past two years as he's been compiling information on the value of the Apache brand. His advisor is community-based innovation expert Dr. Johann F├╝ller. This is a joint project of the University of Innsbruck and the Massachusetts Institute of Technology.

If you have 10 minutes to help, it would be much appreciated. The survey is at

They would like the surveys to be completed this month (February).

They are seeking at least 300 respondents. As such, if you know others who are interested in Apache from a market perspective, feel free to forward the link to them as well.

Thanks in advance for your interest!

Kind regards,
Sally Khudairi
VP, ASF Marketing & Publicity

MLOSS workshop at ICML accepted

2010-02-15 19:31
The workshop on machine learning open source software has been accpted. Find further details on the workshop homepage.

Submissions are open until April 10th, Samoa time.

FOSDEM - video recordings online

2010-02-14 20:32
As published in the FOSDEM blog the video recordings are available online - at least for the main track and the lightning talks. Happy video watching!

FSFE Happy Valentine

2010-02-14 07:05
Today I got woken up with a friendly hug and roses waiting for me:

I do not really care about presents for sort-of-artificial celebration days like valentines day. However, FSFE had a very nice idea: The proposal was to use valentines day to show your love for free software. The website proposed to e.g. hug a free software developer, to make a gift to a team of free software developers:

I love Free Software!

Happy Valentine!

Open Community Camp 2010

2010-02-12 13:07
The following information just reached my via Marten Vijn. Thought it might be interesting to you:

I am pleased to announce OpenCommunityCamp 2010.

The camp is from 10th to 18th July, in Oegstgeest, the Netherlands.

The website[1] is refreshed and the first speakers are booked.
It is time to register[2] if you plan be there (please do this

Currently we need to find more people to attend, self-organizing groups
for the day program and interesting speakers for the evening program.

I look forward to hear your ideas and plan if you come. If you have
any questions don't hesitate to mail me.

kind regards,
Marten Vijn


Berlin Buzzwords - June 2010

2010-02-11 22:42
As announced at FOSDEM: Early June (currently scheduled for 7th/8th) a conference on the topics scalable search, storage and processing will take place in Kalkscheune/Berlin. The conference is co-organised by newthinking store, Jan Lehnardt, Simon Willnauer, Thilo Fromm, and Isabel Drost.

The focus will be on NoSQL databases like CouchDB, Jackrabbit, MongoDB, HBase. Search tracks will cover topics like Lucene, Solr, katta and others. Data munging tracks will focus mainly on Hadoop, MapReduce in general and distributed systems.

More information including the call for presentations will be made available online next week on a separate webpage. Early registration starts in March. Watch this blog for more information or follow @hadoopberlin.

Apache Hadoop Get Together - March 2010 - Update

2010-02-11 14:25
Due to conflicts in the schedule of newthinking store, we had to change the time of the Get Together slightly. We will start one hour earlier than announced.

When: March 10th, 4p.m.
Where: newthinking store, Tucholskystr. 48, Berlin Mitte

Looking forward to seeing you there.

FOSDEM 2010 - part 3

2010-02-10 21:02
Sunday started in Janson with Andrian Bowyer's talk on RepRap machines, that is devices that can be used as manufacturing devices and are able to replicate themselves. After that I went over to the Mono dev room to listen to Miguel de Icaza on Mono Edge. A great talk on the history of Mono, the way the community interacts with Microsoft, the C# language itself and special features only available in Mono.

After this talk we went over to Janson for Andrew Tanenbaum's talk on Minix. We knew quite a bit of the talk already from Froscon two years ago, however Andrew is an awesome speaker, so it's always fun to catch up on the news on Minix.

The scalability talk started with an introduction to Hadoop by myself and continued with a talk on the facebook infrastructure by David Recordon. According to feedback I got after the talk, laughing with Thilo helped quite a bit to get myself calm. Before the talk I received one very good recommendation of one of the audio guys: Imagine you are giving the talk to one of your best friends - and forget about the microphone. Though I had way more slides than minutes to talk, we had enough time for the Q&A session after the talk. I started the talk by learning more about the audience - however this time not by handing the microphone to those listening (room too large) - I just asked them "have you heard about Hadoop?" - half of the audience. Are you Hadoop users: one quarter maybe. How large are your clusters? - 10 to 100 nodes mostly. Have you heard of Zookeeper? - some, Hive - some more, Pig - a few, Lucene - a lot, Solr - a little less, Mahout - maybe 5, Mahout users: 1.

Turns out the Mahout user in the audience was Olivier: It's so nice to meet people you know are active on the mailing lists for real and have a chat with them. Hope to see you more often on the lists - and meet you face to face again.

I used the chance to announce the Berlin Buzzwords 2010, a two day event on search and scalability buzzwords like cloud computing, Hadoop, Lucene, NoSQL and more. It takes place on June 7th and 8th in the center of Berlin. Follow this blog for further information. Judging from the input I got after the announcement there is quite some need for such a conference in Europe.

The slides of my talk are soon to be available online.

After my talk I could stay in Janson: A talk on the Facebook infrastructure (not only the Hadoop side of things) followed. After that I met Lars George at the NoSQL dev room - unfortunately I did not manage to actually talk to Steven Noels, who organised the room.

The afternoon was reserved for Greg Kroah-Hartman on how to "Write and submit your first Linux Kernel Patch" - my personal conclusion: git is really awesome. I really, really need to find a few spare minutes to learn how to effectively use it.

In the evening we met with Pieter Hintjens for dinner - and to finalize an awesome weekend in Brussels and a great 10th anniversary FOSDEM. A huge Thank You to all volunteers and organisers of FOSDEM - you did a great job this year putting together an awesome schedule, you did a fantastic job making the now pretty huge event (with 306 talks and about 5000 hackers attending) run smoothly. Even the wireless was working from minute one. See you again at FOSDEM 2011.

FOSDEM 2010 - part 2

2010-02-09 21:00
The event itself featured 306 talks - so pretty hard to choose what to watch on two days. This time, not only the main tracks were awesome, but also several dev rooms featured very interesting talks by well known FOSS developers.

Saturday started with a FOSDEM birthday dance done by all attendees. The first keynote speaker Brooks Davis explained his experiences promoting open source methods at a large company. After that Richard Clayton gave an amazing talk on the evil on the internet. He explained not only how phishing works on a technical level but also included an explanation of the economics behind these attacks, explained how the money flow from victims to attackers works.

On the afternoon Bernard Li gave an introduction to the cluster monitoring tool Ganglia. Directly after that Lindsay Holmwood gave an overview of the monitoring and notification tools flapjack and cucumber-nagios.

The evening was filled with the speakers dinner. Thanks for the organisers for providing that. We had a really nice evening together with some of the organisers, Andrew Tanenbaum and Elena Reshetova at our table.