Agile, inner source, open development, open source development

2017-05-07 15:53

Last week I had the honour of giving a keynote at Adobe's Open Source Summit EU in Basel. Among many interesting talks they hosted a panel to answer questions around all things open source, strategy, inner source, open development. One of the questions I found intersting is how inner source/ open development and agile are related.

To get everyone on the same page, what do I mean when talking about inner source/ open development? I first encountered the concept at ApacheCon EU Amsterdam in 2008: Bertrand Delacretaz, then Day Software, now Adobe was talking about how to apply open source development principles in a corporate environment. A while ago I ran across the term that essentially wants to achieve the same thing. What I find intersting about it is the idea of applying values like transparency, openess to contributions as well as concepts like asynchronous communication and decision making at a wider context.

What does that have to do with Agile? Looking at the mere mechanics of Agile like being co-located, talking face to face frequently, syncing daily in in person meetings this seems to be pretty much opposite of what open source teams do.

Starting with the values and promises of Agile though I think the two can compliment each other quite nicely:

"Individuals and interactions over processes and tools" - there's a saying in many successful popular projects that the most important ingredient to a successful open source project is to integrate people into a team working towards a shared vision. Apache calls this Community over code, ZeroMQ's C4 calls it "People before code", I'm sure there are many other incarnations of the same concept.

Much like in agile, this doesn't mean that tools are neglected alltogether. Git was created because there was nothing quite like this tool before. However at the core of it's design was the desire for ideal support for the way the Linux community works together.

"Working software over comprehensive documentation" - While people strive for good documentation I'd be so bold as to suggest that for the open source projects I'm familiar with the most pressing motivation to provide good documentation is to lower the number of user questions as well as the amount of questions people involved with the project have to answer.

For projects like Apache Lucene or Apache httpd I find the documentation that comes with the project to be exceptionally good. In addition both projects are supported with dead tree documentation even. My guess would be that working on geographically distributed teams who's members work on differing schedules is one trigger for that: Given teams that are geographically distributed over multiple time zones means that most communication will happen in writing anyway. So instead of re-typing the same answers over and over, it's less cost intensive to type them up in a clean way and post them to a place that has a stable URL and is easy to discoveri. So yes, while working software clearly is still in focus, the lack for frequent face to face communication can have a positive influence on the availability of documentation.

"Customer collaboration over contract negotiation" - this is something open source teams take to the extreme: Anybody is invited to participate. Answers to feature requests like "patches welcome" typically are honest requests for support and collaboration, usually if taken up resulting in productive work relationships.

"Responding to change over following a plan" - in my opinion this is another value that's taken to the extreme in open source projects. Without control over your participants, no ability to order people to do certain tasks and no influence over when and how much time someone else can dedicate to a certain task reaciting to changes is core to each open source project cosisting of more than one maintaining entity (be it single human being or simply one company paying for all currently active developers).

One could look further, digging deeper into how the mechanics of specific Agile frameworks like Scrum match against what's being done in at least some open source projects (clearly I cannot speak for each and ever project out there given that I only know a limited set of them), but that's a topic for a another post.

The idea of applying open source development practices within corporations has been hanging in the air for over 15 years, driven by different individuals over time. I think now that people are gathering to collect these ideas in one place and role them out in more corporations discussions around the topic are going to become interesting: Each open source community has their own specifics of working together, even though all seem to follow pretty much the same style when looked at from very high above. I'm curious to what common values and patterns can be derived from that which work across organisations.

Note to self: Backup bottlenecks

2014-03-23 18:26
I learnt the following relations the hard way 10 years ago when trying to backup a rather tiny amount of data, went through the computation again three years ago. Still I had to re-do the computation this morning when trying to pull a final full backup from my old MacBook. Posting here for future reference: Note 1: Some numbers like 10BASE-T included only for historic reference. Note 2: Excluded the Alice DSL uplink speed - if included the left-hand chart would no longer be particularly helpful or readable...

How hard can it be - organising a conference

2014-03-15 19:37
Setup a CfP, select a few talks, publish a schedule, book a venue, sell a few tickets - have fun: Essentially all it takes to organise a conference, isn't it?

In theory maybe - in practice - not so much. Without scaring you away from running your own here's my experience with setting up Berlin Buzzwords (after two years of running the Berlin Hadoop Get Together, putting up a NoSQL half day meetup).

Disclaimer: Though there's a ton of long-ish posts in my blog, this one is going to be exceptionally long. It's years since I wanted to write all this up for others to learn from and in order to point others to it - never got around to it though.

Venue booking

Lets start with the most risky part: Booking a venue. If you are a student - great - talk to your university to get your event hosted. As much hassle this may entail when it comes to bureaucracy this is by far the largest monetary risk factor in your calculation, so if there is any way to get rid of this factor use it. FOSDEM, FrOSCon, Linux Tage Chemnitz and several other events have handled this really well.

If this is not an option for you depending on the number of targeted attendees you are faced with a smaller (some 500 Euro for 100 people per day and track) or larger (some 20.000 Euros for 300 people, two days and two tracks including chairs, tables, microphones, screens, projectors, lighting, security and technicians) sum that you will have to pay up front, potentially even before your attendees purchased any tickets (here's one reason why there is such a thing as an early bird ticket: The earlier money flows in the less risk there is around payments).

In Germany there's three ways to limit this risk: You create a so-called e.V., essentially a foundation that will cover the risk. You create a GmbH, essentially a company that assumes limited risk, namely only up to the 25.000 Euros you had to put into the company when founding it. The third way is to find a producing event agency that assumes all or part of the risk. After talking to several people (those behind Chaos Communication Congress, those setting up the Berlin Django Con back in 2010, those working on JSConf, those backing Hadoop World) for me it was logical to go for the third way: I had no idea how to handle ticket sales, taxes etc. and I was lucky enough to find an agency that would assume all risk in turn for receiving all profit made from the event.

One final piece of advise when selecting your venue: Having a place that is flexible when it comes to planning goes a long way to a relaxed conference experience. As much as it is desirable you won't be able to anticipate all needs in advance. It also helps a lot to keep the event on your home turf: One of the secrets of Berlin Buzzwords is having a lot of local knowledge in the organising team and lots of contacts to local people that can help, support and provide insight:

By now at Berlin Buzzwords we have a tradition of having an after conference event on Monday evening. The way the tradition started relied heavily on local friends: What we did was to ask friends to take out up to 20 attendees to their favourite bar or restaurant close to the venue in turn for a drastically reduced ticket price. This approach was highly appreciated by all attendees not familiar with Berlin - several asked for a similar setup for Tuesday evening even. Starting from the second edition we were able to recruit sponsors to pay for beers and food for Monday evening. Pro-Tipp: Don't ship your attendees to the BBQ by bus - otherwise the BBQ cook will hate you.

Something similar was done for our traditional Sunday evening barcamp: In the first two years it took place at the famous Newthinking Store - a location that used to be available for rent for regular events and free for community events. Essentially a subsidary of our producer. In the second year the Barcamp moved to one of the first hacker spaces world - c-base. Doing that essentially was possible only because some of the organisers had close contacts among the owners of this space.

The same is true for the Hackathons and meetups on Wednesday after the main conference: Knowing local (potential) meetup organisers helps recruiting meetups. Knowing local startups helps recruiting space for those people organising a meetup though themselves travelling e.g. from the UK.


Another risk factor is providing catering: If catering is included in your ticket price, the caterer will probably want to know at least one week before door open roughly how many people will attend (+/-10 people usually is fine, +/- 100 people not really). When dealing with geeks this is particularly tricky: These guys and gals tend to buy their tickets Saturday/Sunday before Berlin Buzzwords Monday morning. In the first two years that meant lots of sleepless nights and lots of effort spent advertising the conference. In the third night I decided that it's time for some geek education: I suggested to introduce a last minute ticket that is expensive enough to motivate the majority of attendees to buy their tickets at least two weeks in advance. Guess what: Ever since we the problem of "OMG everyone is buying their ticket last minute" went away - and at least I got my sleep back.

The second special thing about catering: As much as I would like to change that Berlin Buzzwords has an extremely skewed gender distribution. When dealing with caterers what they usually only want to know is how many people will attend. If you forget to tell them that all your attendees are relatively young and male they will assume a regular gender distribution - which often leads to you running out of food before everyone is fed.


Except for tickets - are there any other options to acquire money apart from tickets? Sure: convince companies to support your event as sponsors. The most obvious perk is to include said company's logo on your web page. You can also sell booth space (remember to rent additional space for that at your venue). There's plenty of other creative ways to sell visibility. For package sizing I got lots of support from our producer (after all they were responsible for budgeting). Convincing companies to give us money that was mostly on Simon, Jan and myself. Designing the contracts and retrieving the money again was left to the producer. Without a decent social network on our side finding this kind of financial support would not have been possible. In retrospect I'm extremely glad the actual contract handling was not on me - guess what, there are sponsors who just simply forget to pay the sum promised though there is a signed contract...


So, what makes people pay for tickets? A convincing speaker line-up of course. In our case we had decided to go for two invitation only keynote speakers and two days with two tracks of CfP based talks. Keynote speaker slots are still filled by Simon and myself. In early years where CfP ratings, schedule grid (when and how long are breaks? how many talks will fit?) and scheduling itself were on Simon, Jan and myself. As submission numbers went up we decided to share the review load with people whose judgement we trust - ensuring that each submission gets three reviews. All after that is fairly algorithmic: See an earlier blog post for details.

A note on review feedback: As much as we would like to give detailed feedback to those who didn't make it: We usually get three times as many submissions as there are open slots. So far I haven't seen a single submission (except for very clear product pitches lacking technical details) that wasn't great. So in the end, most feedback would boil down to "It was really close, but we only have a limited number of slots to fill. Sorry."

Back in 2011 we tried an experiment to fit more talks into the schedule than usual: Submissions ranked low that were supposed to be long talks were accepted as short versions forcing speakers to focus. Unfortunately this experiment failed: Seems like people rather get a reject mail than getting accepted as a short version. Also you need really good speakers who prepare exceptionally well for the event - in our case many shortened talks would still include all introductory slides even though the speaker just one slot earlier had covered those already.


Next step is to tell people that there is going to be an event. In the case of Berlin Buzzwords this meant telling people all over the world and convincing them to not only buy a conference ticket, but also a hotel room and a flight to the venue (we started with only half of all attendees coming from Germany and pretty much kept this profile until today). As a first step this meant writing a press release and convincing local tech publishers (t3n, heise, Golem, Software und Support, Open Source Press and many more) to actually publish on the event. For some of these I am an author myself, so I knew which people to talk to, for some of these newthinking as the producing event agency could help. It was only years later that I participated in a media training at Apache Con by Sally Khudairi to really learn how to do press releases.

From that day on, a lot of time went into convincing potential speakers to submit talks, potential sponsors to donate money, potential attendees to make it to the event. With a lot of event organising experience on their back (they are running a multiple thousand attendees new media conference called re:publica each year - in addition to many "smaller" events) newthinking told me upfront that they would need half of my time to cover those marketing activities, essentially as I was the only one who knew the community. The offer was to re-imburse 20h per week from the conference budget. I was lucky enough to be able to convince my manager at neofonie (the company I was working for back then) though that sponsoring the event with my time in turn for a silver sponsorship would be a great opportunity. One of the reasons for doing that on their side was that they were themselves providing services in the big data and search space. Without this arrangement though Berlin Buzzwords 2010 would have been a whole lot harder to do.

Now how do you reach people for a tech conference? You talk to the press, you create and promote a twitter account. I still have the credentials for @berlinbuzzwords - by now though it is fully managed by our social media expert, back until 2011 I was the face behind it. Ever since my twitter client is set to follow anything that contains bbuzz, berlinbuzzwords or "berlin buzzwords" - so I can still follow any conversation relating the conference. You create and maintain LinkedIn and Xing as well as Facebook groups for people to follow. You use whatever channels your target audience are reading - in our case several Apache mailing lists, a NoSQL mailing list, sourceforge hosted lists - remember to sign up for these before posting, otherwise your mail won't get through. Also make sure that people at least roughly know your name, trust you and you provide enough context in your mail for others to not view your invitation as plain spam.

Finally you go through your personal address book and talk to people you know would be interested personally. As a result you will wake up to 20 new mails each morning and answer another 20 new mails every evening. For me this got better after having shaped all contacts into a format that I could hand over to newthinking. However even today, every single mail you send to [target], every comment you submit through the contact form, every talk wish you submit through our wishlist still ends up in my personal inbox - as well as the inbox of Simon and everyone involved with the conference at newthinking.

Essentially you need this kind of visibility once for announcing the event, once for filling the CfP with proposals, once the schedule in published, once to convince sponsors to support the event and finally once to convince people to buy tickets.

As for the website - the most frustrating part is being a technical person but lacking the time to "just do things right". Drupal, I'm sorry, but I have learnt to hate your blogging functionality - in particular the WYSIWYG editor. Your administrative backend could be much simpler (I gave those rights away I believe in 2012). I learnt to hate comment spam more than anything else - in particular given the fact that I pretty much would have loved to get everyone involved and able to contribute content. The only thing that helped accepting the deficiencies here was to force myself to hand of any and all content handling to the capable event managers at newthinking.


Videos: Great way to get the word out (and follow the talks yourself, though organisers may find time to go to talks they'll never remember the actual content due to information overload). Make them free - people pay for tickets to take part in the live event. If you fear selling less tickets as a result, make them available only a little while after the event is over.

Pictures: Get someone with a good camera - can be a volunteer, can be a professional or anything in between. I've had it many times that it took half a year for me to go over the pictures again and suddenly realise how many nice people attended Buzzwords without me knowing them when they did - except they remembered my face when we met again (sorry if you are one of those people I should have remembered once :( )

Inbox: Your's will never be empty again. Especially if as in our case your mail address was used as primary point of contact and reference in the first few years. It took two editions to train people to use instead of my personal mail address. Trust me - this mail address actually does get attention: Mails sent there end up in my private inbox, they end up in Simon's private inbox and most importantly they end up in those event managers' inboxes involved with Buzzwords at newthinking. Answers typically don't take much longer than half a day even during holidays. Today there is no reason left to contact neither Simon or myself privately - using info@ is way faster. If you still aren't convinced: Even I'm using that same address for general inquiries and proposals.

Incentives: Think early about which behaviour you would like to see. On site behave accordingly. Pre-conference set incentives: Two weeks before doors open our ticket prices go up drastically to motivate people to buy tickets before our catering deadline bites us. Speakers get travel support and hotel room paid for. However it costs us nothing to list their employer as travel sponsor should they decide to pay for the speaker - and it provides an incentive for the speaker to get their employer to pay for travel costs.

Ticket prices: You will get people arguing that ticket prices are too high. Know where you stand in comparison to other conferences of the same type. Clearly you want students if the main focus of your sponsors is to recruit - provide drastically reduced student tickets to anyone with a student id, that includes PhD. students. For everyone else: Buying early should be rewarded. Also you'll need help on site (people moderating sessions, keeping full rooms closed, people helping with video taping, networking etc.) - hand out free tickets to helpers - if those complaining aren't willing to help their need for a cheap ticket probably isn't large enough.

The "they are stealing our attendees syndrome": Unless there is a clear trademark infringement there's no way to stop other people from running events that on first sight look similar to yours. First of all start by making it hard to beat your offering - not in terms of prices but in terms of content and experience. After you've done that follow the "keep your friends close but your enemies closer" principle by embracing those who you believe are running competing events. What we had in the past was events close in topic to ours but not quite overlapping. Where there was enough overlap but still enough distinction we would go out and ask for partnering. This usually involved cross-raffling tickets. It also meant getting better visibility for our event though different channels. Usually the end result was one of two: A) The competing event was a one-of or otherwise short-lived. B) The seemingly competing event targeted an audience that was much more different from ours than we first believed.

On being special

What makes people coming back? I have been told Berlin Buzzwords has a certain magic to it that makes people want to come back. I'm not sure about the magical part - however involving people, providing space and time for networking, choosing a venue that is not a conference hotel, always at least trying to deliver the best experience possible goes a long way to make attendees feel comfortable. As a last note: If you ever once organised a meetup or conference you will never attend other events without at least checking what others do - you will suddenly see all the little glitches that otherwise slip from your attention (overly full trash bins anyone?). On the other hand each event brings at least one story that when it happened looked horrible but turns out to be hilarious and told over and over again later ;)

Berlin Buzzwords - Associated Events

2014-02-13 17:00
Back in 2011 I had a weird idea: Berlin Buzzwords as a core event kicks off on Sunday evening with a Barcamp but closes on Tuesday evening. There's way too little time to meet with all the interesting people. On the other hand the organising team really was all tired on Tuesday evening, so just adding another day at the end wasn't really an option.

Now Berlin is known as one of the hottest European Startup Capitals. There's literally at least one meetup for any topic you can imagine. There's co-working spaces and companies happy to welcome developers for a one day meetup or Hackathon. So the idea was born to invite people to organise meetups, hackathons, workshops, unconferences or any kind of dev session of their choosing on Wednesday after Berlin Buzzwords - the goal being to keep Buzzwords speakers and attendees in town for longer.

In the mean time several meetup groups actually started by having their first event on Wednesday after Berlin Buzzwords. It's grown increasingly easy to find meetup space for these Wednesday meetups.

So how does it work? Essentially it's very easy: All that is needed is one person interested in a particular technology who is willing to organise a meetup for 20 to 120 people in Berlin on Wednesday after Buzzwords (or the weekend before, or the weekend after, it's really up to you).

It'd be greatly appreciated if you get in touch with or even submit your meetup through the official CfP. In return it will receive marketing help and will be included in the official schedule page. (Note: Meetups are accepted as they come in - no review phase, no nothing).

If you need help finding a venue for your meetup (either for free or for rent, up to your preference) talking to will help you. We'll get you in touch with local startups, co-working spaces and event venues.

Also remember - you don't need to be an expert in the topic you are organising the meetup for. All you need is justified interest in the topic. Don't be shy to invite your favourite developers as speakers for your meetup. Don't shy away from even approaching some of the Berlin Buzzwords speakers.

If you are a developer yourself think about turning your Wednesday event into a hackathon - either in order to get more users familiar with your project or to gt the next release out the door together with other people working on your project also attending Buzzwords. This kind of approach really is the reason why there is a correlation between the date Berlin Buzzwords takes place and Mahout release dates in summer. (If you need inspiration on what it takes to organise a well working Hackathon - a quick search on your favourite search engine for "hackathon howto" should surface plenty of documentation.)

I love FS 2014

2014-02-13 16:49
It's that day of the year again: Time to buy flowers and chocolate for your beloved one. However as with previous years, FSFE wants you to put the day to good use to also celebrate your favourite free software developer (you know, the people who get way more bug reports and complaints than positive feedback):
I love Free Software!
So here's to the people over at Apache, Debian, Eclipse, Elasticsearch, Linux, ZeroMQ and the many other projects that make my life easier: Happy I love Free Software Day - get yourself celebrated!


2014-02-03 14:53
By now FOSDEM turned into some kind of tradition in our family: Since 2007 every year in February we are travelling to that one comfy B&B in Brussels for a weekend - not to take a closer look at the city but to attend (together with thousands of other geeks and open source hackers) one of the biggest conferences on all things open source.

I love the conference concept for scaling: As it happens on a university campus they only have a limited number of huge rooms, but a fairly large number of mid-sized and small rooms. They use the venue to their advantage: Only the main tracks are organised by the official conference organisers. All other rooms are each filled with a community (usually a handful of people doing the heavy lifting of inviting speakers, putting a cfp out etc.) provided track. As a result even though they attract well over 5000 attendees it's only the most popular topics (e.g. Elasticsearch, GPL license discussions, configuration management, JDK specifics, kdbus, discussions on the relevancy of distribution packaging) tend to fill rooms completely.

When stuck with full rooms there's always some other talk on that probably is similarly interesting. In addition there is a huge exhibition area to visit where you can talk to the core developers behind the individual open source projects (and buy stuff like T-Shirts, grab a few flyers, stickers etc.). In addition there's so many people to talk to you usually won't get to see too many presentations anyway. In addition as of this year the crazy people of the FOSDEM (formerly Debian-only) video team managed to get (on a best effort, "hopefully the technician doesn't sleep in after that many beers yesterday" basis) all 445 talks video-taped with the goal of putting these online Monday after the conference.

On top I had the huge advantage of being able to simply follow to the talks my husband would go to if all of the stuff I am interested in is full: That would give me insight to completely different topics but also make it extremely easy for me to identify the good speakers and interesting presentations based on his experience avoiding the "if I'm unfamiliar with the topic and area I usually bump into the boring, badly given talks" problem.

I spent most of Saturday in a few talks on software patents, Daniel Naber's Language Tool, the Jolla BoF, the FLA. The rest of the time I caught up with the FSFE (thanks again for the bringing the onesies!), Debian, Open Office, Lucene and Elasticsearch people. The day ended at the Jolla community dinner - with a waitress that was completely overwhelmed by the number of geeks wanting food, beer and soft drinks.

Sunday I started with Chris Kühl's talk on Memory Tuning Android for Low Memory Devices. After that I helped at the Elasticsearch booth - FOSDEM definitely is an incredibly busy event. But given their target audience also is an interesting mix of people to talk to: Some had no idea about how text search works but came over because they had heard about Wikipedia using the project for search and wanted to know more. Some had a very clear idea of how they could benefit from Lucene for their NLP projects and wanted to know more on how that fits with Elasticsearch. Others were switching from Solr Cloud and needed some advise on how the systems compare for their particular use case. Others again were using Elasticsearch to analyse log files in a distributed fashion and needed advise on how to implement some feature. There was this one guy from Debian I've known ever since helping at the FSFE booth at Chemnitzer Linuxtage back in 2008 I believe who wanted to know more about Elasticsearch because one of his fellow package maintainers had been volunteered to work on the Elasticsearch RFP (#660826 if you are interested in reading more).

Overall (as every year) a really pleasant experience. What was in particular interesting this year was to meet people I knew only from completely other contexts (CCC events, system administrators, core Apache httpd people). Seems like FOSDEM is not only growing bigger but also more diverse.

The only kind of feedback I would provide is to split some dev rooms by finer grained topics to parallelise and scale even better (the Java, NoSQL and configuration management ones come to my mind first but there probably are others as well - of course again this depends on room availability and actual community members volunteering to submit a dev room related to their project in particular).

I'm glad I'm travelling home by train like last year - not only does that give me time to code and write the blog post you are reading just now, it's also comfortable to get rid of the usual sleep deprivation :)

On being aggressivly public

2014-02-01 19:06
If it didn't happen on the mailing list it didn't happen at all.
... with all it's implications this seems to be the hardest lesson for newcomers to the Apache way of development to learn.

In the minimal sense it means that any decision a project takes has to be taken publicly, preferably in some archived, searchable medium. In a wider sense it's usually interpreted as:

  • ... don't ask people getting started questions privately by mail, rather go to the user mailing list.
  • ... don't coordinate and take decisions in video conferences, rather take the final decision on the dev mailing list.
  • ... don't ask people privately on IM, rather go to the user mailing list.

There are even canned answers that can get sent out to those asking privately.

This concept seems to be very hard to grasp for people: As a student, developers tend to be afraid to ask "stupid questions" - it's easy to say that "there are no stupid questions, only stupid answers" - truly feeling this to be true can be hard though in particular for people who are just beginning.

However (assuming the question hasn't been answered before or the answer wasn't easy to find with a search engine of your choice) asking these questions is a valuable contribution: Chances are, there are a couple other people who would have the same question - and are too shy to ask. Chances are even bigger that those developing the project would never think of documenting the answer anywhere simply because they do not see it as a question due to their background and knowledge.

For researchers in particular on their way to getting a PhD. the problem usually is the habit of publishing facts only after they are checked, cleaned and well presentable. When it comes to developing software what this approach boils down to developing a huge amount of code hidden from others. The disadvantages are clear: You get no feedback on whether or not there would be simpler/faster approaches to solving parts of your problem. Developers of the upstream project won't take the needs for this extension into account when rewriting current modules or adding new code. As a result general advise is to share and discuss your ideas before even starting to code.

Even for seasoned software engineers this approach tends to be a problem when not used to it: In a corporate environment feedback on code often is perceived as humiliating. As much as we preach that a developer isn't their code so feedback on coding issues is not to be taken seriously it still takes a lot of getting used to when suddenly people give that feedback - and give it in the open for everyone to see.

Also often decisions are taken privately before communicating publicly in corporations. As a result having heated design discussions tends to feel like a major marketing problem. Instead what is achieved is making the decision process publicly visible to others so they can follow the arguments without repeating them over and over. It also means that others can chime in and help out if they are affected (after all the second mantra is that those who do the work are the ones who ultimately take the decisions).

Again another reason could be for people new to the project or new to some role in the project that asking trivial questions looks intimidating. Again - even if you've contributed to a project for years noone expects you to know anything. If any piece isn't documented in sufficient depth asking the question provides a track record for others to later refer to. It also means you tend to get your answer faster because there are more people who can answer your question. Oh - and if you don't get an answer chances are, everyone else is in the dark as well but was just not asking.

One final word: One fear many people seem to share when asking is to make their own knowledge gaps and flaws public - not only to peers but also to current or future superiors. There's three parts to the advise that I tend to give: Everyone once started small. If your potential future manager or team mates don't understand this, than learning and making mistakes obviously aren't welcome in the environment that you are aspiring to join - is it really worth joining such an environment? Even so you start small, if you continue to contribute on a regular basis the amount of information that you produce online will grow. Guess how much time people spend on evaluating CVs - it's very unlikely they will read and follow each discussion that you contributed to. And as a third point - do assume that by default everything you do online will be visible and searchable forever. However chances are that ten or fifteen years from now what you are writing today will no longer be accessible: Services go away, projects and their infrastructure die, discussion media change - anyone still remember using NNTP on a regular basis?

As an exercise to the reader: Before writing this post I spend a couple of hours trying to dig up the mails I had sent I believe in 2000 to the I think KURT and RTLinux discussion lists in order to prepare some presentation about the state of realtime Linux back at university. I search the web, search the archives I could find, searched my own backups from back then - but couldn't find anything. If you do find some trace of that, I owe you a beer next time we meet at FOSDEM ;)

Scientific debugging - take 2

2014-01-28 20:13
Back in - OMG was that really back in 2010? - 2010 I wrote a post on scientific debugging. Today I was reminded of this post as I actually had the pleasure of watching this principle carried out - except this was for a medical "bug" instead of one in a piece of software.

To quote the book Why programs fail the method of scientific debugging consists of 5 easy to follow steps:
  1. Observe a failure (i.e., as described in the problem description).
  2. Invent a hypothesis as to the failure cause that is consistent with the observations.
  3. Use the hypothesis to make predictions.
  4. Test the hypothesis by experiments and further observations:
    • If the experiment satisfies the predictions, refine the hypothesis.
    • If the experiment does not satisfy the predictions, create alternate hypothesis.
  5. Repeat steps 3 and 4 until the hypothesis can no longer be refined.
In the past I've often seen developers jump immediately from a problem description in a user reported bug to implementing what looks like the obvious solution only to find out later that the actual problem underlying the problem description was something completely different.

I know that making sure one has found the true problem can take time. However today I was reminded a) just how long that time can be and b) just how important each step in that search can be:

So to start with the problem description (step 1 above): "On Saturday evening on my way into the bath tub some pretty bad pain hit me and forced me to lie down flat on the floor. After a few minutes the pain was gone, what remained until today's Tuesday is a pain in my back that gets worse when I walk or lie down on my left side. As a side-note: I'm expecting a baby - not sure if that has anything to do with the above."*

husband urged me to get to our general practitioner. So off I went - told him the story above. He couldn't do a whole lot for me but to him it seemed like the typical sciatic pain syndrome (step 2 above) - so off he sent me to the orthopaedic (he needed an expert's opinion to check his hypothesis apparently).

I called the one doctor I was recommended, was told to go there immediately, went there. Some 15 minutes later I told the doctor the story above (step 1). She followed the hypothesis - except for one tiny little problem: Given the location of the pain and my condition she knew of at least one more hypothesis that would also fit the observation (step 2). She made a manual check (step 3). Her observation was still consistent with the hypothesis (step 4). Now in order to reject said hypothesis she would have needed to do one more check - except she couldn't do that one herself. So she sent me off to my gynecologist to get an expert's opinion (and gave me an appointment for late this week/early next week in case really only the original hypothesis is valid).

Off I went, called the third doctor. I told her the whole story (step 1), she continued where the other doctor had left off and made the missing check (step 3) - and rejected the alternative hypotheses right away (step 4).

So off she went to step 5: Sometimes labour pains start in the back (step 2) - so she put me on a CTG (step 3) - after several minutes that hypothesis was rejected as well (step 4). So off she went to step 5:

The last non-trivial hypothesis would have been that something's wrong with kidneys (step 2). So she checked with her medical ultrasonics (step 3) - she wasn't 100% sure though it seemed to look ok so she send me off to one last expert for a second opinion.

So on my way home I got to tell the story above the fourth time (step 1). Again she continued with where the other doctor had left off - namely step 3: One more medical ultrasonic and a blood test later the last non trivial hypothesis was finally rejected altogether (step 4) So what was left was the initial hunch of there being some blockade in my back. Finally we can step from observing to acting, namely keeping warm (hot water bottle), relaxing and to keep moving regularly.

As annoying as the day for the system to be debugged was there's at least two valuable lessons learnt in it:

Being absolutely certain about a certain problem cause can be expensive (in this case in particular time consuming for me, rest is something the doctors involved have to discuss with my public health insurance). However jumping directly from problem description to bugfix can turn out to be much more expensive if said bugfix takes a long time to implement (and show effects) but is treating the wrong problem.

Proving one hypothesis right sometimes involves ruling out options that are easy to check but have bad consequences if left untreated. Translated back - before jumping from problem description to bugfix it may make sense to stop for a second and think whether the particular problem you see might actually be caused by a bug far worse than what you anticipated.

Now off to search for my warm little elephant.

* Yeah, that's why I didn't make it to the Berlin Open Source meetup on Sunday that I organised - at least my husband had fun together with Lennart and others trying to explain to the non German speaker what on earth is the English translation for the term Hexenschuß.

Children tinkering

2014-01-05 02:07
Years ago I decided that in case I got the same question for at least three times I would write down the answer and put it somewhere online in a more or less public location that I can link to. The latest question I got once too often came from daddies (mostly, sorry - not even a handful of moms around me, let alone moms who are into tech) looking for ways to get there children in touch with technology.

Of course every recommendation depends heavily on the age and interest of the little one in question. However most recommendations are based on using a child's love for games - see also a story on how a father accidentally turned his daughter into a dungeons and dragons fan for a bit more background on what I mean.

There are several obvious paths, including Lego Mindstorms, the programming kits by Fischertechnik, several electronics kits you get at your favourite shop, fun stuff like Makey, makey kits that can turn a banana into a controller. Also many games come with their own level designers (think Little Big Planet, though the older children might remember that even Far Cry, Doom and friends came with level designers).

In addition by now there are quite a few courses and hacking events that kids are invited to go to - like the FrogLabs co-located with FrosCon, the Chaos macht Schule initiative by CCC, meetups like the ones hosted by Open Tech School, Jugend Hackt. In addition quite a few universities collaborate with schools to bring pupils in touch with research (and oftentimes tech) - like e.g. at HU Berlin.

In addition there are a three more less typical recommendations:

  • As a child I loved programming a turtle (well, a white dot really) to move across the screen forward or backwards, to turn east, south, west or north, to paint or to stop painting. The slightly more advanced (both in a graphical as well as in an interactive sense of the word) version of that would be to go for Squeak (all smalltalk, first heard about it about a decade ago at the Chemnitzer Linuxtage) or Scratch (a geek dad kindly showed that to me years ago).
  • When it comes to hardware hacking one recommendation I can give from personal experience is to take part in one of the soldering courses by Mitch Altman - you know, the guy who invented the "TV-B-Gone". Really simple circuits, you solder yourself (no worries, the parts are large and robust enough that breaking them is really, really, really hard). What you end up with tends to be blinking and in some cases is programmable. As an aside: Those courses really aren't only interesting for little ones - I've seen adults attend, including people who are pretty deep into Java programming and barely ever touch circuits in their daily work.
  • If you are more into board games: Years ago one of my friends invited me to a RoboRally session. Essentially every player programs their little robot to move across the board.

When it comes to books one piece I can highly recommend (didn't know something like that existed until my colleagues came up with it) would be the book "Geek Mom" - there's also an edition called "Geek Dad". Be warned though, this is not tech only.

If you know of any other events, meetups, books or games that you think should really go on that list, let me know.

Don't dream it, be it

2013-12-24 12:07
After two years in a row of receiving 120 submissions for Berlin Buzzwords from the usual crowd - young, white, male, caucasian - only this year we decided we needed to work towards increasing diversity.One piece in the puzzle was to get in touch with several Berlin local "tech for non-tech" people groups. In a content exchange kind of setting I was asked to do an interview as some kind of role model.

In addition to a serious lack of time back then I felt the typical way these interviews go would do no good - even anyone who's in IT already learning I co-founded Apache Mahout, am a member of the Apache Software foundation, have co-founded Berlin Buzzwords (after running quite a few successful meetups around related topics in Berlin), am married to a Linux kernel developer tends to shy away (unless the person I'm talking to happens to be into OSS development themselves thus knowing that despite quite some work this also means having lots of fun).

However the invitation did get me started thinking about what kind of advise I would share with the next generation of hackers. Over time though I realised that what was most helpful for me doesn't only apply to those who want to become successful in IT. On first sight it sounds like an extremely easy to follow advise:

Once upon a time after coming back from the Kindergarden provided by my mom's employer I spent part of an afternoon in front of a computer in an office close the hers. The game was trivial: Direct a little Snake through a maze, collect items, avoid biting yourself or the walls.

Years in primary school I got to play with the computer of one of my relatives. Ever since beating their highscore I wanted a computer for myself. When I finally got a first computer on my own I used to play lots of games together with a good friend of mine - until the game supply for my Amiga 500 dried out. Back then I made a decision: To work towards simply coding my own games. Ever since I followed this tiny little dream - by now for almost twenty years.

Even despite the fact I got all support I could wish for from parents, teachers and university professors seen from the outside it may have seemed like not always being easy as pie: More often than not it meant being different - instead of being part of the "I don't know what I want to do after school" it meant being part of that small group of people who know what they are working for. Instead of being part of that large "I hate technology and I'm utterly bad at math" it meant being part of that tiny group of people who love math and who have fun dealing with any new technology.

Instead of being at one of those great parties for New Year's Eve it meant filling in the details of a project proposal a few hours before midnight. Instead of being home at 6p.m. it meant going to meetups more often than not. Instead of being home during weekends it meant flying to California for a conference on a weekend on my private budget. Despite getting 2.5 days a week from March to June to work on Berlin Buzzwords from my employer for the first two years and having lots of help and knowledge over at newthinking who did the heavy lifting of taking on the financial risk, managing registrations, booking the venue and handling speaker travel support it still meant lots of additional mornings, evenings and weekends spent on making the event fly (and an inbox that never went silent - neither at noon nor at midnight - hint: any mail you send to info@berlinbuzzwords ends up not only in an anonymous mailing list - every organiser including Simon Willnauer, Daniela Bentrup and myself will receive your mail and make sure it gets dealt with).

There are a couple of reasons I kept doing this kind of stuff. But I guess the most influential reason is simply that it also is a whole lot of fun for me.

There were several fellow students at school who didn't have the courage to follow their dreams from the very start: There's the girl who was teased into studying biology by her parents - only years later she had the courage to go for professional gardening. There's this guy who didn't know exactly what to do and followed many of his friends to study mechanical engineering. Months later he pivoted towards social sciences and politics, today working on a PhD. thesis on economics in German hospitals. There's the girl who successfully passed her math degree but really also wanted to follow her musical passion - in the end she went to study music in addition to become a math and music teacher.

It takes determination and courage to follow your dreams - especially if that means following a different path than what your parents had on their mind for you or following a path that doesn't quite fit into the cliché of what society has in mind for you*. However in my statistically absolutely non-significant, completely biased and personal opinion it's worth every effort.

* Sorry for as long as it is special to love repairing cars for a girl and to love working in child day care for a boy I don't believe in society not influencing career decisions.