Video: Sebastian Schelter on Recommendation w/ Apache Mahout

2010-10-21 13:55
A few weeks ago we had the autumn edition of the Apache Hadoop Get Together in newthinking store in Berlin. I am glad to announce the first video online:

Mahout Sebastian Schelter from Isabel Drost on Vimeo.



Thanks to JTeam for sponsoring video taping, thanks to newthinking for providing the location and thanks to Martin Schmidt from newthinking for producing the video.

Stay tuned for the second video to be published next week.

Apache Mahout at Apache Con NA

2010-10-15 20:39
The upcoming Apache Con NA to take place in Atlanta will feature several tracks relevant to users of Apache Mahout, Lucene and Hadoop: There will be a full track on Hadoop as well as one on NoSQL on Wednesday featuring talks on the framework itself, Pig and Hive as well as presentations from users on special use cases and on their way of getting the system to production.

The track on Mahout, Lucene and friends starts on Thursday afternoon, followed by another series of Lucene presentations on Friday.



Also don't miss the track on the community and business tracks for a glimpse behind the scenes. Of course there will also be tracks on well-known Apache Tomcat, httpd, OSGi and many, many more.

Looking forward to meeting you in Atlanta!

Reminder: Apache Hadoop Get Together Berlin Today

2010-10-07 09:52
Just a brief reminder: The Apache Hadoop Get Together Berlin is supposed to take place today in newthinking store, Tucholskystr. 48 at 5p.m.

The meeting features two talks on Apache Mahout: Committer Sebastian Schelter will explain how to scale recommender systems with Mahout. Contributor Max Heimel is going to give an introduction to the sequence labeling facilities available in Mahout.

As usual the group will move over to Cafe Aufsturz after the meetup is over.

A big Thanks goes to JTeam for sponsoring video taping as well as to newthinking store for providing the venue for free.

Apache Hadoop Get Together Berlin - October 2010

2010-09-15 07:31
This is to announce the next Apache Hadoop Get Together sponsored by JTeam that will take place in newthinking store in Berlin.




When:
October 7th, 5p.m.
Where:
Newthinking store Berlin, Tucholksystr. 48




As always there will be slots of 30min each for talks on your Hadoop topic. After each talk there will be a lot time to discuss. You can order drinks directly at the bar in the newthinking store. If you like, you can order pizza. We will go to Cafe Aufsturz after the event for some beer and something to eat.

Talks scheduled so far:

Max Heimel: "Hidden Markov Models for Apache Mahout"

Abstract: In this talk I will present and discuss an implementation of a powerful statistical tool called Hidden Markov Models for the Apache Mahout project. Hidden Markov models allow to mathematically deduce the structure of an underlying - and unobservable - process based on the structure of the produced data. Hidden Markov Models are thus frequently applied in pattern recognition to deduce structures that are not directly observable. Examples for applications of Hidden Markov Models include the recognition of syllables in speech recordings, handwritten letter recognition and part-of-speech tagging.

Sebastian Schelter: Distributed Itembased Collaborative Filtering with Apache Mahout"

Abstract: Recommendation Mining helps users find items they like. A very popular way to implement this is by using Collaborative Filtering. This talk will give an introduction to an approach called Itembased Collaborative Filtering and explain Mahout's Map/Reduce based implementation of it.


View Larger Map



Please do indicate on Upcoming if you are coming so we can more safely plan capacities.

JTeam is looking for Java developers and search enthusiasts. Check out their jobs page for more info!

As always a big Thank You goes to newthinking store for providing the venue for free for our event.

Looking forward to seeing you in Berlin as well,
Isabel

Apache Hadoop in Debian Squeeze

2010-07-17 12:04
After using Mandrake for quite a while (still blaming my boyfriend Thilo for infecting not only my computer but also myself first with that system, then with the more general idea of Free Software - but that's another story.) after finishing my master's thesis I started using GNU Debian Linux (back then in the version code-named Woody). Since I always had a GNU Debian on my private box as my main operating system - even installed it on my MacBook following the steps in the Debian Wiki.

As I am also an Apache Mahout committer, closely related to the Apache Hadoop project, I always found it kind of sad that there were no Hadoop packages in the official Debian repositories. I tried multiple times to find some time to get into Debian packaging myself, I learned what "debian/rules" is all about and discovered some of the intricacies of packaging Java based software. However I have to admit that I never was able to find enough time to really finish that task.

A few weeks before this year's FOSDEM I learned on the Apache Hadoop as well as on the Debian Java lists that a guy called Thomas Koch was working on solving bug 535861 - ITP to package Hadoop. We met at FOSDEM where I tried to raise some attention in the audience for Thomas' plans (back then he was in need for help with a few last missing pieces). In addition I invited him for Berlin Buzzwords to get in touch with other Hadoop developers and users for further input.

I am really happy that by now Hadoop has made it into the official Debian package repositories - as soon as Debian Squeezeapt-get install [Hadoop component you need]: Debian package search.

Squeeze

If you want to speed up the process of Squeeze being released as stable version: Help fixing the remaining bugs in that distribution. There are various Debian Bug Squashing Parties being organised around the world. Next one in Berlin is on next Monday, the one for Munich is running this weekend. Just got the information that Fefe posted in his blog a link to the Mozilla bug bounty:

The packages are based on the upstream Apache Hadoop distribution, being comparably new they are intended for development machines at the moment. If you are using Debian and want to work with Hadoop - this is a great opportunity to help making the packages more stable by simply using them and reporting your experiences back to the Debian community.

In addition Debian now also provides packages for Zookeeper as well as HBase - though the HBase version is not yet production ready as the HDFS-append patch is still missing.

To follow the general state and progress of these packages feel free to follow the packages pages for Hadoop, HBase, Zookeeper respectively.

Thomas currently plans to work more closely with upstream e.g. to tidy up the chaos in the start-up scripts and other minor glitches. So watch out for further improvements.

In addition I just saw another interesting ITP in the Debian bugtracker: Wishlist: katta. I am sure there are quite a few others as well.

My highly subjective Berlin Buzzwords recap

2010-06-13 18:32
Last November I innocently asked Grant what it would take to make him to give a talk in Berlin. The only requirement he told me was that I'd have to pay for his flight. About eight months later we had Berlin Buzzwords - a conference all around the topics scalability, data storage and search. With Simon Willnauer, Uwe Schindler, Michael Busch, Robert Muir, Grant Ingersoll, Andrzej Bialecki and many others we had quite a few Lucene people in town.







From the NoSQL community, Peter Neubauer, Rusty Klophaus, Jan Lehnardt, Mathias Meyer, Eric Evans and many others made sure people got their fair share of NoSQL knowledge. With Aaron Kimball, Jay Booth, Doug Judd and Steve Loughran we had several Hadoop and related people at the conference.

The conference also featured two talks on Apache Mahout: An overview from Frank Scholten as well as a more in-depth talk by Sean Owen. It's great to see the project grow - not only in terms of development community but also in terms of requests from professional Mahout users.









In addition we had a keynote by Pieter Hintjens that concentrated on messaging in general and 0MQ in particular - a scalability topic otherwise highly underrepresented at Berlin Buzzwords.







We got well over 300 attendees that filled Berlin Kosmos - a former cinema. Attendees were a good mixture of Apache and non-Apache people, developers and users. People used the breaks and bar tours after the event to get in touch, exchange ideas. It's always good to see developers discuss design issues and architectural challenges.

Monday evening was reserved for local people taking out the speakers and interested attendees for Bar Tours to Friedrichshain. Those from Berlin took Berlin Buzzwords people to their favourite restaurants and bars - or to what they considered to be "typical Berlin". Some spent evenings later that week drinking beer or Berliner Weisse.




The tour for keynote speakers Grant Ingersoll, Pieter Hintjens and friends was organised by Julia and myself. We went over to Kreuzberg - some went to famous Burgermeister for Burgers, the other half went to a nearby Indian restaurant. After that we spent the evening in Club der Visionäre - a club next to the water. Me personally I left at about midnight - several people of the Lucene community moved to the well known Fette Ecke later on.

When asking the audience about repeating the conference next year, all hands went up immediately. Beside lots of praise for the organisation, from the feedback form we put up we got some good ideas on how to improve the conference next year. I'd love to have you guys back here in 2011 - and I'd love to get even more attendees in. Was great fun having you here. Thanks for 5 great days:

Five instead of two days, because:

  • Keynote speakers got a special treatment - that is a personal city guide for the weekend before Buzzwords.
  • We had the official conference start on Sunday with a Barcamp.
  • We had another Apache dinner on Wednesday with those Apache people that live in Berlin. In addition the Aaron and Sarah joined us as they were still in town for the Apache Hadoop trainings. Also Greg Stein had pizza and beer with us - he was in town for the svn conference at the end of the week.












Thanks to all who helped turn this conference into a success: Julia Gemählich for conference management, Ulf and Wetter for WiFi setup, Nils for travel management, Simon and Jan for support ranking talks and reaching out to your communities, all speakers for fantastic talks, those taking pictures of the conference and sharing them on Flickr for showing those who stayed at home how great the conference was, peoplezapping for the videos that will soon be available online, all sponsors for supporting the conference, all attendees for their participation. I'd love to have all of you (and many more) back in Berlin next year. An informal call for presentations has been set up already - submit now and be the one to set the trend instead of just following the Buzzwords!

For those who do not want to wait for another year: We will have another Apache Hadoop Get Together in September 2010 - watch this space for more information. If you'd like to give a talk their and present your Hadoop/ Solr/ Lucene etc. system - please get in touch with me.

Apache Dinner June 2010

2010-06-09 23:54
After Berlin Buzzwords was over yesterday - and as there is an svn conference in the city that starts tomorrow, we thought we could easily put together a smallish Apache Dinner for tonight. So Torsten mailed a few people, booked some space at Heinz Minki in Kreuzberg. We announced it at the end of the conference and invited people to join us.






So after a beautiful day out I spent the evening with a bunch of Apache related guys and girls having drinks and great pizza: We had several svn committers, Greg Stein met us there - he arrived today for the svn conference. Of couse the usual suspects were there as well: Simon Willnauer and his wife, Torsten Curdt, Erik Abele, Thomas Wöhlke, Valerie Hajdik. In addition we had guests from the Apache Lucene and Apache Hadoop communities: Sarah Sproehnle and Aaron Kimball as well as Uwe Schindler joined us.







Thanks to Torsten for putting the meetup together. See you next time in July. If you are interested in joining our meetups: Subscribe to our mailing list.

Scaling user groups

2010-05-26 19:32
A few hours ago, Jan Lehnardt posted a link on How to organise a nerd conference - joking that this is how we planned Berlin Buzzwords. Well, it is not exactly that easy - however the comic actually is not so far from the truth either:

About two years ago, after having started Apache Mahout together with Grant Ingersoll, Karl Wettin and others, several Apache Hadoop user groups, meetups and get togethers started to pop up all around the world. The one closest to me was the Hadoop user group UK. Back in 2008 I was pretty envious to all these user groups - being so distributed, there was no way I could ever attend all of them, though talks were certainly interesting. So the naive thought of a back then naive free software developer was: Let's have that in Berlin. To have initial talks I called Stefan Groschupf. His answer was very positive: Oh yeah, let's do this. I am in Germany for another two weeks, so it should be at about that timeframe. We agreed that if no-one showed up, we could still have some pizza together and share insights from our projects.

For the venue I knew from regular meetups of the Free Software Foundation Europe - read FSF*E* - that newthinking store was available for free for meetups for devs of free software. On I went, calling Martin from the store, booked the room. After that some mails went to the usual suspects, mailing lists and such. At the first meetup two years ago, more than 15 attendees - with two more people who had prepared slides. Pizzas obviously had to wait a little.

If you are wondering what that looked like back then - Thanks to Martin for taking the image back then and putting it online.






We (as in all attendees) decided to repeat the exercise three months later*, talks for the next time were proposed during that first session. Noone objected to having it in Berlin again - everyone knew this was the only way to avoid having to do the organization next time.

The meetup grew steadily in size, talks started being proposed three to six months in advance. I ended up creating not only a mailing list for the meetup but also a blog so I could publish news on Jan's CouchDB talk and Lars George's HBase talk back then. We got video sponsoring from Cloudera (Thanks Christophe), StudiVZ (Thanks Nils), and Nokia (Thanks Matt). Late last year I did the first European NoSQL meetup together with Jan Lehnardt - 80 attendees, lots of potential for more, the newthinking store obviously a bit too small for that :)

If you are wondering what NoSQL and Hadoop meetups looked like last time:



During that meetup the idea was born for a larger NoSQL conference in Berlin in 2010. First ideas were tossed around together with Jan and Simon Willnauer during Apache Con US in Oakland. The topic Hadoop got added there. In January 2010 finally Lucene was added to the mix. We contacted newthinking for support - got a very warm welcome.

Now - two years after the first Apache Hadoop Get Together Berlin we are proud to host Berlin Buzzwords - focussed on NoSQL, Apache Hadoop and search as in Apache Lucene.The conference is co-organised by newthinking communications, Simon Willnauer, Jan Lehnardt and myself. A big thanks to neofonie for supporting me by making it possible that I could do most of the organisation during my regular working hours.

The speaker lineup looks fantastic. Registration is going very well - exceeding expectations (did I mention that registration is still open, group and student tickets still available?).

I am really looking forward to an amazing conference on 7th and 8th of June. We will have a NoSQL barcamp in newthinking store Sunday evening before the conference. Keynote speaker packages have been sent out and were well received. Hotel rooms for speakers are booked. We are about to pull together the last loose ends in the coming days. Happy to have so many guys (and a few girls) interested in scalability topics here in town at the beginning of June. Looking forward to seeing you in Berlin.



* The second meetup turned out to be the first and so far only one that took place w/o the organiser - I broke my leg on my way to newthinking by getting hit by a BMW X5... *sigh* Note for other meetup organizers: Always have a backup moderator - in may case that was my neofonie manager Holger Düwiger who happened to attend that meetup for the first time back then.

Going to Berlin Buzzwords

2010-05-21 08:46
Meet me in two weeks at Berlin Buzzwords. As you may have noticed, together with Simon Willnauer, Jan Lehnardt and newthinking communications I am organising Berlin Buzzwords - a conference on scalable search, data analysis and storage.

I'm going to Berlin Buzzwords - the conference on searching, processing and storing data.

There are a few regular tickets left, so don't wait too long to register. If you want to bring your friends, check out our group tickets with up to 50% discount. If you are a student bring your student ID and register for 100,- Euro.

Berlin Buzzwords 2010 - Scalability conference June 7th/ 8th in Berlin

2010-05-14 15:48
The Berlin Buzzwords schedule was published a few days ago. There are tracks specific to the three tags search, store and scale. We have a fantastic mixture of developers and users of open source software projects that make scaling data processing today possible.

There is Steve Loughran, Aaron Kimball and Stefan Groschupf from the Apache Hadoop community. We have Grant Ingersoll, Robert Muir and the "Generics Policeman" Uwe Schindler from the Lucene community.

For those interested in NoSQL databases there is Mathias Stearn from MongoDB, Jan Lehnardt from CouchDB and Eric Evans, the guy who coined the term NoSQL one year ago.

The schedule has been published online. Visit the webpage and register for the conference - looking forward to seeing you in Berlin this summer!

Regular tickets are available online. In addition we offer student tickets: If you have a valid student ID, you are eligible for one of these tickets. Each costs 100,- Euro. We also have a special group ticket available: If you buy four tickets or more you are eligible for a discount of 25%, when purchasing 10 tickets or more the discount is 50%. Learn more at http://berlinbuzzwords.de/content/tickets

One day before the conference we are having a Berlin Buzzwords Barcamp in town. In addition, directly after the conference, Cloudera will be hosting Apache Hadoop trainings - registration is separate from Berlin Buzzwords.

So just in case you need a good excuse for a long term trip to Berlin: You can spend the weekend in town, attend the Barcamp on Sunday evening, visit Berlin Buzzwords on Monday and Tuesday. The rest of the week could be used to take part in Apache Hadoop trainings. Finally you have one weekend left for a city tour.

Thanks to Jan Lehnardt, Simon Willnauer and newthinking communications for co-organising the event.