I attended ZFS: Bringing Terabytes Under Control by Jay Edwards. He mainly talked about using ZFS on Solaris for large storage systems. The biggest advantage of ZFS is the nice management tools for creating pools and filesystems. RAIDZ, which is RAID5 with tracking of which blocks need to be rebuilt, sounded neat. The copy-on-write behavior which makes snapshots and clones easy sounded very useful. He mentioned some of the problems between the impedance mismatch between smart ZFS and smart hardware. ZFS assumes that the hardware is dumb and lies which causes performance problems with smart drive arrays.

Next, I went to Running Your Programming Language on Parrot by chromiat and Patrick Michaud. They talked about how to write parsers and compilers for Parrot using the Parrot Compiler Tools. The tools make it easy to write a Parrot compiler for a language by doing a parse grammar and AST grammar to transform the source into an AST; the tools can produce PIR from the AST. It sounded like the new tools are being used for more languages and should improve the progress in getting languages implemented on Parrot.

I skipped the next session talk and worked on my computer instead. After lunch, I went to Next Generation Version Control Systems by Sam Villian. He talked about bzr, Mercurial, and git, the three modern distributed version control systems. He talked about why distributed version control is better than centralized ones like Subversion. I liked is distributed systems are more transactional with commits not relying on a server. Also, these distributed systems are more descriptive and keep track of the DAG of the history instead of having to be told about merges and renames. He also talked about how the distributed systems allow the stable development model with each change getting a branch that is merged into an always-stable branch when ready. In comparing the systems, the main thing is that bzr is slow and doesn’t scale as well to large repositories. git is harder to use and impossible to install on Windows. It is really fast and produces compact repositories. Mercurial sounded nice with a speed comparable to git while being easier to install and use.

Next, I went to Jesse Vincent’s talk on Building Domain Specific Languages in Perl. He was talking about so-called internal DSLs where an easy-o=use dialect of Perl is used for a specific domain. He mentioned a couple of tricks to persuade the Perl parser to support a simplified syntax including installing helper functions, using prototypes, and returning closures. He mentioned a couple of DSLs that they had invented for Jifty: Jifty::DBI, Object::Declare, Template::Declare, and Jifty::Dispatcher. Template::Declare was interesting since it is a templating system that instead of embedding code in text or using a special language just uses special Perl code with functions for HTML elements. This allows refactoring and modularizing the templates like code.

Finally, I went to the Perl Lightning Talks, a bunch of five minute long talks. They are just long enough to advertise a project, perform a rant, give a brief summary of a longer talk, or sing a song. I remember some of the same talks from last year. Some of the projects I bookmarked where Yahoo UI Library, qpsmptd, Move My Data, DashProfiler, DBD::Grofer, and Perl 5 Wiki.

I also went to Performance Whack-a-Mole by Josh Berkus. He talked about how to solve database performance problems. He gave a couple of rules to tracking the problems which he called moles. Most problems are not really database problems and are at other levels of the stack. Most slowdown is caused by a few big problems. You can only see the worst problem which hides the others. Different classes of apps have different performance problems. Web apps tend to read heavy and have issues with connections and caching. OLTP have locks, transactions, IO write problems. Data warehousing has IO issues with bad queries. He also talked about doing a baseline inspection for common problems, benchmarks, microbenchmarks, and profiling.

I also went to Care and Feeding of Large Web Applications by Perre Harkins. He talked about the issues dealing with deploying and releasing a large web application in Perl. I was most interested in the system he described of installing CPAN modiules. They have their own tool for deploying specific versions of modules. He also mentioned automating database updates, simplying the configuration, testing, and doing version control and branches.

I don’t expect maintainers to drop everything and fix the bug. I would hope that they would at least send an acknowledgement that they have seen the bug and might work on it. A snarky “that is not a bug but a feature”, “it has been fixed in the latest release”, or “you are an idiot, do this instead” is better than silence.

If you post a module to CPAN, you have an obligation to maintain it. I suspect fixing bugs is more beneficial to the community than writing new, cool stuff. Even if it is really boring. The user has an obligation to report bugs if they encounter them so the maintainer is not in the dark. One big advantage of open source is that the user can provide a fix or at least a test case. The maintainer’s responsiblity is to apply the patch if it is suitable. Not let it sit there gathering dust.

The best example is bug 27, a major limitation in URI which is only 5 years old. People have even provided patches.

Also posted at use perl.

I actually used svk for the spec files for the packages. It was very handy for synchronizing and merging changes between home and work using the repository at my ISP. My repository doesn’t allow public changes but you can work the same way with any Subversion repository.

Setup the mirror:

svk mirror http://znark.com/svn/repos //mirror/znark

Sync the mirror to the svk depot.

svk sync //mirror/znark

Copy the mirror to local directory

svk copy //mirror/znark/cpanbuild //cpanbuild

Checkout the local directory

svk co //cpanbuild
cd cpanbuild

Make some changes and commit:

svk ci

Push the changes to the repository:

svk push

At another location, or after somebody else has made changes, I can do:

svk sync //mirror/znark
svk pull

He went quickly through the difference between modperl 1 and modperl 2 but he demonstrated a lot of the new modperl 2 API. He went through most of the API, all the different stages and handlers that modperl can hook into.

Some of the things he mentioned, like protocol handlers and filters can only be done with Apache 2 and modperl 2. I suspect it would be possible to write an FTP, SMTP, or XMPP server in Perl using modperl 2. I know people are doing things like that in C modules but I think it would be much easier to do in Perl. The other interesting idea I had was to combine POE with mod_perl and use POE to help implement the protocol handlers.

Hopefully, now that modperl 2 is officially out that lots of systems will get ported. I was hoping to hear how the Mason modperl 2 support was coming but Dave Rolsky left early.

The one thing missing from Apache 2 is some way to separate the Apache children by virtual host and run them as different users. This would make modperl much more acceptable for ISPs which could isolate different users but still give them the performance and flexibility of modperl. One big advantage of PHP is that it is easy to support shared hosting.

Shtoom

First, I went to talk by Anthony Baxter about shtoom, a SIP stack written in Python. As he explained multiple times, VOIP and SIP are complicated. Which makes writing a complete stack by himself even more amazing. It also shows that Python and other interpreted languages are good enough (and much easier) to write demanding applications.

I definitely agree with his goal of having open source, interoperable VOIP so we can talk to anyone and do what we want. On one hand, we have Skype which is free but proprietary. On the other, we have the hardware phones which use SIP but can be locked down to certain providers and have proprietary extensions.

I saw Guido van Rossum, the creator of Python, at the talk.

Big Databases

The next session I went to was by Josh Berkus and Joe Conway talking about large data warehouses in PostgreSQL. This is exactly what we do at work so the casen studies were quite useful.

The main thing I learned is that we should upgrade to Linux 2.6 and PostgreSQL 8.0 for x86_64. It sounds like there is a big performance advantage.

I am really looking forward to PostgreSQL 8.1 which is going to have advances in table partitioning.

I was hoping for more useful stuff out of the talks I went to after lunch. Especially since I learned later that there were other more interesting talks going on at the same time.

Subversion

I went to “From CVS to SVN” by Brian Fitzpatrick. He mainly talked about the general issues of managing the migration. He talked about the experiences of some projects, open source and commercial, in doing the migration. I was hoping for more details about the technical aspects. It did inspire me to move forward on getting my workplace to do the migration. And what political issues to work on.

Perl & Unicode

The next talk was Dan Kogai talking about Perl 5.8 and Unicode. Unfortunately, it was a pretty boring talk. It covered material which I had already gleaned from the documentation. He did show some examples including actual code. At work, we have been suffering doing Unicode with Perl 5.6. Moving to Perl 5.8 sounds like the right solution.

MapReduce

Then I went to a talk by Doug Cutting talking about doing Scalable Computing with MapReduce. The Nutch project, is a search engine based on Lucene, written in Java. They swiped the Gogogle File System and Map Reduce processing ideas to experiment with scalable indexing. I originally though the presentation was going to be by someone from Google. Seeing that an open source project had implemented Google’s ideas was just as interesting. I was hoping to see more info on MapReduce but I guess I will have to read the Google papers.

Rails History

Finally, I went to a talk by the creator of Ruby on Rails talking about Extracting Rails from Basecamp. It was mostly about the history of Rails and lessons learned about promoting and growing a project. I was hoping for more technical details and a little humility about his limited experience. Rails is exploding right now and it was useful to hear how he developed it as his first Ruby and open source project.