Sunday, September 2, 2012

John Hunter 1968-2012

It was a shock to hear the news from Fernando that John Hunter needed chemo therapy to respond to the cancer that had attacked him.    Literally days previous to the news we had just been talking at the SciPy conference about how to take NumFOCUS to the next level.   Together with the other members of NumFocus we have ambitious plans for the Foundation: scholarships and post-doc funds for students and early professionals contributing to open-source, conference sponsorship, packaging and continuous integration sponsorships, etc.   We had been meeting via phone in board meetings every other week and he was planning to send a message to the matplotlib mailing list encouraging people to donate to our efforts with NumFOCUS.     Working with John in person on a mutual project was gratifying.   His intelligence, enthusiasm, humility, and pragmatism were a perfect complement to our board discussions.

He had also just spoken at SciPy 2012 and gave a great talk discussing his observations and lessons learned from Matplotlib.  If you haven't seen the talk, stop reading this and go watch it here --- you will see a great and humble man describe a labor of love (and not give himself enough credit for what he accomplished).

When I heard the news, I wrote a quick note to John expressing my support and appreciation for all he had done for Python --- not only because I truly feel that matplotlib is a major reason that projects I have invested so heavily in (NumPy and SciPy) have become so popular, but also because I knew that I had not shared enough with him how much I think of him.  A sinking feeling in my heart was telling me that I may not have much time.

This is what I sent him:
Hey John,

I am so sorry to hear the news of your diagnosis.    I will be praying for you and your family.   I understand if you cannot respond.   Please let me know if there is anything I can do to help.   

I have so much respect for you and what you have done to make Python viable as a language for technical computing.  I also just think you are an amazing human being with so much to give.  

All the best for a speedy recovery. 

This is the response I received.

Thanks so much Travis. We're moving full speed ahead with a treatment plan -- chemo may start Tues.  As unpleasant as it can be, I'm looking forward to the start of the fight against this bastard.

Thanks so much for your other kind words. You've always been a hero to me and they mean a lot. I have great respect for what you are doing for numpy and NUMFOCUS, and even though I am stepping back from work and MPL and everything non-essential right now, I want to continue supporting NF while I'm able.  
All the best,

I had no idea how much I would come to appreciate this small but meaningful exchange -- my last communication with John.  Only a few weeks later, Fernando Perez (author of IPython and a great friend to John) sent word that our mutual friend had an unexpected but terrible reaction to his initial treatment, and it had placed him in critical condition and the prognosis was not good.

I ached when literally hours later, John died.   I thought of his 3 daughters (each only about 3 years younger than my own 3 daughters) and how they would miss their father.   I thought of the time he did not spend with them because he was writing matplotlib.   I know exactly what that means because of the time I have sacrificed with my own little girls (and boys) bringing SciPy to life, merging Numarray and Numeric into NumPy, resurrecting llvmpy, and bringing Numba to life.   I thought of the future time I would not get to spend with him building NumFOCUS into a foundation worthy of the software it promotes.    I have not lost many of my loved ones to death yet.  Perhaps this is why I have been so affected by his death.  Not since my mother died 2 years ago (August 31, 2010), has the passing of another driven me so.

When I thought of John's girls, I thought immediately of what could we do to show love and appreciation.   What would I want for my own children if I were no longer here to care for them?   My oldest daughter had just started college and was experiencing that first transformative week.  Perhaps this was why I thought that more than anything if I were not around I would want my girls to have enough money for their education.  After speaking with Fernando and with approval from John's wife, Miriam, we setup the John Hunter Memorial Fund.  Anthony Scopatz, Leah Holdridge, and I have spent several hours since then making sure the site stays operational (mainly overcoming some unexpected difficulties caused by Google on Friday).

My personal goal is to raise at least $100,000 for John's girls.   This will not cover their entire education, but it is will be a good start and will be a symbolic expression of appreciation for all those who work tirelessly on open source software for the benefit of many.     After a few days we are at about $20,000 total (from about 450 donors).   This is a great start and will be greatly appreciated by John's family --- but I know that all those who benefit from the free use of a high-quality plotting library can do better than that.      If you have already given, thank you!    If you haven't given something yet, please consider what John has done for you personally, and give your most generous donation.  

There are fees associated with using online payment networks.    We will find a way to get those fees waived or covered by specific corporate donations, so don't let concern of the fees stop you from helping.    We've worked hard to make sure you have as many options to pay as possible.  You can use PayPal or WePay (which both have fees of 2.9% + $0.30), you can use an inexpensive payment network like Dwolla (only $0.25 for sending more than $10 and free for sending less --- but you have to have a Dwolla account and put money into it), or you can do as David Beazley suggested and just send a check to one of the addresses listed on the memorial page.

Whatever you decide to do, just remember that it is time to give back!

John has always been supportive of my work in open source.  It was his voice that was one of the few positive voices that kept me going in the early days of NumPy when other voices were more discouraging.    He has also consistently been a calming and supportive voice on the mailing lists when others have been less considerate and sometimes even hostile.    I'm very sorry he will not be able to see even more results of his tireless efforts.  I'm very sorry we won't get to feel more of his influence in the world.   The world has lost one who truly recognized that great things require cooperation of many people.   Obtaining that cooperation takes sacrifice, trust, humility, a willingness to listen, a willingness to speak out with respect, and a willingness to forgive.   He exemplified those characteristics.   I am truly saddened that I will not be able to learn more from him.

When SciPy was emerging from my collection of modules in 2001, one of the things Eric Jones and I wanted was an integrated plotting package.    We spent time on a couple of plotting tools in early SciPy (a simple WX plotting widget, xplot based on Yorick's gist).    These early steps were not going to get us what users needed.  Fortunately, John Hunter came along around 2001 and started a new project called Matplotlib which steadily grew in popularity until it literally exploded in about 2004 with funding from the Perry Greenfield and the Space Science Telescope Institute and the efforts of the current principal developer of Matplotlib: Michael Droettboom.

I learned from John's project many important things about open source development.   A few of them:

  • Examples, documentation, and ease of use matter -- a lot
  • Large efforts like Python for Science need a lot of people and a distributed, independent development environment (not everything belongs in a single namespace).
    • SciPy needed to be a modular "library" not a replacement for Matlab all by itself. 
    • The community needed a unifying installation to make it easy for the end-user to get everything, but we did not need a single namespace. 
    • Open source projects can only cover as much space as a team of about 5-7 active developers can understand.   Then, they need to be organized into a larger integration and distribution projects --- a hierarchical federation of projects. 
    • The only way large projects can survive is by separating concerns, having well defined interfaces, and groups that work on individual pieces they have expertise in. 
  • Backwards compatibility matters a great deal to an open source project (he created numerix for Matplotlib to facilitate for end-users the migration of Numeric through Numarray to NumPy in Matplotlib)
I'm sure if John were here, he could improve my rough outline and make it much better.   From improving plotting libraries to making useful use of record arrays, he was always doing that.   In fact, one of John's last contributions to the world is in improving the mission statement of NumFOCUS.    In a recent board meeting, he suggested the word "accessible" to the mission statement:  The purpose of NumFOCUS is to promote the use of accessible and reproducible computing in science and technology.  

His life's work has indeed been to make science and technology computing more accessible through making Python the de facto standard for doing science with his excellent plotting tool.  Let's continue to improve the legacy he has left us by working together to make computing even more accessible.  We have a long way to go, but by standing on the shoulders of giants like John we can see just that much farther and continue the journey.  

Besides helping his daughters there is nothing more fitting that we can do to honor John's memory than continuing to promote the other work he spent so many hours of his life pushing by contributing to open source projects and/or supporting financially the foundation he wanted to see successful.  

Great people lift us both in life and death.   In life they are gracious contributors to our well being and encourage us to grow.  In death they cause us to reflect on the precious qualities they reflected.  They make us want to improve.  When we think of them, we want to hold our children close, give an encouraging word to a colleague, feel gratitude for our friends and family, and forgive someone who has hurt us.  John Hunter (1968 - 2012) was truly a great man!