19 Sep 2017

Why I'm Writing About ML (ML Series, Part 1)

At the heart of science is an essential balance between two seemingly contradictory attitudes - an openness to new ideas, no matter how bizarre or counterintuitive, and the most ruthlessly skeptical scrutiny of all ideas, old and new.

• Carl Sagan, The Demon-Haunted World

I know of no better tool than writing to explore my own understanding of technical subjects. After years of reading about machine learning in the press, programmer blogs and from technology leaders of our time, I want to investigate ML on my own terms. This investigation may take months or years or, based on how quickly the field is changing, forever - I’m not sure. What I can say is that my goal in writing a multi-part series is no less than what Sagan describes in the above quote. I intend to be both open to extraordinary claims about machine learning and simultaneously skeptical.

The value I believe I can add in writing this series is in infusing each post with as much illustrative code as possible. I want to explore not only programming libraries, but the cloud services that allow library code to be run with GPU enabled computing power. While I have been extraordinarily influenced by previous explorations of ML like Tim Urban’s AI Revolution Series, these posts are going to be technical and experiment heavy.

I want to answer for myself not only why machine learning works, but also the various hows. I will try to approach as much as I can from first principles in answering why machine learning is effective. Where ML relies on ideas from linear algebra and calculus, I promise to explore these fields as well. On the subject of how machine learning works today, I want to start first with as few libraries as possible, and then move slowly into popular libraries like TensorFlow.

What these posts will not do is engage in futurism discussions or projections of what machine learning could do. I am going to stay focused on what machine learning can do at present. Any demonstrations of machine learning capabilities I’ll post on GitHub. As someone with no formal ML background, I encourage anyone (un)lucky enough to be reading these posts to raise issues and provide guidance on what I could be doing better.

Along the way, I’ll be diving into the following resources. I’ll add to this list as this series takes shape:

• Andrew Ng’s deep neural network Coursera specialization
• Sebastian Raschka’s Python Machine Learning

Finally, I want to manage expectations about how often I’ll be posting in this series and post lengths. Deep Work is one of my favorite books, and in the spirit of Cal Newport’s suggestion to budget deep work hours to projects, I intend to devote 3-5 hours of deep work per week to this project. In terms of length, these posts might be short or long depending on the idea being explored. I have set a short-term goal of posting once per week for the next 5 weeks; after that, I’ll evaluate the quality of those posts and make improvements for the next ones.

I’m excited to get started. If these posts can change my own opinion on machine learning, than they were worth doing. If anyone else reads them, it’s an added bonus.

Here’s to openness and ruthless skepticism.