Overview of this blog

Welcome everyone!  I'm Daniel Svoboda, M.Eng and I hope to enlighten you by presenting a thorough background of the subject Machine Learning along with contemporary applications and new discoveries.  If you are looking for an exciting scientific field, you've come to the right place.  While this subject is decades old (technically since the 1940's when the beginnings of artificial intelligence was hammered out), it's only recently that new discoveries in mathematics (in optimization for example), neuroscience, etc. have created a hot new field that allow for predicting reliable outputs from given inputs.  As someone who loves psychology as well mathematics, this has been the perfect field for me.  Even the names of methods used will sound sexy to the scientific mind:  SVM, Neural Networks, Gaussian Mixture Models, Ensemble Methods.

Before I go further, a little about me.  I graduated with my bachelors in electrical engineering in 2005 from Stevens Institute of Technology (in Hoboken, yes the Hoboken, NJ).  I had initially studied signal processing, mainly DSP because of the beautiful mathematics used.  However, going into my career, I found out that its use, mainly in radios and electronics was not to my taste.  I'm primarily a mathematician first and a software engineer second (though these can flip based on what I'm studying at the moment).  I initially went to study my masters at Stevens in DSP.  It was here that thanks to my advisor I was exposed to the subject Machine Learning (as part of my studies for DSP).  Going into this class in 2008, it was love at first sight.  I normally love any mathematically oriented subject, but this was more like an explosion of delight rather than the warm fuzz I get in math classes.  Everything about the subject just appealed to me.  As someone who likes to read history and psychology as a hobby, I soon realized that this subject used many elements of these, mainly in neuroscience (another up and coming hot field).

After taking the course, I was motivated to change my masters to Computer Engineering (my initial bachelors major before having an incomprehensible professor teach Data Structures and Algorithms that made me disgusted enough to switch to Electrical Engineering.) and concentrate in Intelligent Systems.  After some more classes in Pattern Recognition and Control Theory, I graduated with my masters last month.  During this time, I've been attempting to find blogs that explain the basics of what I've learned, but have been unsuccessful given my options were either online lecture notes or blogs written for people for specialists in the subject.

During this time, I also have been reading Mark Chu-Carroll's blog Good Math, Bad Math.  While I love mathematics, many blogs seem to have been written only for the specialist.  Good Math, Bad Math was a big wind of relief as it not only talked about the whole breadth of mathematics, but in a simple, clear style for the layman or budding mathematician.  If you read mathematical papers from the 19th century, one can be struck by how simple and clear the prose is.  The subject was written more as a philosophy than a technical manual.  Now granted, mathematics is in need of more complex symbols and formulas than was used in the 19th century, but still, I believe a system of greater clarification is needed when writing a math textbook or paper.  This blog aims to explain the principles of machine learning for the curious layman.

Any professionals on the other hand who read this blog may be struck by the amateurish route I take on some of the explanations I have.  Well I don't claim to be an expert.  I don't have a Ph.D (yet!), only what I learned from class, the workplace, and my own interest.  I also have an incomplete education on some of the languages used by professionals, some including R, Python, Ruby, etc.  My hope is that writing this blog can help benefit the regular layman as well as a seasoned Ph.D looking for a refresher or new insight into the subject.  I also hope to benefit myself in this endeavor.  I have always had a knack for teaching and explaining complex subjects in simple terms that are understandable.  I also like many others, been able to comprehend a subject better when I teach it someone rather than wracking my brain on decoding it.  So during the course of this blog, in addition to the subjects, I will keep you posted on my progress learning the latest languages and databases so that not only will I benefit, but you also should you be interested in learning these languages.

Finally, one should know how I plan to organize these posts.  I always found that diving into the subject, even with unfamiliar concepts is easier than learning the concepts first, then the subject.  So I plan to give an overview of all methods of machine learning first before diving into secondary tangents.  For instance, to understand Naive Bayes Classifiers, you need to understand some concepts in probability theory.  I plan to give a brief overview of the probability concepts as they relate to the overall concept.  Once all machine learning algorithms have been covered, I will create a post explaining the basics of probability theory.  Machine learning requires knowledge in linear algebra, statistics/probability, and optimization, so the basics of these subjects will be covered afterwards.  After that, I plan to do some digging and look at recent developments in the theory along with applications and useful technologies in the software world that implement them.  One example would be random forests, a contemporary application of decision trees.  Obviously decision trees (and ensemble methods) need to be learned before we can get into this area.

Many writers, even professional mathematicians believe that when writing for the layman, mathematical formulas need to be kept at a minimum.  I heartily disapprove of this.  I believe that employing the right, detailed explanation along with practice and repetition can make a mathematical concept comprehensible.  I hope not to go off on too many tangents, but still I will give detailed explanations on the simplest things.  I also would look forward to any feedback on how to improve my style so as to be more understandable in the future.  Many professional scientists and engineers use a program called LaTex to write stylistic papers that show mathematical formulas clearly.  As I will be learning this in the progression of the blog, I will have a few blog posts on using this beautiful language so other interested people can see how the formulas are presented in each post.