Feeds:
Posts
Comments

Posts Tagged ‘ocw’

Sometime early this week , I finished listening to the excellent video lectures from MIT OCW’s Calculus Revisited course . I had been meaning to listen to MIT OCW’s Single Variable Calculus course for quite sometime as my background in Calculus is a bit flaky. My interests are in machine learning, data mining and AI where Calculus has a nasty habit of making surprise entries 🙂

I somehow finished my Master’s using my old Calculus knowledge. I took a course on Numerical Methods which kind of exposed my weaknesses. I kept getting confused with the error approximations which used ideas from  infinite series. Other advanced ideas like multivariable optimization were also problematic to me. Once that course was over, I swore myself to refresh my Calculus stuff and also learn multivariable calculus.

I started listening to MIT OCW’s Single Variable Calculus lecture videos and felt two things – The course was a bit slow for my pace and the course jumped right away into the mechanics without spending much time on the intuitive explanations of the Calculus. In other words, I felt 18.01 was more focused on the analytic part which emphasized proofs and derivations whereas for my purposes an intuitive explanation of the concept would have sufficed. In fact, I remembered almost all of the Calculus formulas from undergrad – My only problem was the lack of “sense” in how to apply it to the problem I faced (say in machine learning or some optimization).

Then I found the Calculus Revisited course from MIT OCW. It consists of a series of lectures on Calculus but also assumes that students have had prior exposure to it. This assumption had some interesting consequences and I fit the bill perfectly. I downloaded the set of videos and started listening to them. Interestingly, all the lectures were between 20-40 minutes which allowed for maximum focus and also allowed you to listen to multiple lectures in the same day. In fact, Arlington had a heavy snow this week and my university had to be closed for the entire week. I completed around 16 lectures in 3 days and was able to finish it ahead of my target date of Feb 15.

The course starts with the absolute basic ideas of sets, functions, induction and other stuff. If you are from CS and had taken discrete math, you can feel free to skip the first section. But I would suggest you to still take a look as it , in a sense, sets the stage for the entire course. Do take some time to listen to the lecture on limits. (Part 1 , lecture 4). Here, the discussion of limits effortlessly leads to the derivation of the formula for instantaneous speed and hence differentiation.

Part 2 forms the crux of the course and covers differentiation. Professor Herbert Gross had a beautiful way of teaching stuff about derivatives. In particular, he extensively used the idea of geometric proofs or visualizations to expound basic ideas. The way he brought out the tight relation between analysis (as in Math) and geometry was enthralling. He had a huge emphasis on geometric intuition which helped me to “grasp” the key concepts.

Part 3 had some nice discussion on Circular functions. He joked about how teachers don’t provide good motivation for learning trigonometry which I felt very true to me. He also explained some concepts that were new to me – that you do not really need triangles to define cosine and sine. Previously, I was aware of the radian concept but never put it all together. He also explained how sine and cosine tend to come up in unexpected places – like as the solution of the differential equation for harmonic motion 🙂 He also masterfully showed the close relation between circular and hyperbolic functions with a playful title of ‘What a difference a sign makes’  (in Part 5).

Part 4 discussed about integration and how it can be used to calculate 2 and 3 dimensional areas (and volumes). This part also had a great discussion on how differential and integral calculus are related. That being said, I was a bit dissatisfied with the discussion on the two fundamental theorems of Calculus. The discussion on Mean Value Theorem also felt a bit rushed. I got a bit lost on the discussion on 1 dimensional arc length calculations. May be I should revisit the lecture notes for the same when I get some free time.

Part 6 was my favorite part for two reasons – This had a discussion of infinite series and my favorite quip of the course. When discussing about the non intuitiveness and the intellectual challenges posed by infinity , professor Herbert Gross playfully quips (which goes something like this)– ‘ of course, one thing to do is to not study it. I can call it as the right wing conservative educational philosophy’ – Ouch 🙂 I think I mostly understood the idea of infinite series even though there was not much explanation of “why” it works that way. I also felt the topic of Uniform Convergence to be way beyond my comprehension level.

 

Overall, it is a great course and acted as a fast paced refresher for those who had already taken Calculus. The course slowly starts from basic pre-calculus ideas and rapidly gains speed and covers a huge list of calculus topics. I felt few of important Calculus topics were not covered or rushed up – First and second fundamental theorem of Calculus, Mean Value theorem, Taylor series, L’Hospital rule, discussion of exponents and logarithms etc.

But that being said, I feel the course more than makes it up for the way the basic ideas were covered. I had fun learning the ideas of limits, infinitesimals , intuitive ideas of differentiation/integration, geometric explanation of differentiation/integration, how the concept of inverse functions pervades Calculus etc. Prof. Herbert Gross had a jovial air around him and occasionally delved into philosophical discussions which made listening to the lectures more interesting.

He also had an extensive set of supplementary notes and huge amount of problems with solutions. I had to skip the problems part to conserve time. But if you have some time do spend some on it.

Lastly, I found that one of the lectures in the series was missing. Lecture 5 in Part 2 on Implicit Differentiation was the same as the one on Lecture 4. I had sent a mail to MIT OCW about this and got an reply saying they will fix it soon. Hopefully, it will be fixed soon.

In conclusion, this is a great , fast paced course on Calculus that emphasizes geometric intuition of the major ideas in Calculus.  Listen to it if you already know Calculus and want a fast refresher ! I am currently listening to the lectures on Multi Variable calculus. I do intend to listen to Single variable Calculus again , may be in Summer. I will put out another post on how it went 🙂

Advertisements

Read Full Post »

MIT OCW is one of the best things that happened in education in the last few years. It features hundreds of course materials available for free to anyone who is interested. I became interested in OCW to improve my CS and Math skills. So far, I have completed multiple courses and currently I am trying to follow their MultiVariable Calculus course.

It always amazed me that MIT was willing to put such high quality stuff out in the open – Sure, my toiling on Calculus or Linear Algebra does not have a “tangible” outcome – meaning, I am not going to get MIT certified for all the hours I spent on it. But it does help me do well in courses in my university based on these courses. For eg,  I have a different perspective of how things work in data mining or machine learning after watching few of MIT lectures on  Calculus / Linear Algebra etc.

MIT OCW has a detailed page on donating where they provide all necessary information. I do try to support MIT OCW by donating at least couple of times a year – But as a student my monetary contributions are limited which always bothered me. 

A Simple Idea

Few days back, I got an idea after reading this post . Basically, the post details the amount of referral money that GNOME foundation gets from Amazon when users buy songs from Amazon MP3 store via Banshee. Sure, the amount is small as the audience is small – People who use Linux AND Banshee AND buy from Amazon within Banshee !

I also remembered that MIT OCW does have Amazon referral link. More details are given at OCW’s Amazon referral page. Amazon gives MIT OCW 10% referral when people buy items from Amazon when either clicking on book links from OCW or when they visit to Amazon through the URLs in the referral page. Since the OCW audience is considerably bigger, I searched for the amount of referral money that Amazon brings. The details are scarce but this link says the amount is around $30,000 at the end of 2009.  That is pretty impressive considering it is almost 20% of the visitor donation of 2009 !

Whenever I want to buy something in Amazon (that’s my primary choice as I have Amazon Prime) , I directly search using the Chrome’s Amazon search instead of going to Google. So, I reasoned that if I could somehow make MIT OCW as the referrer, then OCW will get the referral fees from whatever purchases I make. Even though it might be limited, it indirectly increases my “contribution” to MIT OCW.

A small technical detour

Before we modify our browsers to make MIT OCW as the referrer , we might have to change some habits – Don’t worry , it will only make you more efficient 🙂

First, when you are buying do consider Amazon. Of course, not at the expense of your finance. But lot of times, I find that Amazon has the cheapest rate or the difference is reasonably negligible. Since most of the blog readers are students, you may be interested to know that they offer one year free Amazon Prime for students. Your shipping will be free if you chose two day shipping. Even otherwise, Amazon has free super saver shipping for most items above $25. Most of textbooks cost  a fortune and you will be eligible most of the time.

The next is to directly search in Amazon instead of searching the item in Google and going to Amazon by clicking the link. If you feel you might miss some cheaper alternatives, check out other book search engines like vialibri.net which searches multiple sources anyway.

One of the coolest Chrome Geek tip is to use its search engines effectively – Chrome comes with lot of built-in ones and allows you to assign keywords for them. To manage them, click on the Wrench icons and select “Preferences” (Options in Windows) . In the “Basics” tab, go to the section on “Search” and click on “Manage”. You will get a new window listing known search engines.

Lets change the way you search items, in say, Wikipedia. Go to the entry for Wikipedia and click on ‘Edit’. You will see a new smaller window. The item we are interested is the second textbox which is labeled as “Keyword”. Set it to some mnemonic you want -  For eg wiki. I set it just w. Now click save and close the window. Go to Chrome’s address bar and type “wiki Blah” (or in my case w blah). Watch Chrome magically take you to the appropriate wikipedia page directly – No intermediaries like Google ! Think of all the seconds that you can shave off for each Wikpedia search 🙂

I have mnemonics for everything in my Chrome. I use y for Yahoo, b for Bing, a for Amazon and so on. So all you need to do to search for some item in Amazon is to type “a item” in Google’s address bar (assuming ‘a’ is mnemonic for Amazon).

If you are using Firefox, then its now as straightforward. Firefox does come up bundled with few search engines and Amazon is one of them – but reaching is a bit tricky. Now press “ctrl+k” to reach the search pane (or use the mouse !) Once you are in search pane, you can switch search engines, by using the arrow keys. Once Amazon search engine is selected, type the appropriate keyword and the Amazon’s result page pops up.

Firefox also offers an easier way. Click on the list of search engines list in search pane and select ‘Manage Search Engines’ . Now in the new window, select the appropriate search engine and click on ‘Edit Keyword’. For eg, you can set ‘a’ for Amazon. Then you will be able to type “a item” in address bar to search for “item” in Amazon directly !

Changing Referrer in Chrome

Changing the referrer in Chrome is easy. Click on the Wrench icon and select “Preferences” (Options in Windows) . In the “Basics” tab, go to the section on “Search” and click on “Manage”. You will get a new window listing known search engines. Now select the entry for Amazon and click Edit. (I f you cannot find the search engine for Amazon, then go to Amazon and search for dummy stuff and check again). In the “URL” textbox append the words “&tag=mitopencourse-20”. In Ubuntu, your URL field will look like this (you can even copy this one !)

http://www.amazon.com/exec/obidos/external-search/?field-keywords=%s&mode=blended&tag=mitopencourse-20

In Windows, my entry looked like this :

http://www.amazon.com/s/ref=nb_sb_noss?url=search-alias%3Daps&field-keywords=%s&x=0&y=0&tag=mitopencourse-20

The word “tag=mitopencourse-20” is what that makes MIT OCW as the referrer. I have assumed that you are using Amazon US. If you are other versions of Amazon, use the following table and swap the word “mitopencourse-20” with the other.

Amazon Locale

Referrer Code

US

mitopencourse-20

Canada

miop-20

France

miop0c-21

Germany

miop0a-21

Italy

miop01-21

UK

miop-21

 

Now search for something in Amazon directly using the keyword you set previously. For eg, if you used “a” as mnemonic then try “a itemName” and find if the tag that you just set appears in the url of the search results page. If so, then great ! Else, recheck the previous steps.

 

Changing Referrer in Firefox

Changing the referrer in Firefox is a two step process – Since Amazon comes a default search engine, modifying it directly does not seem to work. Instead , you have to move the item to your local search plugins folder and then change referrer there .

Ubuntu (Or Linux)

1. Find your user specific firefox search plugin folder. Typically it is at ~/.mozilla/firefox/<profileFolder>/searchplugins. Replace the word <profileFolder> with appropriate name. For me it is at ~/.mozilla/firefox/72p71i29.default/searchplugins . If it does not exist, create it.

2. Assuming English as your firefox locale , move the file /usr/lib/firefox-addons/searchplugins/en-US/amazondotcom.xml to your profile folder. It is important that you move the file and not just copy it.

3. Open the file in some editor like vim. Check for the line that starts like

<Param name="tag" value="someRandomText"/>

4. Now modify that line and change the value to “mitopencourse-20”. Your line must look like

<Param name="tag" value="mitopencourse-20"/>

 

Of course if your Amazon locale is different, use the above table to find the appropriate referred id. Reopen Firefox and search for something in Amazon directly using the search engine (either the search engine in search pane or using its keyword). If the search result has the tag for MIT, then great ! Else please repeat the steps again more carefully.

 

Windows 7

1. Find your user specific firefox search plugin folder. Typically it is at C:\Users\<userName>\AppData\Roaming\Mozilla\Firefox\Profiles\<profileName>\searchplugins . Replace <username> with your actual username and profile name with your actual profile name. For me it is at C:\Users\zionWin\AppData\Roaming\Mozilla\Firefox\Profiles\jon9mxki.default\searchplugins . If it does not exist, create it.

2. Move the file “C:\Program Files\Mozilla Firefox\searchplugins\amazondotcom.xml” to your profile folder. It is important that you move the file and not just copy it.

3. Open the file in some editor like notepad++. Check for the line that starts like

<Param name="tag" value="someRandomText"/>

4. Now modify that line and change the value to “mitopencourse-20”. Your line must look like

<Param name="tag" value="mitopencourse-20"/>

 

Of course if your Amazon locale is different, use the above table to find the appropriate referred id. Reopen Firefox and search for something in Amazon directly using the search engine (either the search engine in search pane or using its keyword). If the search result has the tag for MIT, then great ! Else please repeat the steps again more carefully.

 

Conclusion

The effort needed to make this change will be only few minutes – You are not killing kittens. Also this does NOT increase the price of your item that you buy from Amazon. All that happens is that when you buy something from Amazon, MIT OCW will get a 10% cut.  It is not clear to me whether that applies only to textbooks or to any items. Typically, referrals works for all items.

I think this might be the simplest way that you can contribute to MIT OCW without much effort from you. Please spread the word !

Read Full Post »

Linear Algebra is one of the coolest and most useful math courses you can take. Basically , it deals with vectors , matrices all the cool stuff you can do with them. Unfortunately, I did not really have a dedicated course on Linear Algebra in my undergrad. From what I hear , most of the CS people I meet (from India) also don’t have this course in their undergrad. Sure we have had some of the topics (like vectors, basic matrices, determinants, Eigenvalues) split across in multiple courses or in our high school ; but not a single,unified course on it.

Linear algebra is useful on its own but it becomes indispensable when your area of interest is AI , Data Mining or Machine Learning. When I took a machine learning course , I spent most of the time learning things in Linear Algebra, adv Calculus or Linear Optimization. In hindsight , machine learning would have been an easy course if I had previously taken courses on Linear Algebra or Linear Optimization.

As a concrete example, I had a hard time understanding the proof of optimality of PCA or the equivalence of different techniques for calculating PCA (eg Eigen space decomposition or SVD etc) . But once I learnt all about basis, dimension , ,Eigen space and Eigen space decomposition, QR decomposition,SVD etc (which are btw taught in any intro course of Linear Algebra) the whole PCA concept looked really really simple and the proofs looked like straight forward algebraic derivations. Oh well, the benefits of hindsight 🙂

Ok, enough on my rant on lack of Linear Algebra in undergrad. After I struggled mightily in my machine learning course, I decided that I had to master Linear Algebra before taking any more advanced courses. I spent the entire winter holidays learning Linear Algebra as I was taking an advanced data mining course this spring. So this blog post is a discussion of my experience.

Video Resources


Arguably the best resource to learn Linear Algebra is MIT’s OCW course taught by Professor Gilbert Strang . This course are is one the most popular OCW course and so far had more than 1 Million visits . I also searched for alternate courses, but this course wins hands down both for its excellent teaching style and its depth.

The course website is here. It contains around 35 video lectures on various topics. The lecture are available for download both from ITunes and from Internet Archive. If you prefer YouTube, then the playlist for this course is here.

Books

The recommended book for this course is Introduction to Linear Algebra. 4th ed. by Gilbert Strang. I found the book to be quite costly , even used books for old versions ! I don’t mind buying expensive books (I shell out a lot of money for data mining books , but a rant on it later ) but since I was interested in Linear Algebra primarily to help me master data mining, I preferred the equivalent book Linear Algebra and Its Applications , also by Gilbert Strang. This book had a very similar content to the recommended book but I felt was more fast paced which suited me fine. Also I was able to get an old copy from Amazon for 10 bucks. Sweet ! My only complaint of the book is that the examples and exercises felt a bit disconnected (or should I say, I wasn’t clear of the motivation ? ) from the topics.

If you don’t want to purchase these expensive books , then there is an EXCELLENT free e-book by Professor Jim Hefferon .The book’s website is here , from where you can download the e-book. I have to say, this book really blew me away. It was really intuitive, has excellent (mostly plausible) examples, was slightly more theoretical than Strang’s book with more proofs. It also had a very helpful solution manual , and a LaTeX version of the book. Too good to be true 🙂 I felt this book had a much limited set of topics than Strang’s course/book, (hence this a truly intro book) , but whatever topic it took, it gave it a thorough treatment. Another thing I liked in the book are the  exercises – Most of them were excellent. And having a solution manual helped clarify a lot of things given that I was doing essentially a self-study. Thanks Jim !

Impressions on the Lectures

I felt, overall, the lectures were excellent. They were short (40-50 minutes). So my usual daily schedule was to listen to a lecture, and read the relevant sections in the book , solve the exercises for which the answers are available at the end of book. All these steps took at most 2-3 hours a day. I was also taking notes in LaTeX using Lyx. I have talked about using Lyx previously in this blog post.

I really liked Strang’s teaching style. He often emphasizes intuition , especially geometric intuition rather than proofs. I felt that is how intro courses must be structured. Proofs are important but not before I have a solid understanding of the topics. But I also have to say that the lectures were varying in quality. Some of the lectures were exceptional while some were not so enlightening. But on the whole, I was really glad that he has made the lectures available online. It has certainly helped me learn Linear Algebra.

Topics

If possible see all the lectures as almost all of them cover important topics. I did and have to say all of them were excellent and useful. But if you are mostly interested in applied Linear Algebra and planning to use it in Data Mining/ Machine learning, then my suggestion will be Lectures 1-11 , 14-22,25,27-29,33. If you are interested watch lectures 30,31 too. Again a better way to learn is to take notes during lectures and solving at least a few exercises in the book. If you have Matlab or Octave then you can verify answers to some other exercises for which solutions are not given.

Notes

I have taken LaTeX notes for this course but they are a bit scattered and unorganized. Hopefully, I will organize them together and create a single PDF soon. I kind of put it on a lower priority after I noticed that Peteris Krumins’s blog has a partial set of lecture notes for this course. His lecture notes can accessed here . As of now (Jan 30 , 2010) , he has put notes for first 5 lectures although the frequency seems to be a bit slow.

Have fun with Vectors and Matrices !!

Read Full Post »