Feeds:
Posts
Comments

Posts Tagged ‘Python’

Ubuntu Unity is a really cool and innovative approach for Ubuntu UI. There are many people who love it and many who hate it. I fall somewhere in the middle. I am really fascinated by the possibilities that Unity offers and hope that it becomes better as it matures. Lenses are one of the coolest pieces in Unity and unfortunately not much tutorial is written. So this is my stab to fill that void. I have tried to organize as many material as possible. Your comments on improving the article is always welcome !

In my tutorial on Unity , I mentioned that if no one else writes a Lens tutorial, I will write one and here it is 🙂 Lenses in Ubuntu are a really neat idea – One simplistic way is to view them as specialized search tools. You open the appropriate lens, type something and you get a list of results. But this explanation does not really do justice to the incredible amount of flexibility that they give. Lenses can group their results, have Unity render the results in different styles, have multiple sections that influence they way search is done and so on. Unity also provides a very powerful API that allows you to write literally anything that you can think of. This post will discuss the architecture, components and their potential capabilities in detail.

Ubuntu Lens Terminology

If you are using Unity, then you must be familiar with two default lens that come with it : Application lens and Files & Folders lens. Let me use that to explain the various parts of a Lens.

Places/Lenses : Both these words represent the same idea. Places is a terminology that was primarily used when Unity was introduced. Its usage is now slowly deprecated and the word Lenses is preferred. In the rest of this blog post, I will refer it as Lenses for uniformity. Let us take a look at the various parts of lens by taking the application lens as an example.

If you click on the Application lens, you can see that it is split into two important pieces. At the top, is a search textbox like entity that allows you to enter the query. At the bottom is the results pane that shows results of your query. At the left end of the search textbox is a search result indicator. When the search is still going on, the indicator will show a spinning circle. Once the search is done, it reverts to its default icon of a cross.

Sections : The simplest way to describe sections is that they behave like categories that are non overlapping. So if a given search query might result in different results when searched under different categories, each of the qualify as a section.

Consider a simple example – You are writing a lens for StackOverflow. You may want to search questions, tags, users and so on. The results of search query q as a question is not the same as when searching q as a tag. Hence questions, tags, users become sections. Unity allows you to easily have different sections and switch between them.

Sections are the entries in the dropdown list  at the right end of the search textbox. They are optional and many places need not have it. You can see all sections of a lens by clicking on the dropdown. Taking the example of Application lens, some of the sections are “All Applications”, “Accessories”, “Games”, “Graphics” etc.

Groups : Groups is a mechanism by which you can club certain search results under a meaningful entity. Each search result, might have one or more groups that allow you to organize the results. For eg, when you search in Application lens, the groups are “Most frequently used”, “Installed”, “Available for download” etc. Groups need not be non overlapping like sections. Given a search query, same result can occur in different groups. In addition, Unity allows you to have different icons for different groups. Also, each group can be rendered in a different style. I will discuss further details later in the blog post.

Difference between Sections and Groups : There are lot of differences between groups and sections. Sections are mostly non overlapping set of entries. Each section provides a different perspective of the same search query. In contrast, groups allow you to classify the search results from a query (and a section). Groups can be considered as a way to organize the results rather than influencing how the results are obtained (which the sections do).

Results : These are the output of a search query in a lens. Given a query and a section, you get a list of entries for results. Each of them can optionally be assigned to a group. Each result entry can display information about themselves in a concise way. In addition, Unity allows them to have any URI and a mime type. So the results of one lens might be files clicking which executes them. Or it can be results from web and clicking on them opens the link in a browser and so on. Unity also allows them to have custom icons.

Unity Lens – Technical Terminology

In the following section, we will discuss some terms used in Lens development.

DBUS : DBUS is one of the most important component in the development of lens. One way to view a lens is as a daemon that accepts a search query from Unity, does some processing and returns a result along with information on how to display them. DBUS is the conduit through which the entire communication between the daemon and Unity happens. For the purpose of the lens, you need to own a bus which will be used by Unity for communication.

Lens/Place Daemon : Lens/Place daemon is a program that exposes lens functionality. A single daemon can expose multiple lens functionalities. Each lens is identified by an entry. It accepts search string from Unity and sends out search results that annotated with group and other additional information.

Place File : A place file is how you inform Unity about the various lenses that your daemon is going to expose. It is a text file that ends with .place extension and usually placed in /usr/share/unity/places . It contains different groups of entries that describe the different lenses that are exposed.

Lens Entry : A lens or an entry in the place file is distinct entity that can accept a search query and provide appropriate results. Every place file contains atleast one lens entry. Every entry has a distinct location for itself in the Unity places bar (the vertical strip) on the left. The lens entry has 3 broad components : A model each for sections, groups and results. Results contain information about the groups they fall in and how to display themselves.

Results : As discussed above, a result is an entry that is the response to a query. A result can be assigned to one or more groups. In the most general sense, a result consists of 5 subparts :
(a) URI : This is a resource identifier to uniquely denote the location of search result. It can be a local item (file://) , web item (http://) and so on.
(b) Display Name : This is a name for the result that the user will see in the results pane.
(c) Comment : This is additional information about the result entry.
(d) MimeType : Since the URI can point to anything, it is important to inform Unity what action to perform when the user clicks on the result. The mimetype allows Unity to invoke the appropriate launcher.
(e) Icon : Each result can have its own icon to represent it visually. It can be an icon file or the generic name of the icon (eg video).

Renderer : Unity provides some flexibility in how the results are displayed. Each result falls into some group. We can request Unity to display each group in a specific way. This is accomplished by the use of renderers. When you setup the model for groups for the lens, we also specify how each group will be rendered. Some common renderers are :

(a) UnityDefaultRenderer : This is the default way to display the results. In this renderer, each result shows the icon and the display name beneath it.

Ubuntu Unity Lens results under UnityDefaultRenderer
(b) UnityHorizontalTileRenderer : In this renderer, the icon and display name for each result is shown. In addition, it also displays the comment associated with the result.

Ubuntu Unity Lens results under UnityHorizontalTileRenderer
(c) UnityEmptySearchRenderer : This renderer is used when the current search query returned no results. The result item construction is very similar to the results above. But the display name is displayed across the results pane.

Ubuntu Unity Lens results under UnityEmptySearchRenderer
(d) UnityShowcaseRenderer : I tried using it and the result was not much different from UnityDefaultRenderer. But according to the documentation, it should have a bigger icon.

Ubuntu Unity Lens results under UnityShowcaseRenderer
(e) UnityEmptySectionRenderer : I did not try this as it not make sense in the application I developed. Supposedly, it is used to imply that the current section is empty. If my interpretation is correct, then this is used when the current section returned no results for query but a different section might produce results. UnityEmptySearchRenderer on the other hand is used to imply that the current search query itself (regardless of the section) provides zero results.
(f) UnityFileInfoRenderer : Copying from the reference, this renderer is used in files place to show search results. Expects a valid URI, and will show extra information such as the parent folder and whatever data is in the comment column of the results.

Steps to write a Lens :

There are three major steps in writing a Lens.
(1) Writing a .place file
(2) Writing a Lens daemon
(3) If you want to distribute , Writing a service file

As an example, let us develop a Lens that does the following :
(1) It has two sections – In “Movie Names” section, only the names of the movie are shown. In “Genre” section, all movies are clubbed according to their genre which act as a groups. So if movie X is in genres Y and Z, X will be displayed in both groups Y and Z.
(2) Clicking on the result will open a browser and redirect to the IMDB page for the movie.
(3) The lens accepts queries from two places – When entered directly in the IMDB Lens and also from the dash. When searched from Dash , it will display movie names as a separate group.

At the initial state, the lens will look like this :

Ubuntu Unity Lens Initial state

For the sake of completeness, here is now it looks when you just search for Movie Names. Ubuntu Unity Lens Search for Movie Names

For the sake of completeness, here is now it looks when you just search for Genres. Basically, the results are grouped using the Genre. If a movie falls under multiple genres, it also falls into different groups. Ubuntu Unity Lens Search for Movie Genre

Let us take a look at how to write the IMDB lens as per the major steps above.

Writing a Place File :

As discussed above a place file contains the information that allows Unity to display the entries in the places bar. Information like which bus to use for communication, what shortcut to use etc are also involved. For the purpose of this tutorial, I will put up the place file I used for the IMDB lens. I will then explain the significance of each line. The file is named as imdbSearch.place and is placed in /usr/share/unity/places/ .

[Place]
DBusName=net.launchpad.IMDBSearchLens
DBusObjectPath=/net/launchpad/imdbsearchlens

[Entry:IMDBSearch]
DBusObjectPath=/net/launchpad/imdbsearchlens/mainentry
Icon=/usr/share/app-install/icons/totem.png
Name=IMDB Search
Description=Search Movies using IMDB
SearchHint=Seach Movies using IMDB
Shortcut=i
ShowGlobal=true
ShowEntry=true

[Desktop Entry]
X-Ubuntu-Gettext-Domain=imdbSearch-place

As you can see, the file has three sections. Each section contains a set of key-value pairs.

Place Section :
This is the simplest section and contains two key values – DBusName and DBusObjectPath. DBusName gives the name of the bus under which this entry can be found while DBusObjectPath provides the actual dbus object path. The names can be any valid DBus names.

Entry Section :
A place file can contain one or more entries. Each entry must contain a section that corresponds to it. Each entry will occupy a distinct location in the places bar. If there are multiple entries, each of them must have a unique name and object path. In the daemon code, all those entries must be added individually.

The various keys are :
DBusObjectPath : This is the DBus path used to communicate with this specific entry. Note that this path must be a child of mail DBusObjectPath. In our case, there is only one entry and hence we put it under the path “mainentry” of the original path.
Icon : Link to the icon file
Name : Name of the entry. When you hover over the entry in the places bar, this will be displayed.
Description : A long description of the lens. I am not sure where this is used.
SearchHint : This is the text that is displayed by default in the lens when the user selects it. In the image above, the search hint “Search Movies using IMDB”is displayed when the user selects the IMDB lens.
Shortcut : This gives the key which is used to trigger the lens. Of course, you can always use the mouse to select it. If this key is specified,then pressing Super+shortcut operns the lens. For the IMDB lens, pressing Super+i opens it.
ShowGlobal : This is an optional key and defaults to true. If it is set to false, then searches from the main dash are not sent to the lens. This seems to override the specification inside the lens daemon. ie If the place file specifies ShowGlobal as false and the daemon adds a listener to the event where user enters a query in the main dash, it is not invoked. Most of the time, I think it makes more sense for the lens to set this as false. For eg, it certainly does not make sense for IMDB lens to offer its service in the dash. Most of the time, when the user is searching in the dash, he is looking for some file or application. Polluting those results with IMDB results may not be wise. This is even more true if your lens takes a while to provide all the results.
ShowEntry : This is an optional key and defaults to true. If it is set to false, then the entry will not be displayed in the places bar. If this is set to false, then the Shortcut key becomes useless as well. Pressing the shortcut key does not invoke the lens. But if the ShowGlobal is set to true, then the lens will still be searchable via the dash. For eg, if for some reason, I decide that IMDB lens must only provide results to queries from dash I will set ShowGlobal to true and ShowEntry to false.

Desktop Entry Section :

This section is mostly optional. The most used key is that X-Ubuntu-Gettext-Domain. This key is used for internationalization purposes. If you want the lens should support internationalization, provide the appropriate entry name. If you are not familiar with internationalization in Ubuntu, there are two broad ways of achieving it : Either put all the translations statically in the file. Or put it in the appropriate .mo file which will then be put in some language-pack file. I included this line because Unity whines as “** Message: PlaceEntry Entry:IMDBSearch does not contain a translation gettext name: Key file does not have group ‘Desktop Entry'”.

Writing a Lens Daemon

The next step is to write a daemon that communicates with Unity over dbus path, does the search and returns annotated search results so that Unity can render them meaningfully. The daemon can be written in any language that support GObject introspection. Most common languages include C, Vala and Python. Vey informally, GOject is a mechanism by which bindings for other languages can be done relatively easily. The actual process needs multiple components. Based on the reference [2], this means that you must verify if the following packages are installed in the system – gir1.2-unity-3.0 , gir1.2-dee-0.5 and gir1.2-dbusmenu-glib-0.4 . Of course, the actual version numbers may change in the future. If installing does not make it work, look at the reference for additional instructions [2].

I will use the IMDB Search lens as an example to explain writing a lens daemon in Python. The full source code is in [1]. I will use snippets of the code to make the appropriate points. The first step is of course importing the appropriate libraries. If you see other Python lens file, they used to have a hack that probes Dee before importing Unity. In my tests I found it unnecessary. If you get some error in importing Unity, then take a look at other sample files [3] and do accordingly.

from gi.repository import GLib, GObject, Gio, Dee, Unity

The next step is to define the name of the bus that we will use to communicate with Unity. Note that this must exactly match the “DBusName” key in the place file.

BUS_NAME = "net.launchpad.IMDBSearchLens"

The next step is to define some constants for the sections in your lens. If your lens contain only one section, feel free to ignore the initial lines. Else define section constants appropriately. The only place where you will use these constants is to figure out which section the lens currently is in. The section ids are integers and are ordered from 0. Note that the order given here is the *exact* order in which the sections must be added to our lens in Unity.

Similarly, we can have constants for groups. Instead of creating lot of constants, I created the group ids dynamically. First, I create a list of all IMDB genres and use a hash to map the name to an integer (group id). Also note that I also have constants for groups – GROUP_EMPTY, GROUP_MOVIE_NAMES_ONLY and GROUP_OTHER.

One thing to notice is that if different sections have non overlapping groups, all of them must be defined here. Then based on current section, use the appropriate group. As an example, GROUP_MOVIE_NAMES_ONLY is used only with SECTION_NAME_ONLY. But we define it alongside other groups. Also note the group for empty, GROUP_EMPTY. This will be used if the search query returns no results.  As with sections, the order of groups defined here must exactly match the order in which they are added to group model. If you have reasonably small number of sections and groups, you can avoid the elaborate setup of writing constants.

SECTION_NAME_ONLY = 0
SECTION_GENRE_INFO = 1

allGenreGroupNames = [ "Action", "Adventure", "Animation", "Biography", "Comedy", "Crime", "Documentary", "Drama", "Family",    "Fantasy", "Film-Noir", "Game-Show", "History", "Horror", "Music",  "Musical", "Mystery", "News", "Reality-TV", "Romance", "Sci-Fi", "Sport", "Talk-Show", "Thriller", "War", "Western"]

numGenreGroups = len(allGenreGroupNames)
GROUP_EMPTY = numGenreGroups
GROUP_MOVIE_NAMES_ONLY = numGenreGroups + 1
GROUP_OTHER = numGenreGroups + 2

groupNameTogroupId = {}
#We create a hash which allows to find the group name from genre.
for i in range(len(allGenreGroupNames)):
        groupName = allGenreGroupNames[i]
        groupID = i
        groupNameTogroupId[groupName] = groupID

The next step is to define the skeleton of the lens daemon. It is a simple Python class. In the constructor, you define the entire model corresponding to each entry in the Lens. If you place file contains n entries, you will be defining n different PlaceEntryInfo. In our case we have only one. Hence we create a variable that points to the corresponding DBusObjectPath. It is impertative that the path exactly match DBusObjectPath of the entry.

Each PlaceEntryInfo consists of different models : sections_model, groups_model and results_model.  If you want the lens to respond to searches in dash, you will also need to setup global_groups_model and global_results_model . The code below contains information about the schema of the different models. You can consider them as more or less a code that can be blindly copied. If you want additional information about things that you can tweak in PlaceEntryInfo, take a look this url .

Note on Property based access :
One important thing to notice is that all the access is done using properties. There are two methods to do that :
(1) x.props.y
(2) x.get_property(“y”)

Choosing one of them is mostly based on your coding style. Choose one use it consistently.

class Daemon:
        def __init__ (self):
                self._entry = Unity.PlaceEntryInfo.new ("/net/launchpad/imdbsearchlens/mainentry")

                #set_schema("s","s") corresponds to display name for section , the icon used to display
                sections_model = Dee.SharedModel.new (BUS_NAME + ".SectionsModel");
                sections_model.set_schema ("s", "s");
                self._entry.props.sections_model = sections_model

                #set_schema("s","s") corresponds to renderer used to display group, display name for group , the icon used to display
                groups_model = Dee.SharedModel.new (BUS_NAME + ".GroupsModel");
                groups_model.set_schema ("s", "s", "s");
                self._entry.props.entry_renderer_info.props.groups_model = groups_model

                #Same as above
                global_groups_model = Dee.SharedModel.new (BUS_NAME + ".GlobalGroupsModel");
                global_groups_model.set_schema ("s", "s", "s");
                self._entry.props.global_renderer_info.props.groups_model = global_groups_model

                #set_schema(s,s,u,s,s,s) corresponds to URI, Icon name, Group id, MIME type, display name for entry, comment
                results_model = Dee.SharedModel.new (BUS_NAME + ".ResultsModel");
                results_model.set_schema ("s", "s", "u", "s", "s", "s");
                self._entry.props.entry_renderer_info.props.results_model = results_model

                #Same as above
                global_results_model = Dee.SharedModel.new (BUS_NAME + ".GlobalResultsModel");
                global_results_model.set_schema ("s", "s", "u", "s", "s", "s");
                self._entry.props.global_renderer_info.props.results_model = global_results_model

Once you have setup the models, you need to add listeners to events that you want to catch. The most important one is “notify::synchronized”. This called when Unity sets up your lens and wants to know the various groups and sections in your entry. In the code below, we add three different functions to that event. One gives the sections in the lens, other gives the groups and last gives the groups in dash.

Next we catch two events that are core to Lens – notify::active-search and notify::active-global-search . The first is triggered when the user searches something in the search textbox of the place while the second is triggered when user searches something in the dash. It is important to notice that the same search can trigger the events multiple times. By default, Unity does a decent job of batching calls providing search strings, but it is an important thing to consider if your lens does expensive operations. Take a look at the section of caching for additional details.

The event notify::active-section is triggered when the user changes the section of the lens by using the dropdown. You can then use your section constants to decide which section is currently selected.

notify::active is triggered when the user selects and leaves your place. Hence its an indirect way for your daemon to know if the lens is being visible to the user.

                # Populate the sections and groups once we are in sync with Unity
                sections_model.connect ("notify::synchronized", self._on_sections_synchronized)
                groups_model.connect ("notify::synchronized", self._on_groups_synchronized)

                #Comment the next line if you do not want your lens to be searched in dash
                global_groups_model.connect ("notify::synchronized", self._on_global_groups_synchronized)

                # Set up the signals we'll receive when Unity starts to talk to us

                # The 'active-search' property is changed when the users searches within this particular place
                self._entry.connect ("notify::active-search", self._on_search_changed)

                # The 'active-global-search' property is changed when the users searches from the Dash aka Home Screen
                #       Every place can provide results for the search query.

                #Comment the next line if you do not want your lens to be searched in dash
                self._entry.connect ("notify::active-global-search", self._on_global_search_changed)

                # Listen for changes to the section.
                self._entry.connect("notify::active-section", self._on_section_change)

                # Listen for changes to the status - Is our place active or hidden?
                self._entry.connect("notify::active", self._on_active_change)

Once you have set up the different models and functions to populate them, you need to add the entry to a PlaceController. If your lens has multiple entries, repeat the above process for each entry and once constructed call the add_entry of PlaceController to set them up. Note that the argument to PlaceController’s constructor is same as the DBusObjectPath for the Place section in your .place file.

                self._ctrl = Unity.PlaceController.new ("/net/launchpad/imdbsearchlens")
                self._ctrl.add_entry (self._entry)
                self._ctrl.export ()

Once we have setup all the models, the next step is to define the functions that are used.The first is to define the function that populates the entry’s section model. Note that we call append on the sections_model with two parameters : the name of the genre and an icon for it. You can also observe that when defined the “schema” of the sections_model, it accepted two strings. One other thing to note is the lack of use of any section constants – You must be careful to define the sections in the exact same order as the constants. In our case, it means that SECTION_NAME_ONLY followed by SECTION_GENRE_INFO.

        def _on_sections_synchronized (self, sections_model, *args):
                # Column0: display name
                # Column1: GIcon in string format. Or you can pass entire path (or use GIcon).
                sections_model.clear ()
                sections_model.append ("Movie Names", Gio.ThemedIcon.new ("video").to_string())
                sections_model.append ("Movie Genre", Gio.ThemedIcon.new ("video").to_string())

The next step is to define a set of groups. Due to the nature of this lens, we reuse the list to form it. Notice the use of UnityEmptySearchRenderer for the GROUP_EMPTY. For this example, I have used UnityHorizontalTileRenderer. Your lens might need some other renderer – Take a look at the renderer section above for a discussion of the different renderers.

                # Column0: group renderer, # Column1: display name, # Column2: GIcon in string format Or you can pass entire path (or use GIcon).
                groups_model.clear ()

                #Remember to add groups in the order you defined above (ie when defining constants)
                for groupName in allGenreGroupNames:
                        groups_model.append ("UnityHorizontalTileRenderer", groupName, Gio.ThemedIcon.new ("sound").to_string())
                #GROUP_EMPTY
                groups_model.append ("UnityEmptySearchRenderer", "No results found from IMDB", Gio.ThemedIcon.new ("sound").to_string())
                #GROUP_MOVIE_NAMES_ONLY
                groups_model.append ("UnityHorizontalTileRenderer", "Movie Names", Gio.ThemedIcon.new ("sound").to_string())
                #GROUP_OTHER
                groups_model.append ("UnityHorizontalTileRenderer", "Other", Gio.ThemedIcon.new ("sound").to_string())

The next step is to define the function that is called when the search changes in the place search box. Notice how we use the entry object to get the current section, the current results, current search query and so on. Finally, we invoke search_finished function which calls search.finished() which signals to Unity that we are done processing the query.

        def _on_search_changed (self, *args):
                entry = self._entry
                self.active_section = entry.get_property("active-section")
                search = self.get_search_string()
                results = self._entry.props.entry_renderer_info.props.results_model
                print "Search changed to: '%s'" % search
                self._update_results_model (search, results)
                #Signal completion of search
                self.search_finished()

There are other functions which we can detail. But mostly, they are very similar to code shown here. You can take a look at the python file linked below [1] for full glory.

So far all we have done is using some boilerplate code. From now on though, the code starts to differ based on the application. So I have not included the code that actually produces the results here. Take a look at the python file for details.

Testing Instructions

Once you have written the python file , you want to test your baby. Here are the steps :

(1) Include the following code in your daemon. This code was literaly copied from Unity Sample Place [3] code.

if __name__ == "__main__":
        session_bus_connection = Gio.bus_get_sync (Gio.BusType.SESSION, None)
        session_bus = Gio.DBusProxy.new_sync (session_bus_connection, 0, None, 'org.freedesktop.DBus', '/org/freedesktop/DBus', 'org.freedesktop.DBus', None)
        result = session_bus.call_sync('RequestName', GLib.Variant ("(su)", (BUS_NAME, 0x4)), 0, -1, None)

        # Unpack variant response with signature "(u)". 1 means we got it.
        result = result.unpack()[0]

        if result != 1 :
                print >> sys.stderr, "Failed to own name %s. Bailing out." % BUS_NAME
                raise SystemExit (1)

        daemon = Daemon()
        GObject.MainLoop().run()

(2) Copy your place file to /usr/share/unity/places/ . If you have any custom icons, copy them to the location specified in your .place file.
(3) Open up a new terminal and invoke your daemon file. Also remember to set the executable bit on.
(4) Open up another terminal and type “setsid unity”. Using setsid is a very neat trick I learnt from the sample place file. Basically, it starts Unity in a new session. This forces it to read all the place files again. Also if you make any change to the daemon code, you can kill it and restart it. Unity will start communicating with your new daemon. Additionally, setsid will also ensure that Unity does not have controlling terminal but will still dump all debug information in the terminal it was opened.
(5) Open the lens and type your query and watch the results appear !

Debugging Checklist

If your lens is not working as expected, here are somethings to verify :

(1) Make sure that your place file is copied to /usr/share/unity/places .
(2) Make sure that busname and object path match exactly in your place file and your daemon file.
(3) Make sure that Python file has its executable bit on. This is important if you want to run it a service/daemon that runs automatically. More on that below.
(4) If you are simply testing it, then is your Daemon file running ? If you are running it as a service, is your service file contain information that match your place and daemon file ?
(5) Did you restart Unity  – preferably as “setsid unity”
(6) Did you make any recent changes to place file ? If so make sure it is copied and restart Unity.
(7) Check for any exception information in the terminal that you started your daemon. If its a service, enable logging and examing the log files.
(8) Check if your lens is actually started. If you used “setsid unity” in a terminal, it would be logging the various steps it does. One sample invocation showing successful starting of lenses is this :

    ** (<unknown>:6114): DEBUG: PlaceEntry: Applications
** (<unknown>:6114): DEBUG: PlaceEntry: Commands
** (<unknown>:6114): DEBUG: PlaceEntry: Files & Folders
** (<unknown>:6114): DEBUG: PlaceEntry: IMDB Search
** (<unknown>:6114): DEBUG: /com/canonical/unity/ applicationsplace
** (<unknown>:6114): DEBUG: /com/canonical/unity/filesplace
** (<unknown>:6114): DEBUG: /net/launchpad/imdbsearchlens

(9) If your lens works but puts results in strange groups, make sure that order of groups in synchronize and the constants are same.
(10) Common errors and fixes :


** Message: PlaceEntry Entry:IMDBSearch does not contain a translation gettext name: Key file does not have group ‘Desktop Entry’ :

This is not an error. If this annoys you add a [Desktop Entry] section. See above for details.

** (<unknown>:6114): WARNING **: Unable to connect to PlaceEntryRemote net.launchpad.IMDBSearchLens: No name owner

This is again not an error but indicates that either your daemon program is not running or is not able to own the bus specified in the .place file.

Installation Instructions

Since I primarily wrote the IMDB lens for learning Unity, I do not know much about the actual installation process. Here are some generic tips :

(1) If you want a simple method of installation, use distutils. It accepts a list of files and the location where they must be copied. This is the cleanest way.
(2) If you are going to install it and expect the lens to start automatically, then you should write a .service file that knows which bus start and which file to invoke to make that happen. The service file is usually put in “/usr/share/dbus-1/services/” . In this case, also make sure that the daemon file is actually executable !

[D-BUS Service]
Name=net.launchpad.IMDBSearchLens
Exec=/path/to/daemom/file

Various Tips On Writing Lenses

(1) Most of the simple daemons use constants for sections and groups. If you want them to setup them up dynamically, take a look how it is done in the IMDB lens.
(2) Avoiding repeated searches :
When the user enters a query in the lens, it is not necessary that the search function will be called only once. Remember that there is no concept of return key to indicate that query is complete. In fact, if the user presses the return key, it selects (and opens) the first result. The way Unity handles this scenario is by batching the calls. Lets say the user wants to search “blah blah2”. If the user continuously types the query with minimal interruption of the keyword, then the entire query will be sent to lens in one shot. On the other hand, if the user is a slow typer and enters “blah b” before waiting for a second or so, Unity will call the search function with partial query. When the user completes the query , the lens will be called with the entire query.

This behavior complicates your lens behavior, especially if the processing is time consuming. There are different techniques to avoid running calculations repeatedly (IMDB lens uses the first two techniques):
(a) Have a cutoff : Your code might decide not to search if the length of string is less that say 4 characters.
(b) Have a cache with timeout : You can use a local cache that stores your previous results . When you get a new query, check your cache first and return the results from it. For added benefit, have a cache timeout that removes elements that were added long time ago.
(c) Memoize using functools : A very cool technique that is used in other lenses is to memoize the function call. This technique works if the function is reasonably simple – the key is the function argument and the value is the return value of the function. One simple example is shown below. In the example, we defined a decorator called memoize_infinitely and apply it to function foo. Foo accepts an input and returns an output. This decorator, automagically creates a cache for each function. When the function is called with some argument, it is first checked in the cache. If it is found, then it is returned without invoking the function. Else the function is called and the result is stored in the cache. This technique is widely used. For a specific example,take a look at the AskUbuntu lens [3] .

import functools
def memoize_infinitely(function):
    cache = {}
    @functools.wraps(function)
    def wrapped(*args):
        if not args in cache:
            cache[args] = function(*args)
        return cache[args]
    return wrapped

@memoize_infinitely
def foo(input):
	return input + 1

(3) Handling long computations :
If your lens returns lot of results or the processing for each result entry takes a lot of time, then its a good idea to display the information as it arrives. The results model has a function called flush_revision_queue which periodically sends the data so far to Unity. For eg, in the case of IMDB lens , fetching the genre information is expensive. So, the code flushes the results every time five movies were processed. This allows the user to see some results without waiting for eternity to get complete results.

Links

    [1] The code for IMDB lens which I have tried to comment as much as possible is my github page. For convenience, I have also given the individual files : Readme, imdb-search.py, imdbSearch.place .

    [2] Unity Places – now with 100% More Python : This blog post contains a brief discussion on developing Unity lenses in Python. It has some info on installation and links to a sample lens in Python. I would advise you to start with the sample lens code given in this blog post before making drastic changes. This way you can make sure that the initial lens setup is fine and the issue is really in your code 🙂
[3] Link to some Good Unity Python Lens codes : As of now there are few other good Unity lenses written in Python. Here are some of  my favorites – AskUbuntu lens, Launchpad lens, Evernote lens, Book-Search lens, Music search lens , Spotify lens . I have tried to incorporate all cool features in IMDB Search lens , so that it will become a one stop place for most important Lens features 🙂

    [4] Ubuntu Unity Architecture : This contains some language independent discussion of Unity architecture and additional information of the data structure it uses.
[4] IMDbPy : The IMDB search lens was developed using the excellent IMDbPy library. It has very convenient API that allows development of IMDB based applications a snap.

  [5] Ubuntu Unity Lens Python API Links : Here are some links for documentation that I found  – Dee API, Unity API . The document for Unity also encompasses the documentation for Places.

Read Full Post »

Prelude : My blog has been silent for quite some time. I was busy with my Masters thesis for couple of months. I am starting with my PhD , wrapping up things and slowly getting into the groove. I hope to settle into a routine where I have one long blog post per week. Let us see how it goes !

    Nautilus is one of the most commonly used file manager for GNOME. One of the reasons for its popularity is its extensible architecture that allows developers to write scripts to customize it. Even if you are a command line person, extending Nautilus will result in dramatic increase in productivity. In this post, I will discuss the multiple ways in which Nautilus can be extended and the relative merits in each approach.

    The first step in extending Nautilus is to find the set of actions that are tedious (atleast not straightforward) – Tasks that need additional clicks or switching to terminal to get completed. The next step is to determine if there are command line tools available to automate the task or the task can be completed by additional code – Again since you are extending Nautilus, the task involved has to relate to files or folders. For eg, opening a file as administrator is a "relevant" task but starting a nuclear war from Nautilus is not !

Informally, it is easy to extend Nautilus if your task falls in the following categories : Adding new entries in context menus (or in toolbar) that involve selected files/folders or current folder, add additional custom properties to files and display those details in list view, modify the properties page to display additional tabs with information etc. There are other possibilities but these are the most common ones.

    If the above discussion sounds very abstract let us give some examples :
1. Open a terminal in the current folder or open the selected file as root.
2. Selecting a few audio files and adding them to Rhythmbox "Now Playing" queue.
3. Selecting a few files and sending them to thunderbird for attachment
4. Display IMDB details about the selected movie file in the property page etc.

The above examples show a gradient of use cases in the order of complexity. Some of them are so simple that they can automated using simple means. Tasks like (4) are tricky and need powerful tools. Selecting the right tool is important and we will discuss how to select the best approach.

Different Approaches to Customize Nautilus Context Menus

Like everything in Linux, there is always a variety of ways to customize Nautilus ranging from simple to complex. In this post, we will discuss the three most common approaches :
1. Using tools like nautilus-actions
2. Using Nautilus scripts
3. Using extensions written in nautilus-python

As before, all my discussion will be focused on Ubuntu but it should be relatively easy to apply to other Linux distributions.

Customizing Nautilus context menu using  nautilus-actions

This is probably the easiest method. All you need to know is the shell command or script to perform the task. Nautilus actions provides an intuitive GUI to decide on the filters and the actions to be performed. This approach works best if the following conditions are met :
a. the task you want to be automated is easily translatable in command line
b. the command line utility accepts the arguments in a relatively simple form (eg space separate arguments etc)
c. The command line utility depends only on information pertaining to the selected file/folder.

To install the package, type the following at terminal (or install this package from Synaptic) :
   

sudo apt-get install nautilus-actions

Once the package is installed it can be accessed at System -> Preferences -> Nautilus Actions Configuration. I will only give a basic discussion here as there is a decent tutorial on how to create a new action at How To Add Custom Functionality To Nautilus.

Let us take a simple example – If I right click on a folder , I want a new menu which says, "Open Terminal Here" and when it is clicked, a new terminal must be opened and the working directory of the terminal must be the selected folder. The first step is to find if it can be expressible in a "single" command. Find the name of the command to invoke the terminal – it is called gnome-terminal. Read the man page to find that it accepts an argument "–working-dir". When provided, it starts the terminal in specified folder.

Now start the Nautilus Action from System -> Preferences -> Nautilus Actions Configuration . The steps are :
a. Create an action.
b. In "Action" tab, give the action some name and select "Display item in selection context menu". If you want it to be visible in the toolbar, it can done too ! Select "Display item in toolbar" and choose some icon.
c. In the command tab, give "gnome-terminal" as path and parameters as "–working-directory=%d/%f". The %d and %f are special codes that will be expanded when the command is invoked. To see other special codes and what they mean, click on the "Legend" button.
d. In "Conditions" tab, select "Only Folders".

Now open a new Nautilus window , select a folder and right click. Presto ! You will see a "Open Terminal Here" menu. Select it and you will see a new terminal open with the selected folder as its current directory !

Some Notes On Nautilus-actions

You can take a look at Nautilus-actions project page to find lot of actions that automate your common tasks. To include them in your system, download the schema and then import them. To import, Tools -> Import assistant -> select the schema file. There are literally hundreds of nautilus actions that you can use immediately.

Nautilus Actions is a very nifty GUI tool that makes creating context menus dramatically easy. The best part is that specifying the command and filters is very intuitive. I will say that for a vast majority of scenarios, Nautilus actions is all that is needed to automate some task. But the simple nature of the tool is also its biggest enemy – there are some tasks that are slightly more complex and nautilus action is not very useful in them. A simple example will be a task which needs two commands to complete. Consider task (2) above – Adding selected mp3 files to Rhythmbox queue. The problem here is that if the file is not already in the library, Rhythmbox will not add them to the queue. So this needs two commands – one to add them to library if not already present and then add them to queue. Tasks like this are tricky to implement using Nautilus actions.

Another class of tasks that are not solvable using Nautilus actions is the ones that have tricky input formats. If you go the "Command" tab and click on "Legend" button, you can see that the list of special codes available are limited. They satisfy majority of needs but not all. Let us take a simple example , say task (3) – send selected files to thunderbird as attachment. Thunderbird has a command to compose a new mail with attachments but the arguments must be "comma" separated. In this case you are out of luck with Nautilus actions and must look at more powerful tools.

Nautilus Scripts

Nautilus has an extensible architecture that allows scripts to be executed. There is usually a misconception that Nautilus scripts has to shell scripts. That is no longer the case. Nautilus allows you to run scripts written in any language as long as they are executable and it knows which program to invoke. This means that you can write scripts in shell , python, perl or any other language as long as the first line of the script says which program Nautilus will have to invoke to execute the script (ie the shebang line).

You can consider that Nautilus actions provides a convenient wrapper over Nautilus scripts .The statement is not exactly true , but it is a good start. Before discussing Nautilus scripts, let us compare the two approaches – Nautilus script allows you to do certain things that are not possible in Nautilus actions. For eg, you can have a huge shell script with multiple lines that gets executed instead of a single command. In my opinion, another important advantage is that it is possible to invoke a Nautilus script without selecting a file – For eg one that runs using just the current folder information. This is not possible in Nautilus actions. On the down side, scripts are always available. It is not possible to show a Nautilus script for only mp3 files or only for local files. Another issue is that Nautilus scripts are most useful for local files and folders. For remote files (eg ftp , smb or webdav) they are essentially useless – but Nautilus action based commands will work fine in this scenario.

How to write Nautilus Scripts

All the Nautilus scripts are in the folder $HOME/.gnome2/nautilus-scripts . You can access them in command line or by File -> Scripts -> Open Scripts folder. Alternatively, you can right click on a file/folder and select Scripts->Open Scripts folder from the context menu. Note that the scripts menu will be available only if you have some scripts in that folder.

To write a new script, create a file in $HOME/.gnome2/nautilus-scripts and change its permissions so that the "user" has executable permissions. Make sure that you have a shebang line which tells Nautilus which program to invoke to run this script. If you want to organize your scripts then create subfolders in the nautilus-scripts folder. All the subfolders become sub menus in the Nautilus context menu.

When your script is invoked by Nautilus , at the most 4 environmental variables will be set by it. They are NAUTILUS_SCRIPT_SELECTED_FILE_PATHS, NAUTILUS_SCRIPT_SELECTED_URIS, NAUTILUS_SCRIPT_CURRENT_URI and NAUTILUS_SCRIPT_WINDOW_GEOMETRY. NAUTILUS_SCRIPT_SELECTED_FILE_PATHS and NAUTILUS_SCRIPT_SELECTED_URIS are newline delimited variables about the selected files. The only difference is that one gives the name of the directory and other gives it as a URI. ie for a local file /home/blah/abc.txt , the first variable will /home/blah/abc.txt  and second will give file:///home/blah/abc.txt . NAUTILUS_SCRIPT_CURRENT_URI gives the URI of the current location.

You can immediately see that the information available is pretty limited. You have to write the script using only these variables. The information is available as environment variables in shell script. For other languages, you may have to use additional modules to access them. Once you have written a script, it is time to test it ! The steps are :

a. Open a terminal and type "nautilus -q". This will close all instances of Nautilus.
b. Type "nautilus –no-desktop" . This is very useful when you are developing the script as it open Nautilus without opening your profile or preference data reducing the chance of data corruption.
c. Right click on a file/folder. You will notice that a new "Scripts" menu comes up and it has your script as the submenu. Selecting it will run your script with the options. You can alternatively run the script from File -> Scripts.
d. If you want to run a script without running on any specific file/folder, invoke the script using File -> Scripts. You must note that in this case only NAUTILUS_SCRIPT_CURRENT_URI will have meaningful value.

Debugging Tips for Nautilus Scripts

It is possible that your scripts may not work the first time. There are no easy way to debug the scripts. Here are some ideas that I find useful :

Case 1 : The script does not appear in Scripts menu (or the scripts menu itself does not occur)
1. Have you placed your script in $HOME/.gnome2/nautilus-scripts ?
2. Does it has executable permissions atleast for the user ?

Case 2 : The script appears in the menu but does not seem to work correctly
In this case, the bug is most likely in the logic of the code. Comment your entire script logic except the shebang line. Add few lines to echo the values of the environmental values into some file in say /tmp/blah. Quit nautilus and open it up. Execute the script. This will result in the arguments being dumped into the log file. Now manually set the environmental values with the values in log file and try executing the script from command line. Since we have the values of the environmental variables, we can simulate Nautilus execution from command line. Most likely , this will allow you to find the issue and fix it.

Notes on Nautilus Scripts

A huge list of Nautilus scripts are available at g-scripts page . Download the file
nautilus-scripts.tar.gz . Even though the information available to scripts are limited, many of them use shell commands in innovative ways. For eg, they use gdialog or zenity to show information graphically. Browse through the scripts to get an idea of what is possible using Nautilus scripts.

Nautilus scripts brings certain advantages and disadvantages. On one hand, they allow more complex actions to be performed. They are not limited to have only single command like Nautilus actions. They also can be executed without selecting any files. But the information available to the scripts is very limited – only 4 environmental variables. Also they are not useful when you invoke on remote files. It is also not possible to show the script only to files with selected attributes (eg only audio files). Of course, you can always return prematurely for files not matching your criteria but it is unnecessary work that must be repeated for all files.

Nautilus Extensions

Nautilus extensions can be considered as the Swiss army knife of customizing Nautilus. They have a relatively steep learning curve but once you learn them , you can extend Nautilus in ways that are simply not possible using other approaches. Nautilus extensions allows you to do tasks that are made possible by previous approaches and more . Some of the things that are possible are :

a) Adding menus for files/folders with certain attributes.
b) Adding new menus in toolbars.
c) Adding new custom attributes to files and folders and using them do perform special actions. It is also possible to display them in the detailed view. A simple example is to add say length of song for mp3 file and show it as a column in Nautilus.
d) Add new tabs in the property page – eg show additional IMDB information for movie files.
e) Add additional information about current location near the location bar.

The classical way to write extensions is in C. But gnome programming in C is bit tricky. I will not discuss them as there are easier ways available. If you are still interested , you can take a look at the Nautilus extensions API reference provided in the Notes section. You will have to compile your extension code and put the executable in /usr/lib/nautilus/extensions-2.0/ .

A cleaner and much easier alternative is to use Nautilus-Python . Nautilus-Python by itself is a C extension for Nautilus. What it does internally is that it exposes a "nautilus" module which can be used by Python programs to write Nautilus extensions in Python. The extension is quite well designed and provides a pythonic interface for programmers. To install this package,

    sudo apt-get install python-nautilus

Nautilus Python Interfaces

1. nautilus.ColumnProvider – This allows your program to give additional columns (attributes) for files in the current location when the user is in List mode. Usually gets the information supplied by nautilus.InfoProvider . An example is to show length of the audio for mp3 files.
2. nautilus.InfoProvider – This file is called on all files that the user is currently viewing. You can add new attributes to the files , add new emblems or perform other tasks. An example is to provide track details about the current audio file.
3. nautilus.LocationWidgetProvider – This allows some widget to appear near the location bar. I cannot think of any practical example for this interface.
4. nautilus.MenuProvider – Allows you to provide custom menus for the selected files or folders. It is also possible to create toolbars or background menus.
5. nautilus.PropertyPageProvider – Allows you to augment existing property page (Right click -> Properties) with additional pages displaying specific information about the selected file. An example is to show MD5 of a file or show IMDB details for a movie file.

Writing Nautilus Python Extensions

You can write Nautilus extensions in Python using the Nautilus-Python extension. The scripts you write has to be placed in $HOME/.nautilus/python-extensions. If you have admin rights, you can also place them at /usr/lib/nautilus/extensions-2.0/python-extensions. Usually I prefer my scripts to be in my home folder.

The development process is very similar Nautilus scripts. The steps are :

1. Create a python file in $HOME/.nautilus/python-extensions .
2. Extend one or more of the interfaces and code the relevant functions in the interfaces.
3. Make the script file as executable.
4. Kill all instances of Nautilus using "nautilus -q" . Note that many web pages asks you to use "killall nautilus" but I prefer my apporach as it allows nautilus to die gracefully 🙂
5. Restart Nautilus. During development, using "nautilus –no-desktop" as it starts without reading profile preferences.
6. Test your extension. This will be extension specific due to the various capabilities of the nautilus python extensions.

A Closer Look at Nautilus Python Extensions

As discussed above, Nautilus Python exposes 5 interfaces. Based on your needs your class will inherit from one or more of the interfaces. Each interface will have few methods where the logic of your extension has to be included. After installation , you can take a look at /usr/share/doc/python-nautilus/examples/ for working examples. You can even copy them to $HOME/.nautilus/python-extensions and play around with them till you are comfortable. If you want to see all the functions that the interfaces expose, un-gzip the file /usr/share/doc/python-nautilus/examples/documentation.py.gz. This will produce a file documentation.py which provides the signature and basic information about the methods exposed by the interfaces. Let us take a brief look at them :

1. nautilus.ColumnProvider – It exposes a single function get_columns . This function has to return a sequence of nautilus.Column objects. As an example, the extension can return information like runtime or codec details for an mp3 file.
2. nautilus.InfoProvider – Exposes a single function update_file_info with a file as an argument. The function can update the file in various ways – for eg, it can set new custom attributes, change emblem or perform other actions on the file.
3. nautilus.LocationWidgetProvider – Exposes a single function get_widget . The function returns some gtk widget which will be displayed near the location bar.
4. nautilus.MenuProvider – Probably the most used interface. It exposes three functions : get_file_items, get_background_items and get_toolbar_items . The first two functions determine the entries that appear in the context menu. The difference is that get_background_items is usually called for a folder. The last function is used for toolbar items hence you must name the icon parameter of the menuItem.
5. nautilus.PropertyPageProvider – Exposes get_property_pages function where you reture one or more "pages" or tabs. An example is to create a new "IMDB Details" page which shows the movie information.

Debugging Nautilus Python Extensions

Debugging Nautilus Python extensions is pretty tricky. The main reason is that the "nautilus" package that is used extensively in the code is available only when the script is invoked from Nautilus. But all is still not lost. The following are some of the tips I learnt when I built my library of Nautilus extensions 🙂

Case 1 : You wrote a code for new entries in context menu but it either does not appear or works incorrectly 
1. Is your script in $HOME/.nautilus/python-extensions ?
2. Is the script has its executable bit turned on?
3. Did you restart Nautilus ? (using nautilus -q and nautilus –no-desktop )
4. Check if the file is compiled in extensions folder – basically is there a .pyc file ? If not invoke the file with python to find the syntax error.
5. Import the logging module and check if the relevant functions are called.
6. If you think the menu is being shown incorrectly, then make sure you are using the relevant functions on the FileInfo class. The class has multiple utility functions and deciding the correct one is important.

Case 2 : You wrote a new property page and it does not appear
Try steps 1-5 from Case 1.
6. Check if all the widgets are visible (ie show method is called).
7. Check if the GUI that you try to render works normally. This can be done by moving the code that returns a page to another python file and putting the HBox/VBox into a gtk.window.

Notes On Nautilus Python Extensions

A good discussion about developing the extensions is at Nautilus Extensions. The discussion is primarily for C. A good C API reference can be found at Nautilus extensions API reference. For Python, nautilus-python Reference Manual provides a basic reference.

The reference manuals per se are very skimpy in details. The best way to learn how to code nautilus python extensions is looking at existing code. There are lot of excellent python extensions that you can use for learning. The simplest is the examples provided with the python-nautilus package. The files can be accessed at /usr/share/doc/python-nautilus/examples/ for working examples. To test them copy to $HOME/.nautilus/python-extensions , restart
nautilus using commands provided above. Nautilus pyextensions contains few useful scripts. Download the source code that is available as nautilus-pyextension-<version>.tar.gz file and play around with them. One of the most powerful examples for using Nautilus Python is RabbitVCS. You can take a look yourself at the amazing RabbitVCS screenshots. You can download the source using apt-get source command.

Generic Nautilus Debugging Tips

    I found some websites that mention an alternate way to debug Nautilus extensions. It did not work for me but I will mention it here for the sake of completeness.  Create a file called nautilus-debug-log.conf in your home folder with the following lines :

    [debug log]
    max lines=1000

If Nautilus crashes for some reason, it supposedly writes into the file nautilus-debug-log.txt. If it works for you then great ! You have access to another debugging tool 🙂

Summary and Tips on Choosing the correct Approach

There are three approaches to extend or customize Nautilus. Using Nautilus actions, scripts or extensions. Each of them have their own pros and cons.

1. There is a high likelihood that some one else had a similar problem before. So before choosing one of the approaches, check if the code is available somewhere. Check the discussion (notes section) above for good resources.
2. Is your task very simple? Can it be completed with a single command line utility? Then use the Nautilus actions.
3. Try to complete your task as much as possible using Nautilus actions even if it involves some thing quick and dirty. Most of the case, you are the only person going to use it. One important thing to note is that the command that you provide in Nautilus action can also be your own script !
3. If your task primarily involves invoking multiple command line utilities , needs only basic information about the files and applicable only for local files then use Nautilus scripts.
4. If the task needs execution of multiple commands but need to be used only for certain files (eg only for txt files) , then try to write a script which invokes the appropriate commands. It should able to massage the inputs $* or $@ appropriately. Then use this script as the command to be invoked in Nautilus actions and use the filters to apply it to certain files.
5. If you need to invoke scripts even without selecting any items and the input needed is simple, then use scripts. If more complex processing is needed using extensions.
6. If you need to show the option in toolbar , then use extensions (and implement get_toolbar_items function)
7. If your task is complex – adding attributes to files, adding additional columns , adding new property pages etc use extensions.
8. Using Nautilus scripts does not mean your app is confined to text only. Use gdialog, zenity in Shell scripts for data entry and basic UI. If you use other languages, use the gtk library to develop complex UI. For eg, I have a script that queries opensubtitles.org site for subtitles and shows the result in a gtk window. Selecting one of the results downloads the subtitle file locally.
9. If you are very comfortable with Python, skip scripts and use extensions.

Sample Nautilus Python Extensions

I have a large library of actions, scripts and extensions. In this post, I want to share two of them.

1. Send file to thunderbird as attachment : The task is to select a set of files and compose a mail in Thunderbird with the selected files as attachment. Almost all the scripts / extensions that I see in the net did not work correctly and hence I wrote it myself. This extension will work for single or multiple files. It will appear only if all the selected items are files. It assumes Thunderbird is located at /usr/bin/thunderbird (View File) .

2. IMDB Details Property Page : I wrote this extension as an exercise to learn property pages. It uses the excellent IMDbPY library to fetch details from IMDB. It works only on video files and strips the common unused characters before querying IMDB. Since this was a test, I blindly select the first result and show the details. This extension also uses other gtk widgets to have a more complex UI. Assumes you have IMDbPy and pygtk installed (View file) .

I hope the article showed how to improve your productivity by customizing Nautilus to perfectly suit your work style. If you have any additional tips or ideas please post them in comments !

Read Full Post »

1. Quantum Entanglement Holds DNA Together, Say Physicists
TR summarizes it best – Speculative but potentially explosive work. I learned that Quantum theory is used for things other than Physics when I learnt about Quantum Mind.  Where is this field heading ?

2. Windows 8 Plans Leaked: Numerous Details Revealed
An old story by now but has lot of neat ideas. My favorite idea is to bring appstore inside Windows. I am not sure it will click but it is still pretty clever.

3. Python internals: adding a new statement to Python
A neat article that shows how to add Ruby’s until statement to Python. It is quite striking how easy it is.

4. A Math Problem Solver Declines a $1 Million Prize
As expected , Grigory Perelman has declinded the Clay foundation’s award.

5. Does Cantor’s Diagonalization Proof Cheat?
Another very neat article from Prof.Lipton. I liked the way he explained Diagonalization idea using games. Very cool !

6. Drone Alone: How Airliners May Lose Their Pilots
In an interesting experiment , FAA has initiated research for civil aircraft to share airspace with remotely piloted UAVs – If successful, I am sure people will use it for Cargo flights. In another interesting idea in airspace , Ryanair to sell £5 tickets for standing-room only flights .

7. iTunes accounts hacking more widespread than initially thought. The facts, and what you should do
The big news in the weekend. Looks like some of my friends too were affected.

 

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Read Full Post »

Analyzing my blog traffic is one of my favorite past times. Seeing traffic surge and strangers using my posts gives me gratification. Initially, my analysis was fairly low tech – checking WordPress stats page periodically. After doing it a few times, I realized I could do more methodically – and I can use the statistics to do some rudimentary data mining.

As you know, I have a WordPress.com blog. WordPress exposes the statistics in two ways : As a flash chart in the dashboard and as API service.

Viewing Blog Stats – Low Tech Way

If you want to take a look at your blog’s stats today , then you can do it in the dashboard. Assuming you are logged into your WordPress login, go to your blog. You will see WordPress toolbar at the top of the page. Click "My Dashboard". It will show a chart of total daily visits for the past 15 days. If you want more details, hover your mouse over "My Account" and select "Stats".

This page shows a wealth of information about your blog but it is limited to 2 days – today and yesterday. There are atleast 6 important statistics in that page.

WordPress Stats Sections

a) Views Per Day
This is probably the statistic you are most interested in. This plots the total number of visits each day for the last 30 days. This also has multiple tabs which allows you to aggregate the statistics information. For example you can view the total weekly and monthly visits . This gives a basic view about how your blog is faring.

b) Referrers
Referrers are basically web pages from which visitors reached your blog. There are different types of referrers : pingbacks, related posts, explicit linking etc. If a certain website is driving lot of visitors to your blog , it is time to notice ! Referrers allows you to get that insight. This panel shows the top 10 referrers. You can click on the "Referrers" link to get the full listing.

c) Top Posts and Pages
This shows the top 10 posts for the given day. One interesting thing is that they track home page separately. Also if you have different pages (eg About), then visits to these pages are also tracked. Again, if you want to see the number of visits to each page, you can click on the title link.

One interesting thing is that each of the posts/pages in this section also has an addition chart icon near them. Clicking on them gives the stats for that particular post alone. The new page shows a chart detailing how many visits occurred to that page in the last few weeks. Most interestingly , they also show the average visits per day for the last few weeks and months. This gives a glimpse into the lasting popularity of your post.

d) Search Engine Terms
I am not very clear what this exactly means – The instruction says "These are terms people used to find your blog." . I am not sure if this corresponds to search terms typed in search engines or in WordPress search boxes etc . Anyway, this panel gives information about the search terms that fetch visits to your blog.

e) Clicks
Clicks panel shows the list of links in your blog that your visitors clicked. In a way , you can consider it as an inverse of referrers. In this case, you act as a referrer for some other blog. This post gives some hints about the type of visitors to your blog.

f) Aggregate Statistics
There is also another panel that shows some aggregate stats. This shows the total number of views to your blog so far , number of blogs and posts , email subscribers etc.

A Better Way

Using WordPress Stats page to get your data is fairly low tech. This gives some data which can only give you an instinct of how things go. But it will not give you any deeper insights. For doing more data mining, you need data – lots of data. Fortunately, WordPress makes available a stats API which you can query to get regular data. In the rest of the post , we will talk about the API and how to use the data that you get out of it.

Using WordPress Stats API

The primary url which provides the stats is http://stats.wordpress.com/csv.php . You can click on it to see the required parameters. There are 4 important parameters to this API.

a) api_key  : api_key  is a mandatory parameter and ensures that only owner queries the website. There are three ways to get this information. This key is emailed to you at the time you created your blog. Or you can access it at My Dashboard -> Users -> Personal Settings. This will show your api key. For the truly lazy click this url .

b) blog_id : This is a number which uniquely identifies your blog. Either blog_id or blog_uri is mandatory. I generally prefer blog_id. Finding blog_id is a bit tricky. Go to the blog stats page (My Account -> Stats). Click on the title link of "Top Posts and Pages". This will open a new page which shows the statistics for last 7 days. If you look at the page’s url, it will have a parameter called blog. The value of this parameter is the blog_id . Atleast for my blog , it is a 8 digit number.

c) blog_uri : If you do not want to take all the trouble of getting blog_id , use blog_uri. This is nothing but the url of your blog (http://blah.wordpress.com).

d) table : This field identifies the exact statistic you want. One of views, postviews, referrers, searchterms, clicks. Each of these correspond to the sections of WordPress stats discussed above. If table is not specified , views is selected as the default table.

You can get more details from the stats API page given above.

Sample Python Scripts to fetch WordPress Stats

I have written a few scripts which fetch each of the WordPress stats.  One of them run every hour and gets the total number of views so far for the whole blog. The other scripts runs once a day and fetch the total clicks, search terms, referrer and top posts for that day. All of these store the data as a csv file which lends itself to analysis.

If you are interested in the scripts , the links to them are  :

1. getBlogDaysStats.py  : Fetches the total views for the whole blog at the current time. For best results run every hour.
2. getBlogReferrers.py : Fetches all the referrers to your blog.
3. getBlogPostViews.py : Fetches the number of views for individual blog posts and pages.
4. getBlogSearchTerms.py : Fetches all the search terms used to find your blog today.
5. getBlogClicks.py : Fetches the urls that people who visited your blog clicked.

How to Collect WordPress Statistics

The first step is of course to collect data periodically. I use cron to run the scripts. My crontab file looks like this :

11 * * * * /usr/bin/python scriptpath/getBlogDaysStats.py
12 0 * * * /usr/bin/python scriptpath/getBlogClicks.py
14 0 * * * /usr/bin/python scriptpath/getBlogPostViews.py
15 0 * * * /usr/bin/python scriptpath/getBlogReferrers.py
16 0 * * * /usr/bin/python scriptpath/getBlogSearchTerms.py

Basically, I run the getBlogDaysStats every hour and other scripts every day. I also run the rest of scripts at early morning so that it fetches the previous day’s data.

How to Use WordPress Statistics

If you run the scripts for few days, you will have lot of data. The amount of analysis you can make is limited only by your creativity. In this section, I will tell some of the ways I use the stats instead of giving an explicit how-to.

1. Views per day : It is collected by getBlogDaysStats.py. The most basic stuff is to chart them. This will give a glimpse of your trend – If it is static or climbing, then good news. If it is falling down it is something to worry about. I must also mention that have a more or less a plateau in your chart happens often. For eg in my blog, the charts follow a pattern – It increases for quite some time , then stays at the same level for a long time and then increases again. Also , worrying about individual day’s statistics is not a good idea. Try to aggregate them into weekly and monthly values as they give a less noisy view of your blog traffic.

Another common thing to do is to analyze per hour traffic. This can be easily derived from the output of the script. Basically, if m is the number of views at time a and n is the number of views at time b , then you received n-m views in b-a hours. I usually calculate it for every hour. This gives a *basic* idea of peak time for your blog – You can also infer your primary audience , although the interpretation is ambiguous. As an example , I get most of my traffic at night – especially between 1 AM – 9 AM. Morning time traffic is pretty weak and it picks up again in the evening. Interpreting this is hard as my blog covers a lot of topics – but if your blog is more focused you learn a lot about your visitors.

2. Referrers : This is a very useful statistic if you do some marketing for your blog. For best results, you may want to use just the domain of the url instead of the whole url for analysis. Using it you can figure out which sites drive traffic to your blog. If it is another blog, then it is a good idea to cultivate some friendship with that blog’s owner. For eg, for my blog , I found that digg drives more traffic that reddit. Also facebook drives some traffic to my blog – so I use WordPress’s facebook publicize feature. I also find that I get some traffic due to WordPress’s related posts feature which means that I must use good use of categories and tags. Your mileage may vary but I hope the basic idea is clear.

3. Individual Post Views : This is probably the most useful set of statistics. Basically , it allows you to analyze the traffic of individual posts over a period of time. I have a file which associates a blog post with extra information : For eg it stores the categories, tags, original creation date, all modification dates etc. (If you are curious , I store the information in JSON format). Once you have this information lot of analysis is possible.

a. You can figure out your audience type. If for a post, you have lot of audience in the first week and almost no audience from then on – then most likely your audience is driven by subscription. If it keeps having a regular traffic, then probably it has some useful stuff and traffic is constantly driven to it by search engines. For eg, my Biweekly links belong to the first scenario : When I publish one, lot of people visit it and then after a few days it gets practically no visits. In the other case, my post of Mean Shift gets a steady stream of views every week. If you want to sustain a good viewership, you may want to write more posts which can attract long term views.

b. If you use categories and tags wisely, you can tally the number of views per each category. This will give you an idea of the blog posts which users prefer. I noticed that my audience seems to like my Linux / Data Mining posts than other categories. So it is a good idea to write more of those posts.

c. You can kind of see a pareto effect in your blog posts. For eg, my top 10 blogs account for atleast 70% of my blog traffic. So if I could identify them correctly, I can write lesser posts but still maintain my blog traffic 😉

You can do lot more than these simple analysis but this is just a start.

4. Search Terms : This is another neat statistic. You can use it to figure out the primary way in which users access your blog. For eg, the ratio of total blog post view for a day and number of search terms for the day is quite interesting. If the ratio is high, then most of the people find your blog using search engines. In a way , this a potential transient audience whom you can convert to regular audience. If the ratio is small , then your blog gets views by referrers and regular viewers. This will assure you a steady audience , but it is slightly hard to get new people "find" your blog.

This statistic also tells you which keywords the viewers use to find my blog. You can gleam lot of interesting things from this. For eg, almost all of my search terms are 3-5 words long and usually very specific. This either means that the user is an expert and has crafted specific query. It may also mean that user rewrote the query and my blog was not found in the general query. I also use the terms to figure out if the user would have been satisfied with my blog. For eg, I know that a user searching "install matlab to 64-bit" will be satisfied while some one who searches "k means determine k" will not be. You can do either of two things : augment your blog post to add information that users are searching , or point users to resources that satisfies their query. For eg, I found lot of people reached my blog searching for how to find k. I found geomblog had couple of good posts on it and updated my blog to link to these posts. Some times, I may add a FAQ if same search query comes multiple times and if my page contains the information but is obscure. Eg : lot of people reached my blog searching for Empathy’s chat log location. My post of Empathy  had it but not in a prominent fashion. So I added a FAQ which points the answer immediately.

5. Clicks : This statistic is tangentially useful in finding out which links the user clicks. One way I use it is to gauge the "tech" level of my reader who visits my blog using search engines. I usually link to Wikipedia articles for common terms. If the user clicks on these basic terms often ,then it might mean that I write articles at a level higher that the typical user and I have to explain it better. For eg , in my post of K-Means , this was the reason I explain supervised and unsupervised learning at the start even though most people learning k-means already know it.

Other Resources for Blog Traffic

There are other locations that give some useful information about your traffic. Some of them are :

a. Google Web Master Site : This is arguably one of the most comprehensive information about your blog post’s performance in Google. You can see it a Google Webmaster Page -> Your site on web -> Search queries. It has information like impressions, click throughs, average position etc. You can download all of them in a csv file too ! Literally a gold mine for data lovers.
b. Feedburner : Even though, WordPress has a feed, I switched to FeedBurner. One of the main reason was that it gave me a csv file detailing the number of views by my subscribers.
c. Quantcast : Useful for aggregate information. It has multiple charts that detail the number of views, unique visitors etc. The Quantcast data might not be accurate as it is usually estimated – but it gives a broad gauge of your blog. It also has some statistic which says how many of your visitors are addicts , how many are pass throughs etc. Quite useful !
d. Alexa : Similar to Quantcast . I primarily use my Alexa Rank for motivation to improve my blog ranking.

Pivot Tables

I primarily use Python prompt to play with data. If you are not comfortable with programmatic tweaking, use spreadsheets to do these analysis. If you have Windows, you can do lot of powerful analysis by importing the statistics obtained by the scripts into Microsoft Excel. Excel has a neat feature called Pivot tables. It is also an advanced topic that I will not discuss here. You can do some fantastic analysis using pivots. They also give you the ability to view the same data from multiple perspectives.

In this post, I have barely scratched the surface – You can do lot of amazing analysis using WordPress Stats API . I will talk about more complex analysis in a later post. Have fun with the data !

 

 

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Read Full Post »

gEdit is one of the excellent text editors available with GNOME. Hard core Linux users mostly focus on either vim or emacs. If you are one of those people – Great ! You have found the editor of your dreams. If you are one of the new users to Linux , it will not hurt to learn to use gEdit. Probably this is the editor with least learning curve. Most people underestimate the power of gEdit – comparing it to notepad in Windows. gEdit is much much more powerful than notepad (or Wordpad for that matter).

I primarily use Vim for my coding and use gEdit for some basic stuff like taking notes, writing blog post drafts etc. This post is not a tutorial on using gEdit. Instead, I will focus on its powerful plugin architecture and recommend some plugins which I hope will , dramatically improve your productivity. I will focus on Ubuntu version of gEdit and its plugins – although most of the points are applicable to other Linux distributions.

gEdit Intro

gEdit is a lightweight editor in GNOME with lot of neat features. It is very flexible and extensible – it has an excellent plugin architecture , allows themes to customize UI and so on. gEdit is also very intuitive to use. If you have worked in any general purpose text editor, you will be immediately feel at home with gEdit.

Syntax Highlighting

Before we go to plugins, one of the coolest feature in gEdit is its syntax highlighting. gEdit can highlight syntax for huge number of languages. Most of the time it can figure it out and do the highlighting automatically. In some scenarios where it fails (eg rhtml or phtml) , you can make it highlight using "View -> Highlight Mode -> [language]" . The other option is to select the language from the status bar.

Themes

Themes are another way to customize the colors of gEdit. gEdit comes with a few themes. To play with them, go to "Edit -> Preferences -> Font & Colors" . You can select one of the color schemes that is pleasing for you and also blends with your desktop / shell background. I use a neat style called DarkMate . To install it, download the xml file and put the file in ~/.gnome2/gedit/styles . After this, restart gEdit and follow the procedure described above to change styles.

Installing and Enabling gEdit Plugins

gEdit has a powerful plugin architecture based on GTK. You can use Python to write plugins. I wrote my first plugin (mimicking vim’s autocmd filetype feature) in couple of hours. You can produce excellent plugins within hours if not days. gEdit comes bundled with some excellent plugins. A more comprehensive set can be obtained by installing the package "gedit-plugins". The command is

    sudo apt-get install gedit-plugins

This package contains lot more plugin goodies which should be enough for most users’ needs.

Sometimes some developers might put their own plugins in their websites. You can install them by downloading the file and extracting it at ~/.gnome2/gedit/plugins (If you want the plugins only for you) or at /usr/lib/gedit-2/plugins/ (If you want a system wide install).

Once you installed the plugins , the next step is to activate them. Go to "Edit -> Preferences->Plugins" . Go through the list and enable the plugins that catches your fascination. Each plugins has individual behavior and hence it is hard to explain how to play with them – Lot of plugins add a new menu  or add a menu item in "Tools" . Some add themselves to left or bottom pane. To view them use the "View" menu and select "Side Pane" (or F9) or "Bottom Pane" (ctrl+F9).

Cool Plugins

There are some plugins that I use regularly . I will try to give a brief discussion about the plugins.

Snippets

In my opinion, this is probably the most useful plugin. As the name suggests, you can use it create snippets of text or code that are triggered by some word or shortcut key. It comes preloaded with a huge list of useful snippets for all the commonly used languages.

It also has a neat placeholder feature that is useful when the placeholder has to be replaced with some actual variable or some text. For eg, C’s do while loop has two placeholders. One for the condition and other for the loop body. These can be entered by pressing tab key. Placeholders can do more powerful stuff too. For eg, in C’s for loop, there are many placeholders – like the initialization, condition, increment and loop body. Most of the time, the initialization placeholder is used in all of the other places. So this plugin allows you to say that if the user entered j as initialization variables, replace all other placeholders with j.

Another very powerful feature is its ability to give you choices for categorical variables. You can learn a lot by checking out sample snippets (eg C’s snippets). It even allows Python code within the snippets (see C’s gpl snippet).  To learn all about Snippets’ powerful features check out this site.  A simpler tutorial is here.

Lastly, this plugin allows you to give shortcut keys to each of the snippets which increases the productivity even more for keyboard junkies like me.

External Tools

This plugin allows you to execute some commands through the shell from gEdit. This is pretty flexible as it allows you to start an external command and play with its input and output. Sample scenarios are : committing the current file to SVN , tidy your html by running HTMLTidy , run make or start a nuclear war.

To configure, go to "Tools -> Manage External Tools" . You can add a new command here. It can accept a variety of inputs (from current word to current document), run any arbitrary set of commands and do variety of stuff with the output (from do nothing to replacing current document). It is not necessary that the code you enter is always a shell code. You can even enter Python code as long as you enter the she-bang line correctly. Note that, if you made the output as "Display in bottom pane" then you need to make the bottom pane visible to see it (If you are using other plugins too , then you need to watch the "Shell Output" tab of the bottom pane. Again, you can have keyboard shortcuts for executing these commands.  For more details, see this page

Class Browser

This plugin shows the list of symbols in the loaded source file (eg classes, functions etc). This works pretty well for most common languages (partly because it uses ctags which is versatile). To see it in action, load some C , C++ or Ruby file. View the left pane (View -> Side Pane) . If you have multiple plugins using the side pane, then the click on the icon looking like indented document . It will show a schematic representation of your source file.

Document Word Completion

What this plugin does is simple. It reads all the words in all the open documents and when you type , it shows potential words in a drop down. This is a big time saver for slow typers like me. Of course some people might be irritated by the constant popups – If so a simpler plugin is at this page . You have to download the three files beginning with completion and put it at ~/.gnome2/gedit/plugins .

LaTeX

I know very few people use LaTeX but in case you are one of those , gEdit has a powerful LaTeX plugin. The best way to download it in Ubuntu is to use the command :
   sudo apt-get install gedit-latex-plugin

There are other good LaTeX plugins too. Basically, it allows you to auto-complete LaTeX commands, do syntax highlighting . It has a rudimentary syntax checking which can catch simple errors like missing brackets. It also has multiple wizards for creating new LaTeX files , building and citing.

Embedded Terminal

This is another neat plugin which shows a terminal in the bottom pane. I kinda use it partly because Kate in KDE has a terminal with it. Again, you need to view the bottom pane before seeing this terminal. You can do lot of stuff with this plugin. Recently though, I realized most of the common stuff I do can be automated using the External Tools plugin and hence I no longer use it much. But it is still a very useful plugin.

Python Console

This is probably the geekiest of gEdit plugins. I mainly use it for doing some bulk operations on all the open documents. You can use it only if you knew basic Python and also gEdit’s API. But if you knew it , then this plugin is a great lifesaver. One common thing I do is a "Save All Documents" . gEdit does not have a menu option to do that . Hence I use this code snippet.

    [doc.save(True) for doc in window.get_unsaved_documents()]

Multi Edit

Another real cool plugin. The main purpose is to allow editing multiple parts of the document simultaneously. Most of the time, this scenario occurs rarely but when it does, this is a great time saver. One common thing I use is for renaming a variable. I select the declaration and all its usages – If I rename the variable, it is reflected immediately at all places. This is usually very helpful when I do Python coding. See this screen cast for all its features.

There are lot of other good plugins. Some of my favorites are :

1. Code Comment : Allows bulk commenting of text. Select a block of text and press Ctrl+M to comment and Ctrl+Shift+M to uncomment. It works for lot of common programming languages.
2. File Browser : Allows you to browse your system and open files easily. It also has basic search functionality.
3. Advanced Bookmarks : Allows you set bookmarks inside the document and navigate them efficiently.
4. Session Saver : Very useful if you use gEdit to edit lot of related files (Eg some project code). In this case, you can save all the documents as a session and retrieve them in one shot. Neat !
5. Snap Open : Allows you to effectively search and open files related to the files open currently. It internally uses find and grep. If you find it too slow, change the code to make it use locate (Assuming you regularly do updatedb).

Developing New gEdit Plugins

Using gEdit plugins is only half the fun. Most of the time, you will have unique requirements on your own. For eg, I wanted gEdit to enter some text automatically when I open certain types of files. Eg : fill out all the common includes and empty main function whenever I open a new C++ file. There is a high chance that some body faced the issue and wrote a plugin – If not do not worry. Writing a gEdit plugin is a snap – if you know Python. gEdit has a excellent documentation and tutorial for the same. See the official How-To.

References

1. gEdit Plugins : The official plugin listing page is here.
2. gEdit Plugin Development : The official tutorial is here.
3. The plugins can found at either ~/.gnome2/gedit/plugins or at /usr/lib/gedit-2/plugins/ .

 

If you liked this post , please subscribe to the RSS feed .

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Read Full Post »

My Experiments With Hindi Part I

I had always wanted to learn Hindi although it was usually a low priority task. Hindi’s significance probably increases when you go to some foreign country like US. Most Indians here  know Hindi and the subtle pressure to learn Hindi increases. There is also the usual BS about it being the national language.

So finally I decided to learn Hindi . I had made some half hearted measures before and had abandoned my Hindi learning within weeks. So I was determined not to do it again. So here are the various steps I took to learn Hindi.

Ubuntu Changes

I realized that I will be using Hindi a lot – for eg reading articles or writing some basic text. So I added Hindi support to my OS (Ubuntu  – Karmic) . I am sure the steps given here will be  applicable to older Ubuntu versions too . Again , for other versions of Linux /Windows / Mac OS the steps can found in net.

Enabling Hindi Language Support :

1. System –> Administration –> Language Support.

2. Select Install / Remove Languages

3. Select Hindi from the list of languages.

4. Ubuntu will install necessary Hindi support for Ubuntu.

 

Toggling between Keyboards :

If you are like me , then you will primarily working in English and occasionally toggling to Hindi. To do that :

1. System –> Preference –> Keyboard

2. Select Hindi

3. Click on “Layout Option” button.

4. In the new window, select the option “Key(s) to change layout” . You can select any combination of keys to toggle between languages. I settled on Shift+Caps Lock.

 

Keyboard Indicator :

It is usually very useful to know which language keyboard is being used currently. Sometimes the reason the system is not accepting the password might be because, you were typing it in Hindi !! That has bitten me quite some time. If you are in Ubuntu, it is very easy to fix it.

On any GNOME panel right click and select “Keyboard Indicator” applet. It gives concise information about the current application’s keyboard. In Linux, each application can use one of keyboards simultaneously. For eg I can type in Vim in Hindi and use English in Firefox with no problems.

Hindi Fonts :

There are lot of free fonts from multiple places for Hindi. Do a simple Google search to get them.

Typing In Hindi :

The input methods (IME) in Ubuntu , does not really use a transliteration based typing. The mechanism used took some time to get used and after some time became very natural.

a) Consonants are typed with the appropriate letter for the sound. ie you type k to get क (not ka) . To get ख, you type K (k in caps).

b) The vowels are used to get the various combinations –

ka gives  का 
ke gives के 

kE gives कै
ki gives कि

kI gives की
ko gives को

kO gives कौ
ku gives कु

kU gives कू

 

Books And Courses

1. I looked around and finally settled on Teach Yourself Hindi by Rupert Snell as the book to follow, partly due to SOAS’s recommendation. It can be got at this Amazon link.

2. I found some good courses and course materials at UPenn, NCSU , Yale etc. More can be found by Googling. Take a look !!

Good Hindi Websites

I have not done extensive research into all good Hindi learning sites , I found some very useful ones.

1. ISpeakHindi – I Speak Hindi was probably the most comprehensive of all of the sites. They also have a very useful  podcast series for learning Hindi.  I try to learn a podcast or two everyday. Their wiki page has lot of useful information. It has a good classification of lessons from which you can sample.

2.  News Sites : There are lot of good Hindi news sites which gives news in reasonably simple Hindi. The most prominent of them is BBC Hindi site . Their Hindi site for Indian news is also good.   Of course, to get more authentic , Indian Hindi News from Indian newspapers see Google News Hindi site.

3. LiveMocha : I was told LiveMocha was useful for learning other languages although I did not find it very useful.

 

Other Tools

I also wrote some tools to help make my Hindi learning easier.

1. Hindi Translator : This is a Python script which takes a file and source,destination languages and converts the words in the file (which is in source language) to destination file using Google Translate API.  It can found here.

2. Flash Cards : This is a Flashcard like application which I used to quiz myself on the Hindi/English words. It expects all the files to be put in some folder (for eg one per chapter) . It now allows you to select one /all of the files for quizzing you. It allows both English to Hindi and Hindi To English. Unfortunately , I wrote the code in PythonCard, so I doubt it working in non Linux OS. It is also the code makes lot of assumptions , so might need some changes before being reusable. The code can be obtained here.

3. GNOME Hindi Scroller Applet : This is a simple GNOME applet which when added to a panel, keeps showing some Hindi word and its meaning every 10 seconds. It accepts its input from a file . I had written about this more at this blog post . The code can be obtained here .

 

I will keep adding more posts in this series as I learn more Hindi !!

Read Full Post »

Using LaTeX in WordPress

LaTeX ,for people, who have not used it , is an awesome and a must learn tool . Quoting from its home page , “LaTeX is a high-quality typesetting system; it includes features designed for the production of technical and scientific documentation. LaTeX is the de facto standard for the communication and publication of scientific documents.” Most of the papers in CS, Math and Physics communities use LaTeX in one form or the other.

I have been using LaTeX for almost an year, primarily as a way to make notes when I listen to video lectures or read some technical book. I intend to write (hopefully) many articles , mathematical in nature which will need LaTeX. So I have been exploring ways to add mathematical content in WordPress blog.

Luckily, WordPress supports LaTex and some of its most popular ams packages. You can get more information here. In summary , you create latex text by putting your content within a special markup called latex .

$latex all the LaTeX code here $

For more information look here.

I guess this will be sufficient for most of my needs. There are some inconveniences using LaTeX in WordPress . For a discussion see here. For more power users, Luca has written a Python tool to convert a LaTeX file to a form ready to be copy pasted to WordPress. It can be downloaded here. I have not used it myself per se, but I intend to use it a lot in the near future. Given the fact that it is used both by Luca and Terry Tao, I am sure that it will satisfy all my needs and more.

So get your LaTeX rolling !!

Read Full Post »

Older Posts »